Realtime

A client that shows live data needs to hear about changes as they happen. Polling for them is wasteful and late. The alternative is for the application to push changes to the clients that care.

The application already knows when its state changes: every commit produces events. Realtime is the declaration of which of those events reach which clients.

§Keys

Events are routed to keys, and a client reads the stream of a key. A key is any value that names an audience: a chat room, a project, the identifier of a user.

# messages/manifest.toa.yaml
realtime:
  created:
    key: room
    expose: [id, sender, text]

When a message is created, the event goes to the stream of the key held in its room property. Everyone reading that room’s stream receives it.

A key can be taken from a property that holds several values, and a route can name several keys. A message addressed to a sender and a recipient goes to both of their streams.

§What a stream exposes

expose lists the properties of the event that the stream carries. It is required. A stream is read by clients outside the application, and it shows nothing the route did not name.

This is the same closed-by-default rule as the output of a resource. An event is an internal contract between components; what leaves the application is a deliberate subset.

§Serving a stream

A stream is served at a route like any other resource:

exposition:
  /rooms/:room/stream:
    auth:role: moderator
    GET:
      realtime:stream: room

The route says which key the stream is for, here the value of :room. Whatever authorizes the route authorizes the stream. There is no second permission model for realtime; reading a stream is using a resource.

An identity’s own identifier is a key like any other, and the gateway serves each identity a stream of its own without the application declaring it. Events routed to a user’s identifier, by any component, arrive on that one stream.

§One stream per interest

A client opens a stream per key it is interested in. Where that would be many — every task on a board, every item in a list — the application gives them a common key. The event carries the board it belongs to, and the board is the key.

Choosing keys is therefore a modeling decision. A key is an audience, and the application’s entities usually already say what the audiences are.

§What a reader can rely on

An event reaches every open stream of its key, whichever replica of the gateway serves the reader.

A reader can reconnect without loss. The stream gives the reader a token after every event. Reconnecting with the last token resumes from that point. Events for a key are kept for a short time after its last reader leaves, which covers a dropped connection but not a long absence.

An event may arrive twice, and in any order. Realtime inherits the guarantees of the events it carries. Every entity carries its VERSION, and a reader keeps the version it has when an older one arrives.

A stream starts when it is opened. What was routed to a key before that is not replayed. A client opens the stream first and then reads the current state, so that nothing falls between the two.

Access is checked at opening. A stream that is already open stays open if its reader loses access afterwards.

Nobody listening costs nothing. An event routed to a key that has no reader is not written anywhere.

§State that is not committed

Some state changes many times a second and is superseded each time: where a cursor is, where a shape is being dragged. An operation committing each change would store what nobody reads again, and a replay would hand a returning reader a history of positions it has no use for. So a route may push what a client sends straight to the readers of a key, without an operation.

A pushed event is volatile. It reaches whoever reads the key when it is pushed, and nobody else: it is not replayed, and a reader that falls behind loses it rather than receiving it late. It names its sender. The gateway stamps it with the identity that pushed it, so a reader knows whose state it is. It is what the route admits, by a schema the route declares.

What has to survive — a shape drawn, a comment — stays a change an operation commits and routes to the same key, and a reader is kept current by both.

§Realtime and collection streams

Realtime and collection streams both keep a client current, and they complement each other.

A collection stream is complete: it accounts for every change to the selected entities, including those made by means other than operations, and a reader can return after any absence within the storage’s memory. A realtime stream is immediate: it delivers what operations announce, as they announce it, to whoever is connected.

A client loads its copy with a collection stream and keeps it by the events of a realtime stream. An event carries the change and is applied to the copy; it is never a reason to read. The collection stream is read again only by a client that was away: one that starts, or that returns after its realtime stream no longer holds what it missed.