VII Patterns

Mirroring a Provider’s Object

The shop sells a membership: a monthly payment that gives free delivery. The membership lives at the payment provider. It is created there, renewed there every month, cancelled there — by the shop, or by the customer on the provider’s own page — and it lapses there when a card fails. The shop needs to know, for every order, whether the customer is a member right now, without asking the provider each time.

So the shop keeps a copy and updates it from the provider’s notifications. The naive way is to apply what each notification carries: “renewed until June 1” sets expires, “cancelled” sets cancels. It fails without any component being down:

  • Notifications arrive in any order. The customer cancels and changes their mind within a minute. “Resumed” arrives first, “cancelled” second, and the shop’s copy says cancelled for a membership that is alive.
  • There is often nothing to order them by. Many providers stamp notifications in whole seconds and give the object no version.
  • A notification can be missed, and from then on the copy is wrong until the next one happens to carry the same field.

§A provider’s notification is a signal to read the provider

Do not apply what a notification says. Take from it only which object changed, and whose it is, and read that object from the provider. Whatever order the signals come in, the last read is the current truth.

This is for an object another company keeps, whose notifications have no order and no version. The application’s own events have both, and a client of the application does the opposite: it applies an event and reads nothing for it. See Keeping a copy of a collection.

provider ── POST ──> payments.webhooks.ingest ── call ──┐
the customer returns from the payment page ── call ──┼──> billing.memberships.signal
the shop itself changed it at the provider ── call ──┘      │ under the customer's id
                                                            ├─ call ──> payments.gateway.membership
                                                            │           (reads the provider)
                                                            ~ billing.memberships.sync

§The mirror

# components/billing.memberships/manifest.toa.yaml
name: memberships
namespace: billing

entity:
  associated: true
  properties:
    membership: { type: string }   # the provider's id
    plan: { type: string }
    expires: { type: integer }     # when the paid period ends, ms
    cancels: { type: integer }     # when a requested cancellation takes effect, ms
    timestamp: { type: integer }   # when it was read from the provider, ms

operations:
  signal:
    concurrency: retry
    query: true
    errors: [UNKNOWN_MEMBERSHIP, UNAVAILABLE]
    input:
      type: object
      properties:
        membership: { type: string }
      required: [membership]
// components/billing.memberships/operations/signal.ts
import type { Context, SignalInput, State } from '../types/index.d.ts'

export async function transition (input: SignalInput, entry: State, context: Context) {
  const read = await context.remote.payments.gateway.membership({
    input: { membership: input.membership }
  })

  if (read instanceof Error) return read

  if (read.metadata?.customer !== entry.id) return new Error('UNKNOWN_MEMBERSHIP')

  entry.membership = read.id
  entry.plan = read.plan
  entry.expires = read.period_end * 1000
  entry.timestamp = context.now()

  if (typeof read.cancel_at === 'number') entry.cancels = read.cancel_at * 1000
  else delete entry.cancels
}

The entity is associated, and its id is the customer’s: signal is called with query: { id: <the customer's id> }. Every signal about a customer’s membership reaches one object, the first one creates it, and a customer who never subscribed reads as an object with no expires (Keep Data About a Customer without Creating It). Nothing is required, so the blank is that empty object.

Whose membership it is, is decided by the provider’s answer and not by the caller: the customer’s id was put into the membership’s metadata when it was created (the gateway does that for a checkout), and a membership that names another customer, or none, is refused as UNKNOWN_MEMBERSHIP.

The mirror is “the customer’s membership as the provider held it at timestamp“. signal only sets values, so running it twice ends the same. payments.gateway.membership is one more operation of the gateway, declared with errors: [UNKNOWN_MEMBERSHIP, UNAVAILABLE].

§Who signals

The webhook, before it answers the provider. Its ingest is the one of Receiving Webhooks, which stores a notification the gateway has decoded; it gains this:

// components/payments.webhooks/operations/ingest.ts — after the notification is decoded
if (notification.type.startsWith('membership.')) {
  const customer = notification.data.metadata?.customer

  if (customer !== undefined) {
    const signalled = await context.remote.billing.memberships.signal({
      query: { id: customer },
      input: { membership: notification.data.id }
    })

    if (signalled instanceof Error && signalled.code === 'UNAVAILABLE') return signalled
  }

  entry.membership = notification.data
}

ingest declares errors: [DECODE, UNAVAILABLE], and the entity gains membership. If the provider cannot be read at this moment, the notification is refused with UNAVAILABLE, the provider gets 422, and it sends the notification again later — which is exactly what a failed read needs. A membership that names no customer has no mirror to signal, and the other refusal, UNKNOWN_MEMBERSHIP, would be the same next time, so in both cases the notification is stored and answered with a success.

The customer’s return from the payment page. The browser comes back with the id of the checkout; the shop asks the gateway for the checkout, checks that it is the caller’s, and signals the caller’s membership. The customer sees the membership at once instead of waiting for the webhook. The same route for a top-up is shown in Taking a Payment, End to End.

The shop’s own change. When the customer cancels in the shop, the shop changes the membership at the provider and reads it back in the same breath:

// components/billing.memberships/operations/cancel.ts
import type { CancelInput, Context } from '../types/index.d.ts'

export async function effect (input: CancelInput, context: Context) {
  const current = await context.local.observe({ query: { id: input.customer } })

  if (current.membership === undefined) return new Error('NO_MEMBERSHIP')

  const cancelled = await context.remote.payments.gateway.cancel({
    input: { membership: current.membership }
  })

  if (cancelled instanceof Error) return cancelled

  await context.local.signal({
    query: { id: input.customer },
    input: { membership: current.membership }
  })
}

cancel is declared an effect with customer as its input and errors: [NO_MEMBERSHIP, UNKNOWN_MEMBERSHIP, UNAVAILABLE]; context.local calls an operation of the same component, and payments.gateway.cancel is one more operation of the gateway. If the last read fails, nothing is lost: the provider’s notification about the cancellation follows, and signals again.

§Telling the rest of the shop

The mirror is under the customer’s id, so whoever has to know whether a customer is a member asks for it by the id it already has, and compares:

const membership = await context.remote.billing.memberships.observe({ query: { id: customer } })
const member = membership.expires !== undefined && membership.expires > context.now()

Nothing has to run when a membership lapses. The date passes, and the comparison turns false (Work on the Clock).

Whoever cannot ask — a page that shows the plan, a component that selects by it — follows the mirror’s event. The mirror announces a change only when something a reader would see changed:

// components/billing.memberships/events/sync.ts
import type { Event } from '@toa.io/core/types'
import type { Entity } from '../types/index.d.ts'

const SEEN = ['plan', 'expires', 'cancels'] as const

export function condition (event: Event<Entity>) {
  return SEEN.some((property) => event.origin?.[property] !== event.state[property])
}

An event file named sync replaces the inherited one. A read that found nothing new moves timestamp and publishes nothing.

The shop’s own events can arrive out of order too. Here there is something to order them by: timestamp, the moment the provider was read. A copy kept from sync stores it and refuses a read older than the one it has. Keeping copies in step is the subject of Ask the Owner, or Keep a Copy, and ordering of Messages Out of Order.

§Why read the provider instead of applying its notification

Applying needs an order, and the outside world rarely gives one you can trust. Reading needs none.

When something that lives elsewhere changes, treat every message about it as “look again”, and copy what you find. Apply the content of a message only if its sender gives every state of the object a number that only grows.

The cost is one request to the provider per notification. If the provider does number the states of an object, carrying that number and refusing older ones is cheaper; Messages Out of Order shows that form.

The call from ingest to signal, rather than an event, follows from who can try again. If the read fails inside a receiver, the shop would have to repeat it itself, and the provider — the only party that knows something changed — has already been told all is well. Done by a call before the reply, a failed read is handed back to the provider, which repeats the notification for days.

§What can still go wrong

Two signals for one membership at once. Both signal runs read the provider and try to write the same object. One wins. concurrency: retry runs the other again from the start, so it reads the provider again and writes what is true after the first (Two Writers at Once). Running again is harmless here because the call inside signal only reads; a transition that changes something elsewhere and may run again needs more care.

The same signal twice. The provider is read twice and the same values are set twice. If nothing a reader would see changed, no sync follows.

Signals out of order. There is no order to respect: each one reads the present.

The provider is down. From the webhook: refused, and repeated by the provider. From the customer’s return or the shop’s own change: the caller is told UNAVAILABLE, and the webhook about the same change follows anyway.

A notification never arrives. The copy stays as it was read last. For a membership that is bounded: expires passes, the shop stops treating the customer as a member, and the next signal of any kind — a renewal, a return to the shop — reads the truth. Where staleness must be bounded more tightly, a pulse that signals every membership whose expires is near or just past does it.

sync reaches a copy late, or twice, or after a newer one. Late: the copy is behind for a moment. Twice: the same values are set. After a newer one: it is refused by its timestamp.

The mirror is not the cash register. Reading a membership grants nothing. Money a membership brings in arrives as paid invoices, and each is recorded once under the provider’s id of the invoice, exactly like a top-up in Receiving Webhooks. Keep “what state is it in” and “what was paid” on separate paths.

§Rules of thumb

  • One object per provider object, associated, under the id of the customer it belongs to.
  • A message about it carries one useful thing: its id. Read the rest.
  • Read and write in one transition with concurrency: retry; only set values.
  • Record when you read, and let copies refuse anything older.
  • Publish to the rest of the application only when something they keep has changed.
  • Signal by a call where the sender can retry and you cannot; refuse only for “ask again later”.
  • After changing the object at the provider yourself, signal at once and let the webhook cover a failure.
  • An expiry is a date to compare with, not an event to wait for.
Search
Search the documentation