Do Something Later

An order that is not paid within an hour is cancelled. Each order has its own hour, so this is not a schedule: the order arms a delay, one call handed over to be made later. It is stored, so it survives restarts and deployments.

§Arm a delay

A component that delays calls names the cadence extension in its manifest. There is nothing to declare under it; a component that declares pulses or schedules there has delays too.

The orders are those of store.orders, the component Create a Component writes. Add the extension, a property that keeps the id of the delayed call, and the operation to call later:

# components/store.orders/manifest.toa.yaml
cadence: ~

entity:
  properties:
    # …
    deadline: { type: string }

operations:
  # …
  lapse:
    type: transition
    scope: entry
    concurrency: retry
    query: true
    errors: [NOT_PENDING]

That puts delay on the operation context. create, the transition Add Configuration writes, arms the delay before it returns:

// components/store.orders/operations/create.ts
import type { Context, CreateInput, State } from '../types/index.d.ts'

const MINUTE = 60 * 1000

export async function transition (input: CreateInput, entry: State, context: Context) {
  if (input.total > context.configuration.limit)
    return new Error('LIMIT_EXCEEDED')

  Object.assign(entry, input)

  entry.deadline = await context.delay(
    'store.orders.lapse',
    { query: { id: entry.id } },
    { interval: 60 * MINUTE, overdue: 10 * MINUTE }
  )

  return entry
}

// components/store.orders/operations/lapse.ts
import type { State } from '../types/index.d.ts'

export async function transition (input: null, entry: State) {
  if (entry.status !== 'pending') return new Error('NOT_PENDING')

  entry.status = 'cancelled'

  return entry
}

An hour later store.orders.lapse is called with that query, as if a component had called it then.

context.delay takes three arguments:

1. The endpoint, <namespace>.<component>.<operation>: an operation of any component, the caller’s own included. A component without a namespace is default.<name>.<operation>.
2. The request, what an ordinary call would pass: input, query, or both. Pass null for an operation that takes nothing.
3. The options. interval and overdue are both required.

A delay is stated in milliseconds; the manifest’s cycle and a schedule’s overdue are in seconds.

§Cancel it

context.delay answers an id, and the id cancels the call. approve, the transition Create a Component writes, cancels the one its order armed:

// components/store.orders/operations/approve.ts
import type { Context, State } from '../types/index.d.ts'

export async function transition (input: null, entry: State, context: Context) {
  if (entry.status !== 'pending')
    return new Error('NOT_PENDING')

  await context.delay.cancel(entry.deadline)

  entry.status = 'approved'

  return entry
}

cancel throws for an id that was never issued.

Do not rely on cancellation alone. It is reliable only while the call is more than a minute away (see Timing); closer than that the call may already be on its way. That is why lapse checks the status it finds.

§Start the service that keeps the calls

Restart and create an order, and the call does not answer:

$ npm run restart
$ npx toa call store.orders.create "{ input: { customer: 'c1', items: [], total: 1800 } }"

Nothing is printed, and nothing is logged. context.delay hands the call to a service of Toa, cadence, which stores it and makes it when it is due, and the application toa create wrote does not run that service: the operation waits for an answer nobody gives. An extension that is declared while its service is not running fails silently, here as a call that never returns.

sys is the process that runs what Toa serves beside the application. Add the service to its line in ecosystem.config.js, and restart:

// ecosystem.config.js
{ ...common, name: 'sys', args: 'serve configuration exposition introspection cadence' },

$ npm run restart

The opposite mistake is reported. A service named on that line while no component declares cadence stops sys when it starts:

Error: TOA_MONGODB_CADENCE_METRONOME is not set.

npm run env writes that variable only once a component declares the extension. Declare first, then name the service.

§Try it

Lower the interval to a couple of minutes for the experiment, restart, create an order, and observe it after the interval has passed:

$ npm run restart
$ npx toa call store.orders.create "{ input: { customer: 'c1', items: [], total: 1800 } }"
$ npx toa call store.orders.observe "{ query: { id: '<the id it printed>' } }"

The order is cancelled. Create another, approve it in time, and it stays approved.

In a test, call lapse directly instead of waiting: what matters is what it does with a pending order and with an approved one.

§Choose overdue

A call is normally made at its moment. It is late when something failed: the target was away, or nothing was running to make the call. overdue says what to do then.

There is no default. Only the caller knows whether a late call is still the right call: a payment reminder is useful whenever it arrives, an order that expires at noon expires on time or not at all. A call past its bound is dropped silently.

§Timing

Stored calls are looked over on a period called discreteness: 60 seconds unless the Context says otherwise.

• A delay shorter than the discreteness is served late, up to a minute by default. A longer one is made on time.
• An overdue shorter than the discreteness can be missed when nothing is wrong. State an overdue several times the discreteness, or null.
• A cancellation closer than the discreteness to the call is a race.
• One replica takes at most 200 calls per pass, oldest first. When more come due at once, the rest are made on a later pass, and it is logged.

A call that must be made, or cancelled, within seconds of its time needs a discreteness of seconds.

§What can go wrong

The call can be made more than once. Where a second one would cost something, the target refuses it, as lapse does; see Handle Repeated Calls.

The call can be not made at all:

• The caller stopped at the wrong moment. Arming is not part of the operation’s commit, so an operation that stops between the two leaves one without the other. Where it fits, arm first, as create does: a call about an order that was never stored is better than an order nobody cancels.
• It passed its overdue.
• It was cancelled.
• It could not be handed to its target. One attempt is made; a failure is logged and the call is dropped.

Once the call is handed over, it is handled like a task: it waits for a target that is not running, an exception has it tried again, and a returned error ends it.

A delay of years is accepted, but by then the operation may be gone or may no longer accept that input.

§A call that repeats itself

A delayed call is made once. Work that repeats per item — remind about this invoice every day until it is paid — arms the next call as it finishes this one. Such a call must say unchained: true, or the third round is refused as a loop:

// components/notifications/operations/chase.ts
import type { ChaseInput, Context } from '../types/index.d.ts'

const DAY = 24 * 3600 * 1000

export async function effect (input: ChaseInput, context: Context) {
  // ... remind about the unpaid invoice

  await context.delay(
    'notifications.chase',
    { input },
    { interval: DAY, overdue: null, unchained: true }
  )
}

An operation declared unchained: true in the manifest needs no option:

# components/notifications/manifest.toa.yaml
operations:
  chase:
    unchained: true

Stopping is the operation’s to write, and one round that is not made — expired, refused, lost in a crash before the next was armed — stops it for good. Where the item’s work recurs on a calendar, hand a schedule over instead. Where the same work is due for everything at once, a pulse is simpler.

§Where a delay is refused

§What it needs to run

Redis, named in the Context, as for schedules. The Context toa create wrote names the one npm run dock starts; add the one of a deployment:

# context.toa.yaml
atomicity:
  redis: redis://localhost
  redis@production: redis://redis.example.com

The cadence service. It is deployed with the application without anything declared, and keeps the calls in the application’s MongoDB. Locally it is added to sys: above. Where the processes are started by hand, toa mono starts it; with toa compose, name it or run it beside:

$ npx toa compose ./components/* --service cadence
$ npx toa serve cadence

Without either, no delayed call is made.

§Context settings

# context.toa.yaml
cadence:
  discreteness: 60 # seconds between passes over the stored calls
  regions: [0] # the regions whose calls this deployment makes, by rank

In an application deployed as several regions, each region makes its own delayed calls and there is nothing to configure. When a region is lost, redeploy a surviving one naming the lost rank beside its own:

# context.toa.yaml
cadence:
  regions: [0, 1] # this region's calls, and those of the region that was 1

It then makes both regions’ delayed calls, and the pulses and schedules declared for the lost one, and keeps the delayed calls it took over. Two regions naming one rank make every one of its calls twice, and nothing detects it. A cancellation made in another region arrives more slowly than one made in the same region.

toa types adds delay to the component’s generated Context type; see Use TypeScript. The concept is in Cadence.