Keep Something in a Process

The mail provider the shop sends its emails through offers a client that keeps a connection open, and opening it for every email is wasteful. Operations keep nothing between calls, so open it once when the component starts, keep it in the memory of the process, and close it when the component stops.

§Open it when the component starts

A module in the component’s rc directory exports run commands: functions called at fixed moments of the component’s life, outside any operation.

// components/notifications/rc/mailer.ts
import type { Context } from '../types/index.d.ts'

export async function preflight (context: Context) {
  context.state.mailer = await connect(context.configuration)
}

export async function dispose (context: Context) {
  await context.state.mailer.close()
}

notifications is the component Call an External Service writes, with the provider’s address and key in its configuration; connect stands for the provider’s client.

Each receives the operation context. The file name is yours to choose, and a component may have several modules. A module exports at least one command.

§Keep it in context.state

context.state is a plain object, empty when the component starts in a process, shared by its run commands and operations. Declare the extension to have it:

# components/notifications/manifest.toa.yaml
state: ~

send, the component’s effect, then uses the connection instead of making a request:

// components/notifications/operations/send.ts
import type { Context, SendInput } from '../types/index.d.ts'

export async function effect (input: SendInput, context: Context) {
  await context.state.mailer.send(input.to, input.text)
}

Nothing is configured in the Context. Values are kept as they are: objects, class instances, a Map, a Set.

An operation can also fill it on first use, without a run command:

// components/store.catalog/operations/price.ts
import type { Context, PriceInput } from '../types/index.d.ts'

export async function computation (input: PriceInput, context: Context) {
  context.state.prices ??= new Map()

  if (!context.state.prices.has(input.product))
    context.state.prices.set(input.product, await lookup(input.product))

  return context.state.prices.get(input.product)
}

It is the memory of one process, not state:

• every replica has its own, and two calls may land on different replicas;
• it is gone when the process stops, and a deployment replaces every process;
• all operations of the component in the process share it and run concurrently, so keep no per-call data in it.

What has to survive belongs in the entity. What every replica should see belongs in the stash.

§Try it

$ npm run restart
$ npx toa call notifications.send "{ input: { to: 'ann@example.com', text: 'Hello' } }"

The connection is opened once, before Composition complete is printed to the log of app (pm2 logs app), and every send uses it. Stop the application with npm stop: dispose runs, and the process exits.

§Release what you opened

dispose is the counterpart of preflight. During it the component can still call other components, but nothing calls into it any more, so do not call its own operations.

Anything left open holds the process. A composition exits when its last handle is released: a timer, a socket or a stream that dispose did not close keeps it alive, in production and in tests alike. If toa compose or a test run does not exit, look for what a run command or an operation opened and did not close.

§The run commands

settle and ready have no counterpart on stop.

§Stop your own timers on pause

Before a deployment is halted, every process is told to stop doing anything of its own accord. Whatever a component started itself — an interval, a watcher, a subscription to something outside — it stops in pause:

// components/store.catalog/rc/poller.ts
import type { Context } from '../types/index.d.ts'

export function preflight (context: Context) {
  context.state.poller = setInterval(() => poll(context), 1000)
}

export function pause (context: Context) {
  clearInterval(context.state.poller)
}

export const resume = preflight

export const dispose = pause

After pause, one of two things happens:

• The process is stopped and later started anew, and preflight runs.
• The halt is called off. The component is the same one, with the same context.state, and only resume runs.

So resume takes again whatever pause released. A component that exports pause and no resume does not start.

A component that keeps a timer running through pause goes on making calls while the deployment is supposed to be quiet, and that calls the halt off.

In a test, stage.halt() and stage.restore() run pause and resume; see Write a Test.

§Call one particular process

Sometimes a thing exists in exactly one process, and a call has to reach that process and no other. terminals drives card terminals at the tills. A terminal keeps one connection open, in the process that opened it, and charging a card has to happen there.

Declare the operation stateful:

# components/terminals/manifest.toa.yaml
name: terminals

state: ~

entity:
  properties:
    holder: { type: string }

operations:
  charge:
    type: computation
    stateful: true
    input:
      type: object
      properties:
        terminal: { type: string }
        amount: { type: integer }
      required: [terminal, amount]
    errors: [NOT_HELD, DECLINED]
  connect:
    input:
      type: object
      properties:
        terminal: { type: string }
        address: { type: string }
      required: [terminal, address]
  assign:
    input:
      type: object
      properties:
        holder: { type: string }
      additionalProperties: false

Every process serves a stateful operation under its own name, which it reads as context.instance. Record the name where a caller will find it, usually in the entity:

// components/terminals/rc/connections.ts
import type { Context } from '../types/index.d.ts'

export function preflight (context: Context) {
  context.state.connections = new Map()
}

export async function dispose (context: Context) {
  for (const connection of context.state.connections.values()) await connection.close()
}

// components/terminals/operations/connect.ts
import type { ConnectInput, Context } from '../types/index.d.ts'

export async function effect (input: ConnectInput, context: Context) {
  context.state.connections.set(input.terminal, await open(input.address))

  await context.local.assign({
    query: { id: input.terminal },
    input: { holder: context.instance }
  })
}

connect is an ordinary operation: whichever process takes the call opens the connection and records itself as the terminal’s holder. It does so with the inherited assign, which sets the properties the manifest declares as its input; open stands for the terminal’s driver. The stateful operation works with what its process holds:

// components/terminals/operations/charge.ts
import type { ChargeInput, Context } from '../types/index.d.ts'

export async function computation (input: ChargeInput, context: Context) {
  const connection = context.state.connections.get(input.terminal)

  if (connection === undefined) return new Error('NOT_HELD')

  // ...
}

A caller names the process in instance, beside input. Here it is pay, an effect of store.orders:

// components/store.orders/operations/pay.ts
import type { Context, PayInput } from '../types/index.d.ts'

const ADDRESSEE = 403
const ABANDONED = 404

export async function effect (input: PayInput, context: Context) {
  const terminal = await context.remote.terminals.observe({
    query: { id: input.terminal }
  })

  try {
    return await context.remote.terminals.charge({
      input: { terminal: terminal.id, amount: input.amount },
      instance: terminal.holder
    })
  } catch (exception) {
    if (exception.code === ADDRESSEE) return new Error('TERMINAL_OFFLINE')
    if (exception.code === ABANDONED) return new Error('TERMINAL_UNCERTAIN')

    throw exception
  }
}

A component calls its own stateful operation the same way, through context.local.

§A call to a process can fail

An ordinary call waits for its component. A call to one process has nobody else to answer it, so write its caller to expect two exceptions, told apart by exception.code:

After Addressee it is safe to recover: here, to have the terminal connected again by another process. After Abandoned the caller does not know whether the card was charged, and has to find out before trying again.

The process named may also be alive and no longer hold the thing: it was restarted under the same name, or let the connection go. That is the operation’s own answer, here the error NOT_HELD.

§How long the caller waits

Five seconds, unless the call says otherwise in a second argument:

await context.remote.terminals.charge(
  { input: { terminal, amount }, instance },
  { timeout: 60_000 }
)

await context.remote.terminals.charge(
  { input: { terminal, amount }, instance },
  { signal: AbortSignal.timeout(10_000) }
)

When either ends the wait, the call throws Abandoned. A signal already aborted when the call is made, or aborted while the component is still being looked up, means the call is made nowhere. For a streamed reply the wait covers the start of the stream, not its length.

A call not yet taken when its timeout passes never runs. A call whose signal ended the wait earlier stays in line until its timeout, and may still run.

The default is set for the whole application:

# context.toa.yaml
addressed:
  timeout: 5000 # milliseconds

§Calls that are refused

These throw RequestContract before anything is sent:

• a call to a stateful operation that names no instance;
• a call to an ordinary operation that names one;
• a task or a delayed call that names one;
• a call to an ordinary operation that states timeout or signal.

A receiver may be bound to a stateful operation: its adaptation names the process in the request it returns. Here the event’s payload carries the terminal and its holder:

// components/terminals/receivers/store.orders.cancelled.ts
export const request = (payload: { terminal: string, holder: string }) => ({
  input: { terminal: payload.terminal },
  instance: payload.holder
})

§The name of a process

A name is generated when the process starts. To choose it, set TOA_INSTANCE:

$ TOA_INSTANCE=till-7 npx toa compose ./components/terminals

A process can be called by its name only once it serves its operations. A process that hands its name out as it starts does so in ready, not in preflight or settle; a call made earlier may throw Addressee.

// components/terminals/rc/register.ts
import type { Context } from '../types/index.d.ts'

export async function ready (context: Context) {
  await context.remote.store.tills.register({ input: { process: context.instance } })
}

§Keep it small

• Keep durable facts in the entity. Only charge is stateful; connect, observe and enumerate are ordinary operations.
• Record the holder, and clear it in dispose, while the process can still call other components.
• Notice a holder that vanished. A process that crashed clears nothing. A pulse that probes the recorded holders with a cheap stateful operation, and treats Addressee as “gone”, lets the rest recover.

§Limits

• While a process reconnects to the broker, calls to it throw Addressee and calls waiting for it are abandoned, though the process keeps running. It answers under its name again once it is back.
• A name given by TOA_INSTANCE passes to the next process started with it, which then answers calls meant for the one before. Check what the process holds, as charge does.
• Two live processes given one name can both answer calls. Each logs Instance name taken.
• The first call to a component waits for as long as the component takes to appear; the timeout counts from when it is found.
• Declaring stateful on an operation that used to be ordinary changes how it is called. Every caller has to run a runtime version that knows the declaration before it is deployed.

Over HTTP the gateway takes the process name from the path; see Streams and Flow. Why a call to a process gives up what other calls have is explained in Stateful Operations.