Change the Storage, the Transport or the Language

Most components never say where their state is kept, how calls reach them or what runs their code: there is a default for each. A few requirements make you name something else: “quotes are computed, never stored”, “receive the payment provider’s events from its own broker”, “resize accepts an uploaded stream”, “this one step is a shell script”.

Each of these is a connector. The model article Connectors explains the three kinds.

A connector is named by its shortcut, by a package reference such as @toa.io/storages.mongodb, or, for one of your own, by a relative path.

§Storages

A component with an entity stores it in MongoDB without saying so. All that is needed is the address in the Context. toa create wrote the address of the one npm run dock starts; give production its own:

# context.toa.yaml
mongodb: mongodb://localhost
mongodb@production: mongodb://mongo.example.com

It may differ per namespace or component, and its credentials are a secret; see Addresses and Credentials:

# context.toa.yaml
mongodb:
  .: mongodb://mongo.example.com
  store.orders: mongodb://orders.mongo.example.com

What to know about it:

• One database per application, one collection per component. The database is named after the Context’s name, so applications sharing a MongoDB do not share data. A database name cannot exceed 63 bytes; a process whose name is longer does not start.

• Use a replica set, locally too. Against a standalone server, events are published without the delivery guarantee, and the process says so at startup. A single-member replica set is enough; the generated docker-compose.yaml runs one.

• URL options are passed to the driver. An option the driver does not know stops the component at startup:

  option foo is not supported

• An outage is waited out. While the database is unreachable, calls wait and are answered when it is back.

• Indexes and data changes are migrations, files in the component’s migrations directory: Indexes and Data Migrations. Only this storage applies them.

• Deleted entities stay as tombstones, and moments are stored as dates. Both matter when you write an index; see Describe State.

• Commands are traced. Each is a client span named <command> <collection> in the trace of the operation that caused it, and an observation of toa.storage.query.duration when metrics are on.

§Reading changes of a collection needs images

A stream of a collection ends with a token from which changes can be read later. That needs a replica set and a migration that turns images on:

# components/store.orders/migrations/0003-images.yaml
- images: true

Images cost a copy of every changed record for as long as the database’s operation log holds it. - images: false stops keeping them, and a token issued before that is then refused with 410. A token is valid while the operation log still holds the point it names, and reading from an old token costs in proportion to everything written since.

§Keep nothing: storage: ~

A component that declares an entity but should store nothing sets its storage to null:

# components/store.quotes/manifest.toa.yaml
name: quotes
namespace: store

entity:
  storage: ~
  properties:
    product: { type: string }
    price: { type: integer }

Every write is accepted and discarded, and every read finds nothing. Nothing is written in the Context for it.

A component that declares migrations against it does not start:

Component 'store.quotes' declares migrations, which storage '@toa.io/storages.null' does not apply

A component with no entity at all uses no storage; this is the usual form for components made of computations and effects.

§When a storage cannot do something

A component that asks for a feature its storage cannot support does not start, and says which feature and which storage.

§Bindings

Calls between compositions, events and tasks travel through an AMQP broker. A component declares nothing for it; the Context gives the address, which toa create wrote for the RabbitMQ npm run dock starts:

# context.toa.yaml
amqp: amqp://localhost

This is short for:

# context.toa.yaml
amqp:
  context:
    .: amqp://localhost

context is the broker the application itself runs on. As with other addresses it can be given per namespace or component and may hold several addresses of one cluster:

amqp:
  context:
    .@local: amqp://localhost:5672
    .@production:
      - amqp://rmq-0.example.com
      - amqp://rmq-1.example.com

Credentials are a secret; see Addresses and Credentials.

Components that share a composition call each other in memory. Nothing is declared for that, and the calling code is the same.

§Brokers of other systems

To receive an event published by a system that is not part of the application, a receiver names the binding and a source. Here the payment provider’s event approves an order, with the approve of store.orders that Create a Component writes:

# components/store.orders/manifest.toa.yaml
receivers:
  payment_received:
    binding: amqp
    source: payments
    operation: approve

and the Context says which broker that source is:

# context.toa.yaml
amqp:
  context: amqp://rmq.example.com
  sources:
    payments: amqp://rmq.payments.example.com

Each source has credentials of its own. How the event’s payload becomes a request is in React to an Event.

§The HTTP binding

An operation whose input holds a stream cannot be called through a broker: resize of a component that keeps pictures, pictures, takes the upload as one. Declare the http binding beside the default:

# components/pictures/manifest.toa.yaml
bindings: [amqp, http]

Otherwise the component does not start:

Operation 'resize' takes a stream, which none of its bindings carries

bindings can also be given for a single operation:

operations:
  resize:
    bindings: [amqp, http]

The http binding carries streamed calls only. Ordinary calls, tasks and events of the same component still go through the broker, so amqp stays in the list.

In a deployment nothing more is needed: each component is reached on port 8005. For a local run where several processes share one machine, the http annotation gives each component an address:

# context.toa.yaml
http@local:
  pictures: http://127.0.0.1:31006
  store.catalog: http://127.0.0.1:31007

A key applies to the component it names and to everything under it; . is every component, and a component without a namespace is keyed by its name. A process listens on the port in the address of each component it serves.

Each streamed call opens a connection of its own, so do not stream what is neither large nor long. Calls between components of one composition pass the stream as it is and need none of this.

§Bridges

Operations are JavaScript or TypeScript modules in the component’s operations directory: Write Operations, Use TypeScript.

§A shell script as an operation

Name the bash bridge for the whole component:

# components/tools/manifest.toa.yaml
name: tools

bridge: bash

operations:
  checksum:
    input:
      properties:
        text: { type: string }

#!/bin/bash
# components/tools/operations/checksum.sh

while [[ "$#" -gt 0 ]]; do
  case $1 in
    --text) text="$2"; shift ;;
  esac
  shift
done

echo -n "$text" | sha256sum | cut -d ' ' -f 1

$ npm run restart
$ npx toa call tools.checksum "{ input: { text: 'hello' } }"

or for one operation of a component whose other operations are TypeScript:

# components/store.catalog/manifest.toa.yaml
operations:
  checksum:
    bridge: bash
    input:
      properties:
        text: { type: string }

// components/store.catalog/operations/label.ts
import type { Context, LabelInput } from '../types/index.d.ts'

export async function computation (input: LabelInput, context: Context) {
  const sum = await context.local.checksum({ input: { text: input.title } })

  return `${input.title} (${sum.slice(0, 8)})`
}

The rules of a Bash operation:

• Input must be an object. Each property is passed to the script as a named argument, --name value, with the value converted to a string.
• Output is what the script writes to standard output, as a string.
• Exit code 1 makes the operation return an error whose message is what the script wrote to standard error. Any other non-zero code is an exception.
• There is no entity state and no operation context, so only computations are possible.
• The script does not see the platform’s environment: every TOA_* variable is removed.