Halt

Stopping a running system is easy. Stopping it cleanly is not. At any moment a busy application has requests in flight, events committed and not yet delivered, and tasks waiting. Killing its processes leaves that work half-done. The runtime’s delivery guarantees will eventually repair it, at the cost of retries, duplicates, and a period in which the state is not what anyone intended.

Sometimes a clean stop is required: the database is being upgraded, the broker replaced, or a release is going out that cannot run beside the previous one.

A halt stops an entire deployment at a moment when nothing is owed, keeps it stopped for a stated interval, and brings it back.

§What a halt does

A halt proceeds in stages across every process of the deployment.

1. Everything goes quiet. The gateway stops accepting requests. Pulses stop firing and delayed calls stop being dispatched. No new work enters.
2. What is in progress finishes. With nothing being added, the remaining work is finite. Calls are answered, events are delivered, and tasks run.
3. The deployment checks that it is idle. The topology records whether anything was called. When it shows nothing was, the stop is confirmed.
4. Every connection is closed. No process holds a connection to the broker, the database, or the cache.
5. After the interval, every process resumes on its own.

§What it gives

Nothing is lost and nothing is left half-done. The deployment stops only after every call is answered and every owed message is delivered.

The infrastructure is free. For the length of the interval, nothing is connected to the broker, database, or cache. They can be upgraded, moved, or restarted without the application noticing an outage in the middle of its work.

The processes stay up. A halted process is not terminated or replaced. It keeps its place and answers that it is alive, and the gateway tells clients when to return.

It ends by itself. Nothing has to reach the deployment to wake it. This is necessary, since a process with no connections cannot be told anything.

§What it refuses to do

A halt is conditional, and the condition is the point.

A deployment that does not go quiet is not stopped. If work is still happening when the check is made, the halt is called off and every process continues as before. Nothing had been closed, so nothing has to be recovered. A deployment is never stopped on a guess.

It cannot be undone from outside. Once the processes have disconnected, only the interval brings them back. The interval is therefore chosen as one the service can afford.

It is off unless enabled. A capability that stops everything is not available by default. An application turns it on and states the limits of what a halt may ask for.

§What it asks of components

The runtime can quiet what it runs: operations, pulses, deliveries. It cannot see what a component started on its own, such as a timer polling an external system or a connection opened outside the context.

A component that does such things takes part in a halt explicitly. It is told when to pause its own activity and when to resume it. A component that keeps working makes the deployment appear busy, and the halt is called off.

A halt is therefore as complete as the components make it. An application whose components do everything through the context can be halted with no extra work, which is one more consequence of keeping operations genuine.

§Rolling what cannot be rolled

Contracts noted that some changes cannot be deployed by a rolling update, because two versions cannot serve together.

A halt provides the alternative. The deployment is halted, the new release is deployed while nothing is serving, and the processes that start are all of the new version. The old and the new never run side by side. Since the halt waited for all work to finish, nothing in transit was shaped for the old version.

§A tool to rehearse

A halt is the kind of capability used rarely and under pressure. A stop that has never been exercised is one nobody knows to work, and whether an application can be halted depends on its components behaving as described above.

It is worth running a halt on a schedule, in ordinary conditions, so that the day it is needed is not the first time it is tried.