Turn on crash recovery
Leases, a heartbeat and a sweep, so an answer survives its server dying. What to configure and what happens.
A turn that outlives its invocation needs to survive the process dying. Latch does this with a lease, a heartbeat and a reaper — no workflow engine, no queue service, no worker that must stay alive.
createRuntime({
storage,
context,
agents,
durability: { enabled: true },
});What happens:
- Claim — the run is claimed with a lease (
owner,fencingToken, expiry). - Heartbeat — the holder extends the lease while it works.
- Die — the process is killed mid-turn. Nothing is lost: every step so far is already in the log.
- Reap — a later
sweep()(orcron()) finds runs whose lease has expired, marks themerrored, clears the owner and bumps the fencing token (invalidating any zombie holder). - Resume —
resume(runId)reclaims the run under a fresh lease, then reads the tail of persisted history to decide what it needs. A turn whose last step called no tool had already finished, so its row is reconciled (completed, orawaiting_inputwhen a gate is pending) with no model call. Anything else re-runs from history and completes server-side.
Point any cron at cron() — it fires due schedules, then reaps and resumes in one call:
await runtime.cron({ maxAttempts: 5 });
// → { due: {…}, reaped: RunRecord[], resumed: RunOutcome[], errors: string[] }maxAttempts (default 5) is the poison-run cap: a run that keeps dying stops being retried instead of looping forever. sweep() is the reap-and-resume half on its own, and resume(runId) reclaims and re-runs one run — for when you want the pieces separately.
On a long-lived Node process, use the in-process scheduler instead of an external cron:
import { createIntervalScheduler } from "@intentface/latch-core";
const scheduler = createIntervalScheduler({
intervalMs: 30_000,
onTick: () => runtime.cron(),
onError: (error) => log.warn({ error }, "sweep failed"),
});
scheduler.start();Ticks never overlap — a slow tick skips the next fire rather than stacking — and the timer is unref'd so it won't hold the process open on its own.