Choose a backend
Precedence is
preloop serve --store URL, then PRELOOP_STORE_URL, then the SQLite default. PRELOOP_HOME defaults to $HOME/.preloop (or ./.preloop when no home is available); a system service normally uses /var/lib/preloop.
Postgres uses 16 writer and 16 reader connections by default, plus one dedicated wake listener. Set PRELOOP_PG_WRITERS and PRELOOP_PG_READERS to size the pools; each value is clamped to at least one. Queue claims use row locks with SKIP LOCKED, and notifications are wake hints rather than the source of truth.
Apply schema migrations
The control schema uses embedded, forward-only refinery migrations. A fresh local SQLite store is initialized by the supported setup paths (preloop serve on a fresh $PRELOOP_HOME, preloop init, preloop server install); existing stores are upgraded explicitly:
<db>.pre-migrate-<unix>.bak before migration. --no-backup disables that backup; use it only when an external backup already exists. --dry-run reports the migration plan but is not a strictly read-only filesystem probe. Migrations are forward-only; restore a backup or apply a corrective migration rather than expecting a down migration.
Useful flags:
refinery_schema_history; it, not schema_meta, is the version authority. schema_meta retains the key fingerprint used to reject an incompatible state key. preloop serve initializes a brand-new local store (absent or empty) and otherwise refuses an uninitialized, legacy, older, newer, incomplete, or divergent schema rather than migrating it, and points to preloop store migrate. A Postgres store is the operator’s to provision.
Import a legacy database
Legacypreloop.db files are not read automatically. Stop the old engine, then import into a new control database:
--active refuse is the default; use --active requeue or --active cancel when you have explicitly decided how unfinished work should be handled. --key FILE supplies the cluster key when it is not at PRELOOP_HMAC_KEY or <state-dir>/hmac-key.bin.
Retention and quotas
Retention sweeps run once at startup and hourly. Only terminal runs are candidates; queued, pending, and in-progress runs are retained regardless of age. Diagnostic reservations expire after one hour, with a per-job cap of 32 pending upload tokens.
State layout and backups
The default state home contains:
For SQLite, stop the engine or use a consistent database backup; retain the database together with its
-wal and -shm files when making a live file copy. Back up the credential store, config, HMAC key, and any blobs, logs, artifacts, or snapshots whose retention matters. preloop.sock, serve.lock, and PID files are runtime artifacts, not backup data.
Keep the state directory on one filesystem. Snapshot, blob, and cache publication uses atomic renames; splitting the directory across devices causes EXDEV failures. The HMAC key must remain stable across restarts and nodes or stored secrets and state fingerprints cannot be read.
On Postgres, a dedicated wake listener receives preloop_webhooks notifications, so every node’s webhook worker wakes when another node makes a delivery claimable. Notifications are only a latency gain: workers re-read the queue at least every five seconds, and durable queue state and migrations remain shared through Postgres. SQLite is single-node and wakes only its own worker. Live logs, debug sessions, and file-backed cache and artifact reservations remain node-local, so a shared store alone is not a complete active-active deployment.