Server deployment

Current published checkpoint e08ed3b2684bdf04f0288c7162747a2783177026: CI 37600158231 passed all four jobs: general checks, Python compatibility, Docker installation and empty-host acceptance. Python 3.12 passed 2102 tests. Website 37600311118 passed build and tests. Empty-host acceptance now runs automatically on main pushes as well as explicit manual requests; pull-request behavior is unchanged.

The current native checkpoint summary binds the new Docker artifact to 13 reports and 254 passing checks for project creation, Supabase features, backup/restore, upgrades, rollback and restart. This is the new run's fresh-only artifact, not the earlier baafbad archive or historical installation transcription. Its runtime evidence is bound to core code commit e08ed3b2684bdf04f0288c7162747a2783177026; a subsequent documentation publication preserves that identity and does not establish runtime acceptance for later code changes.

The fresh empty-host evidence contains six reports and one run manifest, with 58 unique checklist checks and a separately reported 21-check live Auth bootstrap. The identical rehearsal handoff copy adds no checks. The native log and final script guards establish that the shipped system unit became active and its console answered after the rehearsal. The earlier JSON unit snapshot records the deliberate inactive rehearsal phase; it is not the final restored service state. This is a fresh Ubuntu runner installation checkpoint, not physical reboot, Kali verification, complete lost-source recovery, cold/warm Cron continuity or production acceptance. Static-serving and TLS stub checks retain their separate scope; a public certificate is not proven.

A separate Fedora 44 systemd descriptor probe preserved five descriptor identities across same-PID Python exec under bounded systemd scope custody. Its two-file public archive contains the probe and summary. It launched no guest or Docker container and accepts only that Python descriptor behavior, not a Linux guest, reboot or portable production deployment.

Historical integration checkpoint before e08ed3b, 2026-10-07: Docker installation CI passed after baafbad, with 2086 Python tests and 254 current native checks. Manual standalone-service acceptance failed at the socket configuration handoff; its later rerun passed on e08ed3b. Full recovery and production remain unaccepted. See current status; the following checkpoint paragraphs preserve historical snapshots and their limits.

Historical checkpoint, 2026-10-06: complete snapshot verification passed 1449 tests with no failures, errors, warnings or skips and independent actual review. Repairs cover SQLite and HTTPError response closure, tempfile ownership and expected test diagnostics. All 52 preparation cases also passed; native PostgreSQL startup, Cron/Vault restart continuity and physical restore remain unaccepted. See current status for CI results and evidence scope.

The separate public55 V3 role has actual MET: 25 component and 30 pump tests passed, original 290 and added 4 actual mode/byte captures matched, and all three known helpers were removed with no unresolved resources. Public BusyBox V8 help metadata also has actual MET: fixed-path Bash 5.3.3 and BusyBox 1.37.0 help returned zero, with its one helper removed. Neither result accepts private request operation, timeout timing/signal behavior, installed binary source provenance, candidate startup, cold/warm Vault or Cron continuity, physical restore or production.

The preserved warning-window V4 negative run reported 1447 ordinary tests OK but six ResourceWarnings, native 1 and observed window 839 for every warning. Three records have exact sqlite3.Connection type; three report tempfile.py line 484 with NoneType source. Observation windows are not allocation causes, and the three tempfile origins remain unknown. At that historical checkpoint, fresh complete source/identity/mode/full bindings, final-revision CI and manually dispatched empty-host acceptance remained required; published 0e745394 CI and Website results retain only their earlier revision scope.

Runbook for the native Linux installation path. The primary container workflow is described in Install with Docker. September workstation and local-VM rehearsals below retain their original scope; they are not fresh production acceptance.

A real server runs main at sbarbase.sbarah.com (Ubuntu 26.04, systemd, the TLS proxy behind Cloudflare), upgraded from 0.2.0 on 2026-10-16; local and R2 restores passed there on one environment without application data. Historical published 0e745394 passed Docker installation CI, while empty-host acceptance was skipped at that checkpoint. No independent public-server pilot is established. Current startup tooling defines only an explicit prepare-only material-handoff status; no actual native material-preparation run is accepted by this guide. The earlier V9 initV2 run passed 52 focused tests but its historical actual 1447-case full regression failed with one failure and 15 errors.

Earlier mode, scratch and warning verification attempts remain historical refusals bound to their own source and date. Resource ownership and fixture diagnostics were repaired, and complete verification of that earlier snapshot passed 1449 tests. This does not transfer native startup or recovery evidence from another run. The status history preserves the earlier observations and their limits.

Prerequisites

Pinned images are pulled by digest on install; no floating tags are used. See upstream update policy before changing any pin.

Install

One command on the server covers prerequisites, preflight, the full rehearsal and the acceptance evidence:

deploy/server-acceptance.sh --rehearse --bootstrap-file /path/to/operator.json

Without --rehearse the same command runs the prerequisites and the preflight only. It refuses before touching anything when a prerequisite is missing (docker, bun, git, /usr/bin/python3 3.12+, native Linux daemon), when the bootstrap file is not mode 600, or when the preflight reports a blocker, and it never prints a secret. The step-by-step sequence below is what it runs, for an operator who wants to drive each stage by hand.

/usr/bin/python3 lab/install_server.py check        # read-only preflight, non-zero on blockers
/usr/bin/python3 lab/install_server.py plan         # print the exact steps
/usr/bin/python3 lab/operator_file.py /root/sbarbase-operator.json   # prompts, no echo, mode 0600
/usr/bin/python3 lab/install_server.py install --bootstrap-file /root/sbarbase-operator.json

Create the file with lab/operator_file.py rather than by hand: it prompts for the email, the organization and the password twice without echoing, refuses a password shorter than 12 characters, a malformed email, a relative path, a symlink and an existing file (unless --force), creates it with mode 0600, and never prints the password. For automation it also accepts the same JSON on bounded stdin with --stdin, so a password never reaches an argument or the shell history.

install performs, in order: preflight, private state and secret directories (0700), pinned image pull when not local, bun install when needed, console build, owned runtime startup, and the operator identity bootstrap. The bootstrap file is a 0600 JSON object with exactly email, password and organization; it is piped on stdin and never passed as an argument or printed.

A fresh install initializes exactly one HBA generation for the new database. A retained installation (containers already present) is refused until its source and current recovery target carry generation pins:

/usr/bin/python3 lab/adopt-retained.py source
/usr/bin/python3 lab/verify_retained.py source
/usr/bin/python3 lab/adopt-retained.py target
/usr/bin/python3 lab/verify_retained.py target

Adoption starts and stops only the captured database container and preserves its existing rules byte for byte. Never delete a pin, journal or receipt to bypass the gate.

Supervise

Install and start the unit with one command. It renders the shipped unit for this installation (paths, service user, Bun directory), verifies the result with systemd-analyze verify, then installs, reloads, enables and starts it:

sudo /usr/bin/python3 lab/install_server.py supervise --apply \
     --service-user ops-account --home /srv/ops-account --bun-dir /srv/ops-account/.bun/bin

sudo replaces PATH, so a Bun outside a system directory must be named with --bun-dir or the unit starts without it.

Without --apply it only renders and verifies, prints the exact commands it would run, and writes docs/evidence/supervisor-unit.json. It refuses to install a unit that does not verify, refuses --apply without root, and refuses to install for an account that does not exist on the host. Point it at a different layout with --service-user, --home and --bun-dir; the shipped unit is never hand-edited. deploy/server-acceptance.sh forwards the same three flags, so the one-command acceptance path can name the server's account too.

The unit carries what updates need (upgrades):

• Its first ExecStartPre runs deploy/host-preflight.sh --runtime. Read-only host admission refuses before locks, upgrade changes or leftover runtime cleanup on every restart.
• The next ExecStartPre is the upgrade guard (lab/upgrade_guard.py, or the copy an upgrade left in .lab/upgrades/guard.py). It runs before the preflight and before any code of the version the checkout holds, and moves the checkout back when a new version keeps failing its start.
• After the guard, ExecStartPre (lab/leftover_runtime.py) handles a supervisor that was killed (SIGKILL, the OOM killer) instead of stopping. Docker, not the unit, owns the containers, so they keep running, and the preflight used to refuse every later start because owned containers were running. This step stops them the way the supervisor's own stop does (docker stop, every container and volume kept), but only when no supervisor, worker or effect owner holds its lock and no provisioning receipt, HBA journal or migration record waits for reconciliation. Otherwise it refuses and names the reason. The supervisor runs the same step after it takes its locks, which covers Docker and terminal starts.
• RestartForceExitStatus=42: after an update or rollback from the console moves the checkout, the supervisor stops everything cleanly and exits with code 42 so systemd starts it again on the new version. Restart=on-failure already covers that exit; the line keeps it so if the restart policy changes.
• StartLimitIntervalSec=0, so systemd never gives up restarting while the guard needs several starts to go back, and TimeoutStartSec=600, because the guard's way back reinstalls dependencies before the preflight runs.

A unit installed before host admission was added must be rendered and reinstalled with supervise --apply to obtain the admission boundary on every restart. Without the leftover cleanup line, a start after an unclean stop still refuses at the preflight; stop the leftover containers once with /usr/bin/python3 lab/leftover_runtime.py as the service account. Reinstall the unit with supervise --apply when you move to the version that has them.

Four things must be true before the unit can serve, and the preflight names each one rather than failing obscurely:

1. The service account exists. Create it and give it the checkout, or install with --service-user/--home/--bun-dir.
2. The service can reach the declared local Docker daemon. The default is /var/run/docker.sock, with socket permission for the service account. DOCKER_CONTEXT is refused. For a nondefault local socket or daemon data root, set SBARBASE_DOCKER_SOCKET and SBARBASE_DOCKER_DATA_ROOT in the service's drop-in and acceptance environment. Any DOCKER_HOST must match that socket. A drop-in uses Environment=SBARBASE_DOCKER_SOCKET=/run/docker-custom.sock, Environment=DOCKER_HOST=unix:///run/docker-custom.sock and Environment=SBARBASE_DOCKER_DATA_ROOT=/srv/docker with the actual existing paths. The first admission step refuses missing or mismatched paths without creating them. When Docker is unreachable, preflight reports the endpoint and does not guess about pinned images or existing containers.
3. The host has the memory. The next start needs the limits of the containers it runs plus a 2560 MiB reserve: 4352 MiB on an empty server, more with each environment, 8.8 GiB for the development host's retained split placement. The unit's ExecStartPre, the preflight and the runtime admission derive the same figure, and the gate refuses with host_memory_headroom rather than half-starting.
4. Write access stays inside the checkout. ReadWritePaths names the installation root only. A ReadWritePaths entry for a directory that does not exist makes systemd fail the unit with 226/NAMESPACE before it runs anything, so the unit never lists a path the installation does not create; every secret lives in <checkout>/.secrets/upstream. The rendering has been verified on the development host and the install commands are recorded in that evidence file; the install itself needs root on the target server.

The unit runs lab/dev.py, which builds the console, starts the owned runtime, runs the API and the provisioning worker, and stops the runtime on SIGTERM. ExecStartPre re-runs the preflight, so an unfit host fails before any container is touched. For a foreground run instead, execute /usr/bin/python3 lab/dev.py in a terminal and stop it with Ctrl+C.

The one command that produces the acceptance evidence is deploy/server-acceptance.sh (below); it runs the preflight, the checks and the rehearsal in order and keeps the handoff copy. To drive the two stages yourself instead:

/usr/bin/python3 lab/install_server.py check
/usr/bin/python3 lab/deployment_rehearsal.py --bootstrap-file /path/to/operator.json

The install itself needs the placement's memory plus the reserve (4352 MiB on an empty server): the owned runtime refuses to start when the host cannot support the placement containers plus the reserve, and names the reason (host_memory_headroom) rather than half-starting. The installer releases the installation operation lock after the console build and before it starts the owned runtime, because the runtime takes that lock itself and holds it for its lifetime.

On a server where the unit still has to be installed, run the acceptance path as root with --install-unit: it renders and verifies the unit, installs and starts it, proves the console and the TLS termination, releases the unit so the rehearsal can own the containers and state, runs the rehearsal with the unit required, starts the unit again and asserts it is active, and leaves the evidence in one place. systemd calls the unit active the moment the supervisor process starts, long before the console can answer, so after every start of the unit the run waits until the console answers over loopback, at most 300 seconds (--console-timeout changes the bound), and fails with the last thing it saw when it does not. The release happens whenever the unit is found active, with or without --install-unit, and a trap starts it again on any exit, so a failure in the middle of the run cannot leave the installation down. Two supervisors cannot own the same containers, so the rehearsal never runs against a live installation.

sudo deploy/server-acceptance.sh --rehearse --install-unit \
     --service-user ops-account --home /srv/ops-account --bun-dir /srv/ops-account/.bun/bin \
     --bootstrap-file /path/to/operator.json

sudo replaces PATH with a secure default, so a Bun installed under the invoking user's home is invisible to the script and to the unit it installs. Pass --bun-dir (the script adds it to PATH and uses it in the unit) whenever Bun is not in a system directory; without it the script stops at the prerequisite step and names the flag.

sudo is used for the two unit steps only. Everything that touches the installation runs as the account that owns it (--service-user, or the checkout's owner when the flag is omitted), because the state carries its owner and the runtime refuses a process whose uid does not match the files it holds (HBA ownership inode mismatch): running the preflight, the checks or the rehearsal as root against a service-account installation fails, and a root-run install would leave files the service cannot use. The script does that substitution itself, so the single sudo invocation above is still the whole command.

Admission selects the declared local socket explicitly and forwards the declared profile, socket and data root when a step runs as the installation account. Saved Docker contexts do not select the endpoint. sudo can discard shell exports, so supply the same public deployment inputs explicitly to the root invocation and service drop-in. The default endpoint can also be restated:

deploy/server-acceptance.sh --rehearse --docker-host unix:///var/run/docker.sock

For a custom local endpoint, declare its existing socket and actual Docker data root, then pass the matching --docker-host. A mismatch or context override refuses before rehearsal effects. The native route still requires host Python, Bun and systemd; the primary Docker route uses the public launcher.

An acceptance run writes its rehearsal to docs/evidence/server-acceptance-rehearsal.json and copies it to server-acceptance-latest.json, leaving the plain docs/evidence/deployment-rehearsal.json untouched. That record carries the host facts (kernel, Docker, Bun, Python, headroom, free disk), the exact command that ran, the full pin set with digests, the state of the sbarbase.service unit, start and finish times, and one row per check with its result. A rehearsal that cannot run records the blocking finding instead of a pass, and exits non-zero.

Check: does the supervised path work, not just a direct run?

The rehearsal runs the supervisor directly. It records the systemd unit's own state, and fails the the shipped supervisor unit is installed for an acceptance run check when /etc/systemd/system/sbarbase.service is not installed, so a green acceptance run means the unit was present and enabled. The unit it verifies with systemd-analyze is the installed file itself, and the evidence says which file that was; the checkout's template is verified only when no unit is installed, and the evidence records that too.

Verify after install

• lab/console_build_check.py rebuilds the console and proves the served page is intact (non-empty index, every local asset it references present and hashed). The installer now refuses a build that produces an unusable page, and the rehearsal records built console page is intact before it starts anything.
• lab/pinned_images_check.py proves every pin is present locally and resolves to the pinned digest. It never pulls, and a tag that now points at a different digest fails instead of passing.

/usr/bin/python3 lab/install_server.py smoke         # management Auth, per-environment routes, console pid
bun lab/combined-gateway-check.ts                    # 14 simultaneous gateway checks (needs the console built)
/usr/bin/python3 lab/combined-supervisor-check.py   # full rehearsal: start, checks, supervised shutdown
/usr/bin/python3 lab/deployment_rehearsal.py         # one command: install, supervise, verify, shut down, evidence
/usr/bin/python3 lab/target_placement_rehearsal.py   # retained target placement: start, probes, stop

deployment_rehearsal.py writes docs/evidence/deployment-rehearsal.json and exits non-zero on any failure; on a host without the required headroom it records the preflight refusal and stops without starting anything. target_placement_rehearsal.py exercises the adopted recovery-target placement end to end and returns it to the paused, stopped state.

The smoke command reports each endpoint status and does not modify state. The supervisor rehearsal additionally requires the same headroom as a normal start.

HTTPS and network exposure

The console and the gateway bind loopback and are reachable only through the host. Terminate TLS in a reverse proxy and keep the Docker networks internal: the installer creates no published ports. Do not expose the management Auth endpoint or the provisioning API directly. Set the public URL the console should advertise in the proxy, not in the console build. Auth's email links and OAuth callbacks take the public address from SBARBASE_PUBLIC_URL in a drop-in of the service (sign-in settings).

Studio exists as an optional per-environment feature in the legacy shared-engine workflow, with separately scoped CI evidence. Its proxy and authentication requirements remain essential; native-dedicated Studio is not accepted by that legacy result. Each environment's Studio has its own address. The shipped proxy serves them with --studio-host studio-{id}.example.com (the same pattern as SBARBASE_STUDIO_HOST), passing only names that match it with their own host, after a wildcard DNS record and a certificate that covers it are in place (Studio); without it, Studio is reached through an SSH tunnel. The proxy also reads SBARBASE_STUDIO_HOST when it has no --studio-host flag, and lab/studio_address.py apply DOMAIN writes that setting for the console and the proxy unit in one step. Studio has no login of its own and its pages carry a database connection string, so it must never be reachable unauthenticated, and it must not ride the data plane's route pattern. Integration specification, sections 4 and 5, covers the gate and the routing options.

A reference termination ships with the repository and is exercised by the check (/usr/bin/python3 lab/tls_termination_check.py, 27 checks, docs/evidence/tls-termination.json). It needs only Bun and a certificate:

Pin the console's loopback port first, so the proxy's upstream survives restarts and reboots. Without it the console picks a new ephemeral port at every start:

sudo mkdir -p /etc/systemd/system/sbarbase.service.d
printf '[Service]\nEnvironment=SBARBASE_CONSOLE_PORT=8787\n' | sudo tee /etc/systemd/system/sbarbase.service.d/console-port.conf
sudo systemctl daemon-reload && sudo systemctl restart sbarbase

bun deploy/console-tls-proxy.ts \
    --cert /etc/letsencrypt/live/console.example.com/fullchain.pem \
    --key  /etc/letsencrypt/live/console.example.com/privkey.pem \
    --public-host console.example.com \
    --upstream http://127.0.0.1:8787 \
    --https-port 8443 --http-port 8080

To keep it running across reboots, give it its own systemd unit that runs as the service account. Name it sbarbase-proxy.service: lab/studio_address.py apply writes its drop-in there unless --proxy-service names another unit. On Fedora, SELinux refuses a unit that executes Bun under /home directly (status 203/EXEC, permission denied), so start it through a shell: ExecStart=/bin/sh -c 'exec /home/sbarbase/.bun/bin/bun deploy/console-tls-proxy.ts ...'. The rehearsal VM ran it this way, with a local CA, across a reboot (docs/evidence/vm-https-first-project.json). It redirects plain HTTP to https://<public host>/ on port 443, so publish it on 443 on a real server.

It refuses to start unless the certificate and key are regular files, the key is not group or world readable, --public-host is set to a bare host name, and the upstream is loopback. The loopback assertion is applied to the console URL whichever way it was supplied, including the one read from .lab/upstream/server.json when --upstream is omitted. It answers plain HTTP with a 308 redirect to HTTPS, adds Strict-Transport-Security, X-Content-Type-Options, Referrer-Policy, X-Forwarded-Proto and X-Forwarded-Host, and logs only method, path and status: never bodies, query strings, cookies or credentials. The forwarding headers carry --public-host, never the client's own X-Forwarded-* or Host values, which are dropped before the request reaches the console. It serves only --public-host, loopback names and Studio addresses that match --studio-host, and answers any other host with 421, so a wildcard DNS record never shows the console under another name.

Its own hardening is part of the checks: the redirect and the forwarded host come from --public-host, never from the client's Host header (an attacker supplied host cannot turn the redirect into an open redirect), hop by hop headers are stripped before forwarding, and a request body over --max-body (the upload limit plus 1 MiB by default) is answered 413 as soon as the stream passes the cap, without buffering it in full. An operator may prefer nginx, Caddy or the platform proxy; the checks above state which behaviour any replacement must keep.

Firewall: two routes from containers to the host

Two services in containers call the host on its Docker bridge addresses: Studio reaches Auth and Storage through its route on port 25432, and Edge Functions reach Storage on port 25433. A host firewall that drops incoming traffic by default (ufw's default) drops these too, and Studio's Storage, users and API pages then fail. Allow exactly these two, which keeps SSH, 80 and 443 the only ports open from outside:

SUBNET=$(docker network inspect sbarbase-durable-net --format '{{(index .IPAM.Config 0).Subnet}}')
GATEWAY=$(docker network inspect sbarbase-durable-net --format '{{(index .IPAM.Config 0).Gateway}}')
sudo ufw allow from "$SUBNET" to "$GATEWAY" port 25432 proto tcp comment 'sbarbase Studio route'
sudo ufw allow in on br-+ to any port 25433 proto tcp comment 'sbarbase Functions to Storage'

The first rule admits only the runtime network, so a container on another network cannot reach the Studio route. The second admits any Docker bridge on that one port; each listener then refuses sources outside its own environment's network. Opening Studio checks the first route from inside Studio and stops with a message naming the firewall when it is blocked.

If a restore is interrupted

A failed restore leaves .lab/upstream/recovery-target.json, and a plain rerun refuses because the descriptor exists. Do not edit that state by hand:

/usr/bin/python3 lab/recovery_reconcile.py        # stop the retained containers
/usr/bin/python3 lab/retire_recovery_target.py --reason "why it failed"
/usr/bin/python3 lab/recovery-restore-db.py       # now a fresh restore may run

Retirement refuses unless the descriptor's status is interrupted, failed or cleanup-failed, every container of that prefix is stopped, and no per-target HBA operation is pending. It archives the descriptor into recovery-target-history/, records the path in the cutover journal, and removes the active descriptor; containers and volumes are never deleted. A target that already holds a usable database can instead be adopted in place with lab/adopt-retained.py target.

Backup, upgrade and rollback

The step by step versions are backup and restore and upgrades.

• Backup: every environment, Storage's shared metadata and the management database are backed up daily while Sbarbase runs (lab/backup.py, SBARBASE_BACKUP_HOUR), and lab/offsite.py sends encrypted copies to an S3-compatible bucket. The encrypted export and independent-restore path is documented in INDEPENDENT-RESTORE, a historical independent-restore fixture. Separate legacy Docker and local-VM backup/restore observations are recorded in status; none establishes complete candidate or physical fresh-host recovery.
• Upgrade: the console's Updates page shows a newer signed release and installs a safe one with one click; lab/upgrade.py does the same from the command line. Each upgrade backs up every environment first, holds application traffic until the new version passes its health checks, and moves back by itself if it does not. Automatic updates are opt-in. The upgrades guide has the steps, the classes that need a rebuild or a migration, and the limits.
• Rollback: before confirmation it is automatic. After it, Roll back on the Updates page or lab/upgrade.py rollback keeps everything written since the update, and refuses when the previous version cannot open the control catalog.
• Changing a pinned upstream image is a maintainer's task: follow UPSTREAM-UPDATE-POLICY, which records the rollback pin and any data migration in a dated entry under docs/upstream/.

Historical rehearsal evidence and current limits

• No end-to-end install rehearsal on an independent public server is established. An empty server has been simulated in a local virtual machine (Fedora 44 Cloud, clean clone, lab/vm-rehearsal.sh), whose later rehearsal found ten defects the development host could not show; see deployment readiness. The documented command itself completed end to end on the development host at 7:25 PM, 13 of 13, including the unit install as root and its restart (docs/evidence/server-acceptance-latest.json); what a server adds is root, the service account, a public certificate and the absence of a retained installation. The preflight, the runtime startup, adoption and verification are also each proven separately in the evidence files under docs/evidence/.
• No production capacity claim: 5888 MiB and 5.75 CPUs are configured ceilings, not measured peak demand. Sustained mixed load and 10/100-project capacity are unproven.
• Realtime, Functions and Studio have optional implementations in the legacy workflow, with separately scoped CI observations. They are not thereby accepted for native-dedicated placement. The connection pooler and exposed operator Cron feature remain unimplemented in that legacy workflow; accepted original-image Cron/HTTP fixtures are a separate bounded prerequisite.
• Legacy encrypted copy/fetch/restore and second-VM restore have scoped historical observations. Complete fresh-host installation recovery, candidate physical restore and coordinated multi-host operation remain unaccepted.
• Updates from the console and opt-in automatic updates exist, with a health-gated way back and a start guard (upgrades). They are covered by unit tests plus historical clean-runner CI and local-VM observations summarized in status. On the real server, signed releases from the canonical repository with the maintainer's key were installed from the command line (lab/upgrade.py start --release, first 0.3.1); the console's install button, automatic updates and Roll back have not run there.
• After an unclean supervisor stop, lab/leftover_runtime.py provides a lock- and receipt-gated stop of owned containers before preflight, retaining volumes. This correction has source regression evidence only; no live kill rehearsal is established. A live owner or pending receipt/journal still refuses reconciliation. Reinstall the unit to gain its documented preflight cleanup step; this does not establish arbitrary crash recovery.
• Bootstrap creates the first operator; invitations are a separate legacy feature with scoped local-VM evidence. The console has sign-in factors (MFA) and rate limits on its sign-in, which answer 429 when exceeded. See status for the historical scope.

Private directory permissions

Installation and startup create .secrets and .secrets/upstream with mode 0700, owned by the account running the service. Symlinks and existing directories with a different owner or mode are refused. The runtime state directory .lab/upstream is also kept 0700: unlike the secrets, one left open by an install under umask 002 is closed at start, because restore refuses its journal in an open directory; a link or another owner is refused. If a new version refuses an older checkout whose parent was created as 0755, stop the installation normally and verify both directories are real directories owned by the service account before setting only these two directory modes to 0700. The shipped Docker image runs as root inside the container. This repair does not require changing ownership or internal file modes; retain all secret data and backups. Runtime private directory ownership or mode refused and Runtime private directory identity changed require correcting permissions or the checkout path before retrying.

Edit on GitHub