60 lines
2.8 KiB
Markdown
60 lines
2.8 KiB
Markdown
|
---
|
||
|
|
okf: "0.1"
|
||
|
|
type: service
|
||
|
|
visibility: private
|
||
|
|
status: active
|
||
|
|
updated: 2026-08-03
|
||
|
|
links:
|
||
|
|
- ../decisions/deploy-runner-uzasadnienie.md
|
||
|
|
- ../runbooks/deploy-runner-install.md
|
||
|
|
---
|
||
|
|
|
||
|
|
# deploy-runner — host-side executor for `redeploy` actions
|
||
|
|
|
||
|
|
Runs on each node that should be able to execute an operator-approved
|
||
|
|
`redeploy`. It is the missing half of the self-healing loop: the control-plane
|
||
|
|
executor can generate and dispatch redeploys, but nothing on the node could ever
|
||
|
|
perform one.
|
||
|
|
|
||
|
|
## How it works now
|
||
|
|
|
||
|
|
```
|
||
|
|
supervisor → actions/pending/<id>.json (unchanged)
|
||
|
|
operator → actions/approved/<id>.json (unchanged, HITL)
|
||
|
|
executor (vps) → actions/deploy/<node>/<id>.json ← dispatch only, no execution
|
||
|
|
deploy-runner ← rsync-pull (node initiates; VPS never connects to a node)
|
||
|
|
→ scripts/deploy/deploy-service.sh --force-recreate
|
||
|
|
→ events/<node>/evt-…-action_result-….json → rsync-push
|
||
|
|
executor → completed / failed (via _reconcile_running_actions)
|
||
|
|
```
|
||
|
|
|
||
|
|
Design points, and why:
|
||
|
|
|
||
|
|
- **Host-level, not a container.** Compose resolves relative bind mounts and the
|
||
|
|
project name against the filesystem of whatever runs it. In a container those
|
||
|
|
resolve to container paths that the daemon then interprets as host paths — a
|
||
|
|
silent way to produce broken mounts, or to land in a different Compose project
|
||
|
|
where `--remove-orphans` deletes the running stack. On the host it behaves
|
||
|
|
exactly like a human `deploy.sh`.
|
||
|
|
- **Independent of node-agent** (own rsync pull, own result push) so it can
|
||
|
|
redeploy `node-agent` itself — the solaria case, where node-agent has been
|
||
|
|
docker-blind for weeks.
|
||
|
|
- **Separate inbox** from `actions/dispatch/<node>/`: node-agent deletes and
|
||
|
|
failure-reports every file in its own inbox that is not `container_restart`.
|
||
|
|
- **No `git pull`.** A redeploy reconciles the node to the checkout it already
|
||
|
|
has. Shipping new code stays a human `scripts/deploy/deploy.sh` action.
|
||
|
|
- **No `--build`, no `--remove-orphans`, always `--force-recreate`.** Plain
|
||
|
|
`up -d` is a no-op when config is unchanged — exactly the unhealthy-container
|
||
|
|
case; building on the 4 GiB VPS mid-incident is an OOM risk.
|
||
|
|
|
||
|
|
Every action is validated before anything runs (`action.py`): type must be
|
||
|
|
`redeploy`, `node` must match this node, the service name must be a plain
|
||
|
|
kebab-case name, the service must be listed in `hosts/<node>/services.yaml`, and
|
||
|
|
its compose file must exist. Nothing from the action payload is ever executed —
|
||
|
|
only a validated service *name* reaches the deploy script. Actions are
|
||
|
|
idempotent (marker in `state/processed-deploy-actions/`), single-instance
|
||
|
|
(`flock`), and time-boxed (`DEPLOY_TIMEOUT_SECS`, default 600 s, below the
|
||
|
|
executor's `REDEPLOY_TIMEOUT_SECS` of 900 s so the node reports before the
|
||
|
|
control plane gives up).
|
||
|
|
|