61 lines
1.6 KiB
Markdown
61 lines
1.6 KiB
Markdown
|
---
|
||
|
|
okf: "0.1"
|
||
|
|
type: runbook
|
||
|
|
visibility: private
|
||
|
|
status: active
|
||
|
|
updated: 2026-05-27
|
||
|
|
links:
|
||
|
|
- ../subsystems/control-plane.md
|
||
|
|
---
|
||
|
|
|
||
|
|
# Control-plane — deployment i recovery
|
||
|
|
|
||
|
|
## Deployment
|
||
|
|
|
||
|
|
### From SATURN (primary control node)
|
||
|
|
```bash
|
||
|
|
# Full deploy via SSH
|
||
|
|
./scripts/deploy/deploy-control-plane.sh --ssh
|
||
|
|
|
||
|
|
# Or manually:
|
||
|
|
ssh oskar@100.95.58.48 "cd ~/homelab-codex-ws && git pull origin master && cd services/control-plane && docker compose up -d --build --force-recreate"
|
||
|
|
```
|
||
|
|
|
||
|
|
### Direct on VPS
|
||
|
|
```bash
|
||
|
|
cd ~/homelab-codex-ws/services/control-plane
|
||
|
|
docker compose up -d --build --force-recreate
|
||
|
|
```
|
||
|
|
|
||
|
|
`deploy-local.sh` also creates the required `/opt/homelab/` directory structure and sets ownership to UID 1000 (requires `sudo`). If directories already exist, skip to the `docker compose` step directly.
|
||
|
|
|
||
|
|
### Verification
|
||
|
|
```bash
|
||
|
|
# On VPS
|
||
|
|
docker ps --filter "name=control-plane"
|
||
|
|
curl -s http://localhost:18180/summary | python3 -m json.tool
|
||
|
|
```
|
||
|
|
|
||
|
|
## Recovery
|
||
|
|
|
||
|
|
### World state is stale or corrupt
|
||
|
|
```bash
|
||
|
|
# On VPS — delete checkpoint to force full replay
|
||
|
|
rm /opt/homelab/state/observer_checkpoint.json
|
||
|
|
docker restart control-plane-observer
|
||
|
|
```
|
||
|
|
|
||
|
|
### Flood of pending actions after bootstrap
|
||
|
|
Check if node-agent is running and emitting `service_healthy` events on each node. Without `service_healthy`, the supervisor sees all services as missing and queues redeployments every cycle.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check node-agent on each node
|
||
|
|
ssh oskar@<node> "docker ps --filter name=node-agent && docker logs node-agent --tail 20"
|
||
|
|
```
|
||
|
|
|
||
|
|
### Rebuild from scratch
|
||
|
|
```bash
|
||
|
|
ssh oskar@100.95.58.48 "cd ~/homelab-codex-ws/services/control-plane && docker compose up -d --build --force-recreate"
|
||
|
|
```
|
||
|
|
|