2026-05-15 23:29:59 +02:00
|
|
|
services:
|
|
|
|
|
redis:
|
|
|
|
|
image: redis:7
|
|
|
|
|
container_name: agent-system-redis
|
|
|
|
|
ports:
|
|
|
|
|
- "6379:6379"
|
|
|
|
|
restart: unless-stopped
|
|
|
|
|
|
|
|
|
|
webui:
|
|
|
|
|
build: ./webui
|
|
|
|
|
container_name: agent-system-webui
|
|
|
|
|
ports:
|
|
|
|
|
- "18180:8080"
|
2026-07-20 19:55:54 +02:00
|
|
|
environment:
|
|
|
|
|
# Empty by default (standalone/dev mode: reads/mutates local ACTIONS_DIR).
|
|
|
|
|
# Set on mirror nodes (see hosts/<node>/runtime/agent-system/) to make
|
|
|
|
|
# this panel read actions from world/actions.json and proxy approve/
|
|
|
|
|
# reject to the VPS control-plane instead.
|
|
|
|
|
CONTROL_PLANE_URL: ${CONTROL_PLANE_URL:-}
|
2026-05-15 23:29:59 +02:00
|
|
|
volumes:
|
|
|
|
|
- /opt/homelab:/opt/homelab
|
feat(observer): 3-state node liveness (fresh/stale/dead) + transitions + read-time net
Fixes the "dead node shown NOMINAL" silent outage: node status was set only by
events and never expired, so a node that crashed/lost connectivity stayed
"online" forever (chelsty-infra was online for 16d, piha ~6d). The only thing
that flipped status to offline was a node_offline event, which an unreachable
node can never emit.
Now node status is derived from freshness (now - last_seen), recomputed every
observer cycle (incl. cycles with no new events):
- always-on: fresh <=180s, stale 180-600s, dead >600s (3x the 60s heartbeat)
- remote/LTE (chelsty-*): fresh <=900s, stale 900-3600s, dead >3600s
Thresholds + tier logic live in ONE shared helper, services/control-plane/src/
liveness.py, imported by the observer and both operator UIs (bind-mounted into
the agent-system webui image). No 3x copy.
Transitions are not silent: the observer emits node_stale / node_offline /
node_online (recovery) events tagged source=observer (skipped on re-ingest so
they never reset last_seen), routed by the supervisor to alert_only actions.
Read-time safety net: both UIs recompute liveness from last_seen at request
time, so a stalled observer still surfaces dead nodes. Services inherit their
node's liveness (cascade, variant B) without mutating services.json.
Replaces the earlier binary NODE_OFFLINE_TTL_SECS flip.
Tests: liveness unit tests, observer 3-state + transitions/recovery/baseline +
self-event skip, operator_ui read-time net + cascade, supervisor node-event
routing. 89 passed. docker compose config valid for both stacks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 20:07:25 +02:00
|
|
|
# Shared liveness logic — single source of truth, bind-mounted read-only
|
|
|
|
|
# so this image does not carry a second copy of the thresholds/tier logic.
|
|
|
|
|
# web.py imports it defensively; absence degrades to observer status only.
|
|
|
|
|
- ../control-plane/src/liveness.py:/app/liveness.py:ro
|
2026-05-15 23:29:59 +02:00
|
|
|
depends_on:
|
|
|
|
|
- redis
|
|
|
|
|
restart: unless-stopped
|
|
|
|
|
|
|
|
|
|
runtime-materializer:
|
|
|
|
|
build: ./runtime-materializer
|
|
|
|
|
container_name: agent-system-runtime-materializer
|
|
|
|
|
environment:
|
|
|
|
|
REDIS_HOST: redis
|
|
|
|
|
REDIS_PORT: "6379"
|
|
|
|
|
HOMELAB_WORLD_ROOT: /opt/homelab/world
|
|
|
|
|
WORLD_DIR: /opt/homelab/world
|
|
|
|
|
MATERIALIZE_INTERVAL: "10"
|
|
|
|
|
volumes:
|
|
|
|
|
- /opt/homelab:/opt/homelab
|
|
|
|
|
depends_on:
|
|
|
|
|
- redis
|
|
|
|
|
restart: unless-stopped
|
2026-05-16 21:53:06 +02:00
|
|
|
|
|
|
|
|
telegram-bot:
|
|
|
|
|
build: ./telegram-bot
|
|
|
|
|
container_name: agent-system-telegram-bot
|
|
|
|
|
environment:
|
|
|
|
|
TELEGRAM_BOT_TOKEN: ${TELEGRAM_BOT_TOKEN}
|
|
|
|
|
TELEGRAM_ALLOWED_USER_IDS: ${TELEGRAM_ALLOWED_USER_IDS}
|
2026-05-17 23:42:52 +02:00
|
|
|
CONTROL_PLANE_URL: ${CONTROL_PLANE_URL:-http://webui:8080}
|
|
|
|
|
ENABLE_LLM_FALLBACK: ${ENABLE_LLM_FALLBACK:-false}
|
|
|
|
|
OPENCLAW_BASE_URL: ${OPENCLAW_BASE_URL}
|
2026-05-16 21:53:06 +02:00
|
|
|
ACTIONS_ROOT: /opt/homelab/actions
|
|
|
|
|
volumes:
|
|
|
|
|
- /opt/homelab:/opt/homelab
|
2026-05-17 23:42:52 +02:00
|
|
|
restart: on-failure
|