oskar/homelab-codex-ws
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
homelab-codex-ws/CLAUDE.md
Oskar Kapala bf1415e4c1 feat(control-plane): route ha-diag-agent events through supervisor 
- 8 HA event types mapped to existing action types
- ha_websocket_dead → container_restart (homeassistant), 30-min cooldown
- 6 events → alert_only (entity_unavailable, integration_failed,
  automation_failing, update_available, recorder_lag,
  system_health_degraded), 1-hour cooldown
- ha_websocket_recovered → cancels matching pending container_restart
- state-aware suppression: skip HA events when homeassistant has an
  active containers_not_running incident < 5 min ago (avoids alert
  storms during HA restarts/updates)
- location_tag preserved through action pipeline for per-house
  telegram alerts
- executor: alert_only acknowledged as no-op success
- 18 tests covering all 8 event types, suppression, cooldown,
  dedup, location_tag, recovery cancellation
- CLAUDE.md: supervisor event routing table added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 15:59:23 +02:00

6.9 KiB
Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What This Repo Is

GitOps-lite orchestration for a distributed homelab. The repo is the source of truth for infrastructure definitions; runtime state lives at /opt/homelab/ on each execution node and is never committed.

Node Roles

Host Role
SATURN Primary control node — only node where commits are made
SOLARIA GPU/compute/AI workloads
PIHA Infra, monitoring
VPS Public ingress, reverse proxy, control plane host
CHELSTY-INFRA LTE edge hypervisor (site: chelsty); Zigbee2MQTT, Mosquitto, stability-agent — offline-first
CHELSTY-HA LTE Home Assistant VM (site: chelsty); connects to CHELSTY-INFRA MQTT broker — offline-first

All nodes communicate over Tailscale. CHELSTY-INFRA and CHELSTY-HA have an intermittent LTE uplink; their services must never depend on SATURN, VPS, or Forgejo at runtime. Full node capabilities: hosts/<node>/capabilities.yaml.

Deployment

scripts/deploy/deploy.sh                        # fresh deploy on current node
scripts/deploy/deploy.sh --resume              # resume after interruption
scripts/deploy/deploy.sh --stage verify        # specific stage only
scripts/deploy/deploy.sh --service mosquitto   # specific service only
./scripts/deploy/deploy-control-plane.sh --ssh # SATURN/SOLARIA → VPS
./scripts/deploy/deploy-node.sh chelsty-infra  # CHELSTY nodes (individually)
./scripts/bootstrap/prepare-node.sh            # general node bootstrap
./scripts/bootstrap/chelsty-runtime.sh         # CHELSTY-specific bootstrap

Pipeline stages: prepare → validate → deploy → verify → diagnose (on failure) → complete. Stage state persisted in /opt/homelab/state/deploy/.

Service Structure

Every service must follow this layout:

services/<service>/
├── docker-compose.yml
├── service.yaml       # Machine-readable contract (primary source of truth for agents)
├── README.md
├── env.example        # Template — never commit actual secrets
└── healthcheck.sh     # Returns 0 (healthy) or 1 (unhealthy)

service.yaml defines owner_node, exposure, dependencies, healthcheck, restart_policy, persistence.paths, and runtime.env_vars. This is what AI agents read to understand how to manage a service.

Host-specific runtime config and secrets live at /opt/homelab/config/<service>/ on the target node (not in Git). Docker Compose overrides are version-controlled at hosts/<node>/runtime/<service>/docker-compose.override.yml in this repo and applied during deployment.

Agent System Architecture

The platform uses a multi-agent model with human-in-the-loop for destructive actions:

  1. Stability Agent (services/stability-agent/) — Per-node watchdog. Monitors Docker containers, disk, Tailscale, MQTT. Emits filesystem events. Does NOT restart services autonomously.
  2. Observer (services/control-plane/src/) — Synthesizes world state from events into /opt/homelab/world/{nodes,services,deployments,incidents}.json.
  3. Supervisor — Detects drift between desired state (from hosts/*/services.yaml) and actual state (from Observer output). Writes pending action JSON files.
  4. Executor — Executes actions only after they transition to approved.
  5. Operator UI + Telegram Bot — Operators review and approve/reject pending actions.

Action approval flow

Agent → /opt/homelab/actions/pending/<id>.json
      → Telegram notification → Operator approves
      → /opt/homelab/actions/approved/<id>.json
      → Executor runs → completed / failed

Agents must never execute destructive actions (restarts, deploys, config changes) without a corresponding approved action file.

Event System

Events are append-only JSON lines at /opt/homelab/events/YYYY-MM-DD/<node>/events.jsonl.

Emit via scripts/lib/events.sh (shell) or scripts/lib/events.py (Python).

Normalized event types: deployment_started/completed/failed, service_unhealthy/recovered, node_offline/online, healthcheck_failed, remediation_started/completed.

Supervisor event routing table

Event type Source Action generated Cooldown
containers_not_running stability-agent container_restart dedup via stable ID
mqtt_unreachable stability-agent container_restart dedup via stable ID
service_unhealthy / other stability-agent redeploy dedup via stable ID
disk_pressure (high) stability-agent disk_cleanup dedup via stable ID
ha_websocket_dead ha-diag-agent container_restart (homeassistant) 30 min after completion
ha_websocket_recovered ha-diag-agent cancels matching restart —
ha_integration_failed ha-diag-agent alert_only 1 hour
ha_entity_unavailable_long ha-diag-agent alert_only 1 hour
ha_automation_failing ha-diag-agent alert_only 1 hour
ha_update_available ha-diag-agent alert_only 1 hour
ha_recorder_lag ha-diag-agent alert_only 1 hour
ha_system_health_degraded ha-diag-agent alert_only 1 hour

HA events are routed directly from the events directory by the supervisor (not via world-state drift loop) to avoid conflicts with stability-agent's independent container health tracking. HA events are suppressed if homeassistant had a containers_not_running incident within the last 5 minutes (planned restart/update in progress).

Discovery Entry Points for Agents

When exploring the system, use these files in order:

  1. inventory/topology.yaml — node list, roles, mesh type
  2. hosts/<node>/capabilities.yaml — hardware and software constraints
  3. hosts/<node>/services.yaml — desired services and exposure classes for that host
  4. services/<service>/service.yaml — operational contract for a service

CHELSTY-Specific Rules

  • Zigbee coordinator is SLZB-06U over TCP (192.168.1.105:6638, ezsp adapter). Never use /dev/ttyUSB0.
  • CHELSTY nodes run docker-compose v1 (1.29.2) — use docker-compose (hyphenated), not docker compose.
  • Critical backup sets: HA config+data, Zigbee2MQTT config+db+network key, Mosquitto config+persistence, SLZB-06U coordinator state.

Runtime Path Conventions

/opt/homelab/ layout on each node:

  • data/<service>/ — persistent volumes
  • config/<service>/ — secrets and host-local overrides (not in Git)
  • logs/<service>/ — service logs
  • state/ — deployment stage markers, agent heartbeats
  • events/ — append-only event store
  • world/ — Observer output (synthesized state)
  • actions/ — pending / approved / running / completed / failed

Naming Conventions

  • Hosts: ALL CAPS (SATURN, PIHA)
  • Services: kebab-case (stability-agent, zigbee2mqtt)
  • Container names must match service names
  • Always restart: unless-stopped unless service.yaml says otherwise