oskar/homelab-codex-ws
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

fix(supervisor): drop dead mqtt_unreachable restart trigger

Browse source 
The observer never creates incidents with trigger_type=mqtt_unreachable,
so the CONTAINER_RESTART_TRIGGERS branch for it could never fire (recon
D15). stability-agent keeps emitting the event; it just never becomes an
incident. Tests: 145 passed.

SERVICE_NAMES in ai-cluster's service_ops_worker.py (the other dead
constant from the plan) is NOT touched: that code is legacy-frozen in
the unmerged task/ai-cluster-solaria worktree and nothing on this branch
references it (verified by grep — only the recon and plan docs mention it).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
oskar 2026-07-29 18:48:46 +02:00
parent e67868f5da
commit d03288e5b3
1 changed files with 9 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
services/control-plane/src/supervisor.py
View file

@ -33,10 +33,13 @@ try:
except Exception:
NODE_ALIAS_MAP = {}
# Event trigger types that should result in a lightweight container_restart
# rather than a full redeploy. The container is present but not running,
# or a dependency (MQTT) is unreachable — a restart is the right first step.
CONTAINER_RESTART_TRIGGERS = {"containers_not_running", "mqtt_unreachable"}
# Incident trigger types that should result in a lightweight container_restart
# rather than a full redeploy: the container is present but not running.
# mqtt_unreachable was removed 2026-07-28: the observer never creates incidents
# with that trigger_type, so the branch was dead code (recon
# docs/architecture/RECON-multiagent-2026-07-27.md, D15). stability-agent still
# emits the mqtt_unreachable *event*; it just never becomes an incident.
CONTAINER_RESTART_TRIGGERS = {"containers_not_running"}
# Nodes where automatic disk_cleanup actions must NOT be generated.
# On chelsty nodes disk fullness is overwhelmingly caused by Frigate recordings
@ -303,7 +306,7 @@ class Supervisor:
"""
Return the trigger_type of the active incident for a service, or None.
trigger_type is set by the observer when it creates an incident from
a specific event type (e.g. 'containers_not_running', 'mqtt_unreachable').
a specific event type (e.g. 'containers_not_running').
"""
svc_info = self.actual_state["services"].get(svc_key, {})
incident_id = svc_info.get("incident_id")
@ -409,8 +412,7 @@ class Supervisor:
if trigger_type in CONTAINER_RESTART_TRIGGERS:
# Lightweight remediation: the container exists but is not running
# (containers_not_running) or its MQTT dependency is unreachable
# (mqtt_unreachable). A docker restart is sufficient and low-risk.
# (containers_not_running). A docker restart is sufficient and low-risk.
container_name = self._get_container_name(service)
action = {
"action_id": action_id,