_generate_recommendation() built container_restart ids as the bare
container-restart-<node>-<service>. Two DIFFERENT incidents for the
same node+service (e.g. a generic containers_not_running restart,
later followed — after recovery and recurrence — by an unrelated
restart for the same service) produced the identical id. Once the
first action reached cancelled/completed/failed, the second action's
own transition into that same directory silently overwrote the first
one's history file. This is exactly what happened 2026-08-26 to a
shadow-mode HA-websocket restart colliding with an unrelated 08-06
entry (docs/sessions/2026-08-26.md) — worked around by hand-renaming
the file that session.
Fix: suffix the id with the triggering incident's started_at —
container-restart-<node>-<service>-<unixts> — NOT time.time() at
generation call time. reconcile() calls _generate_recommendation() on
every loop iteration while the drift persists, and the pending/
approved/running existence check immediately below is what makes that
idempotent; it only works if repeated calls for the SAME ongoing
incident produce the SAME id. started_at is fixed for an incident's
whole life (observer._handle_incident only bumps
last_occurrence/occurrence_count on repeat occurrences — see
COMMIT-1-adjacent code) and changes only when a genuinely new incident
opens for that service, which is exactly "same id while ongoing,
different id on recurrence". Falls back to time.time() if the
incident record is missing/malformed so a restart is still generated.
Scope: only the generic CONTAINER_RESTART_TRIGGERS path
(_generate_recommendation). Left unchanged, deliberately:
- redeploy-<node>-<service> ids — no observed collision, out of
scope for this fix (flagged as a latent follow-up below).
- The HA-specific container-restart-<node>-homeassistant id used by
_generate_ha_container_restart / _generate_ha_shadow_alert /
_cancel_ha_container_restart: these three functions rely on an
exact-match lookup of that fixed id (cooldown check via
_ha_action_recently_completed, and the cancel path finding the
specific pending file to move) — adding a suffix there would
break both without a broader refactor to prefix-glob lookups.
- alert-ha-*/alert-node-* ids: _ha_action_recently_completed also
exact-matches these for cooldown dedup; a suffix would defeat
cooldown entirely (every occurrence would look "new").
node-agent idempotency gate confirmed unaffected: _already_processed()
in node_agent.py does a full-string action_id match against
processed-actions/<id>.done, guarding against RE-processing the exact
same dispatched action file (e.g. a duplicate rsync delivery) — not
against a new action_id for a new occurrence of the same service. A
suffixed id is legitimately a new action to node-agent, which is the
correct behavior (a genuine new incident should actually restart the
container again).
Tests: new test_supervisor_action_id_uniqueness.py covers (1) repeated
_generate_recommendation() calls for the same ongoing incident produce
the same id and do not duplicate the pending file, (2) a new incident
after the old one completed gets a different id and does not overwrite
the old completed record, (3) fallback to time.time() when the
incident record is missing, (4) redeploy ids stay bare. Updated
test_observer_container_events.py's end-to-end assertion to match by
prefix instead of exact filename. Full control-plane suite: 183
passed; node-agent suite: 70 passed (unchanged, confirming the
idempotency gate needed no code change).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017WDKj5LRY8vdQMx57dfNnu
17ac070b46