From f6a016f551bf7236851267bc60e8b9493131c270 Mon Sep 17 00:00:00 2001 From: oskar Date: Tue, 4 Aug 2026 14:52:23 +0200 Subject: [PATCH] =?UTF-8?q?fix(vps):=20M1=20mitigation=20=E2=80=94=20NODE?= =?UTF-8?q?=5FTYPE=3Dlte=5Fnode=20disables=20unfiltered=20prune?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit node-agent calls docker containers.prune() with no filters every CHECK_INTERVAL (60 s on VPS). The Docker API removes EVERY non-running container regardless of RestartPolicy or compose labels — this destroyed ollama@solaria on 2026-07-30 (19 s after an operator `docker stop`). On VPS the blast radius is worse: humanai-mailer and humanai-landing have no compose definition in this repo (recreated by hand from `docker inspect`), so a pruned container there is an irreversible loss of the only config source. self.node_type is read only by run_safe_cleanup() (node_agent.py:648,654) and two log lines (250, 1103). Verified additionally for VPS: the control-plane filesystem rotation and health probe in run_once() are gated on `node_name == VPS_NODE_NAME`, not node_type — so they keep running. Monitoring, event shipping and action dispatch are likewise unaffected. Temporary — remove once R1 (explicit-enumeration prune) is deployed. Refs docs/incidents/2026-07-30-ollama-solaria-vanish.md §7 M1. Co-Authored-By: Claude Opus 5 (1M context) --- .../runtime/node-agent/docker-compose.override.yml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/hosts/vps/runtime/node-agent/docker-compose.override.yml b/hosts/vps/runtime/node-agent/docker-compose.override.yml index ec4545f..ac56513 100644 --- a/hosts/vps/runtime/node-agent/docker-compose.override.yml +++ b/hosts/vps/runtime/node-agent/docker-compose.override.yml @@ -3,6 +3,18 @@ services: environment: - NODE_NAME=vps - CHECK_INTERVAL=60 + # TEMPORARY mitigation (M1) for the unfiltered-prune incident + # (docs/incidents/2026-07-30-ollama-solaria-vanish.md §7). node-agent runs + # `docker container prune()` with NO filters every CHECK_INTERVAL, and the + # Docker API removes EVERY non-running container regardless of restart + # policy or compose labels — this already destroyed ollama@solaria. On VPS + # the loss is worse: humanai-mailer and humanai-landing have no compose + # definition in this repo, so a pruned container cannot be recreated. + # node_type is read ONLY by run_safe_cleanup() (plus two log lines), so + # lte_node disables cleanup and nothing else — monitoring, event shipping + # and action dispatch keep working. + # REMOVE once R1 (explicit-enumeration prune) is deployed to VPS. + - NODE_TYPE=lte_node # host network mode: node-agent on VPS shares the host's network namespace # so that localhost:18180 resolves to the control-plane's exposed port. # Without this, localhost inside the container is the container's own loopback