homelab-codex-ws/services
oskar 0526af1299 fix(node-agent): R1/R2/R3 — stop deleting deliberately stopped containers
R1 (critical): _prune_stopped_containers no longer calls containers.prune().
The unfiltered API call removes EVERY non-running container, ignoring
RestartPolicy and compose labels — that is what deleted ollama@solaria 19 s
after an operator `docker stop`. No filter argument can fix it (`until` filters
on creation time, not stop time). Replaced with explicit enumeration over
containers.list(all=True, filters={"status": "exited"}), skipping anything with
restart policy unless-stopped/always/on-failure or a com.docker.compose.project
label. A stopped managed service is recorded operator intent and now belongs to
the module's NEVER TOUCHED list; only one-off leftovers are removed. Dangling
image and build cache prune are unchanged.

R2 (high): _sd_card_rate_ok → _cleanup_rate_ok, applied to every cleanup-
eligible node type. ai_node and standard pruned every 60 s (1440/day, "0 MB
reclaimed" in practically every cycle) with no rate limit at all; they now share
sd_card's CLEANUP_INTERVAL_SECS (24 h) and mark the cleanup timestamp.

R3 (medium): removals are named in the log — WARNING with the container names
when the list is non-empty, INFO otherwise. The old line reported megabytes
only, which is why ollama's deletion looked identical to every no-op cycle.

Tests: services/node-agent/tests/test_safe_cleanup.py (22 cases) — the ollama
regression (exited + unless-stopped survives, prune() never called), all three
restart policies, compose-label protection, disposable-leftover removal, sweep
continues past a failed remove, WARNING-level naming, and the rate limit for
ai_node/standard/sd_card plus lte_node still doing nothing.

Refs docs/incidents/2026-07-30-ollama-solaria-vanish.md §7 R1-R3.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 14:55:26 +02:00
..
agent-system fix(actions): mirror pending z VPS na panel PIHA + mutate proxy do VPS — Action Queue byla slepa 2026-07-20 19:55:54 +02:00
brain-watchdog feat(brain-watchdog): poll Prometheus /api/v1/alerts as second alert source (Telegram) 2026-06-30 18:51:34 +02:00
control-plane fix(control-plane): redeploy wykonywalny — dispatch do host-side deploy-runnera 2026-08-03 18:26:30 +02:00
fleet-prometheus feat(kb): faza 3 krok 5 — cykliczny ingest (systemd timer) + alerting 2026-07-17 15:54:13 +02:00
forgejo fix(inventory): correct owner_node — forgejo→piha, mosquitto→vps (per 2026-07-02 verify) 2026-07-02 16:48:12 +02:00
gokapi feat(gokapi): config publicznego file-share na VPS (share.okit.pl, E2E, Tailscale-bind za npm@VPS, disk-protection) — do deployu 2026-07-09 17:29:28 +02:00
ha-diag-agent fix(ha-diag): node_name from env + fail-fast on unknown — evt-unknown-* files poisoned observer checkpoint (see d5139c9) 2026-07-15 19:48:58 +02:00
ha-mcp feat(ha-mcp): read-only MCP server (faza 2a) 2026-07-30 16:47:27 +02:00
home-assistant feat(ha-mcp): read-only MCP server (faza 2a) 2026-07-30 16:47:27 +02:00
kb-postgres feat(kb): faza 3 krok 2 — migracja 004 (document_summary) + pilot streszczeń A/B 2026-07-17 13:30:47 +02:00
kb-query test(kb-query): luki T1–T3 z test_fallback.py + status kalibracji ollama-piha (salvage S3+S4) 2026-07-30 16:40:24 +02:00
llm-gateway feat(llm-gateway): wciagniecie shadow-serwisu z PIHA do GitOps 2026-07-03 15:08:33 +02:00
mosquitto fix(hosts): align manifests with observed reality (recon B6/B7/C8/F20.5) 2026-07-29 19:15:55 +02:00
narty27 feat(narty27): statyczny hosting viz.html na PIHA (nginx + named volume) 2026-07-31 16:38:55 +02:00
nextcloud feat(kb): configi Paperless/Nextcloud wg 9 decyzji — NC na PIHA, domeny kapala, Redis requirepass, backup SOLARIA, NC pin 34 2026-07-09 16:17:27 +02:00
node-agent fix(node-agent): R1/R2/R3 — stop deleting deliberately stopped containers 2026-08-04 14:55:26 +02:00
node_exporter feat(kb): faza 3 krok 5 — cykliczny ingest (systemd timer) + alerting 2026-07-17 15:54:13 +02:00
npm Add node capability model 2026-05-11 20:46:50 +02:00
ollama fix(ollama): restore GPU reservation on SOLARIA, close 07-15 cutover docs 2026-07-16 14:43:29 +02:00
ollama-piha test(kb-query): luki T1–T3 z test_fallback.py + status kalibracji ollama-piha (salvage S3+S4) 2026-07-30 16:40:24 +02:00
paperless fix(paperless-worker): celery command bypassed manage.py + missing shared scratch dir 2026-07-12 20:51:04 +02:00
paperless-worker fix(paperless-worker): celery command bypassed manage.py + missing shared scratch dir 2026-07-12 20:51:04 +02:00
planner-agent fix+debug(planner-agent): use base_url (not api_base) for litellm.acompletion, add print [TEMP] 2026-05-28 13:07:58 +02:00
stability-agent fix(stability-agent): tag containers_not_running events with compose service 2026-07-29 19:22:07 +02:00
vikunja fix(vikunja): PUBLICURL na vikunja.kapala.org 2026-07-02 17:28:56 +02:00
zigbee2mqtt docs: compress CLAUDE.md + fix zigbee2mqtt coordinator docs 2026-05-29 14:17:23 +02:00
.gitkeep Add infrastructure standards and deployment conventions 2026-05-07 21:16:03 +02:00