chore(vps): remove gokapi (operator decision 2026-08-26)

gokapi was VPS desired state (hosts/vps/services.yaml) with no matching
runtime on the node — a real deployment gap left open at the end of the
2026-08-26 recon session ("redeploy-vps-gokapi pozostawiony — realna
luka wdrożeniowa", docs/sessions/2026-08-26.md). Operator decision this
session: drop it instead of deploying it. Verified zero footprint on
VPS: no data, no container, no image, no /opt/homelab/config/gokapi.

Removed the desired-state entry from hosts/vps/services.yaml and the
services/gokapi/ compose stack. No hosts/vps/runtime/gokapi override
existed to remove.

Grepped the repo for dangling references: jobs/deploy-runner/tests and
services/control-plane/tests use "gokapi" only as an arbitrary example
service name in synthetic tmp_path fixtures (not reading the real
services/gokapi/ directory) — unaffected, left as-is. Fixed one stale
mention in services/control-plane/env.example's example-services
comment. kb/ and docs/sessions/ mentions (service doc, cutover
runbook, an open backlog item, prior session logs) are historical/
narrative record, not code or active config — left untouched, out of
this task's scope; flagged as a follow-up below.

Full control-plane (183), node-agent (70), and deploy-runner (44) test
suites pass unchanged.

Follow-up (not done here — kb/ editing is out of scope for this
worktree task): kb/decisions/backlog-aktywne.md still has an open
"gokapi: deploy-node VPS rzuca błąd — brakujący .env" entry that is now
moot and should be closed/removed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017WDKj5LRY8vdQMx57dfNnu

hosts/vps/services.yaml

@@ -60,23 +60,6 @@ services:
       data_path: /opt/homelab/data/fleet-prometheus
       logs_path: /opt/homelab/logs/fleet-prometheus
 
-  gokapi:
-    role: public-file-share
-    deployment_model: docker-compose
-    exposure: public
-    offline_required: false
-    depends_on:
-      local: []
-      external: []
-    ports:
-      - name: http
-        container_port: 53842
-        protocol: tcp
-    runtime:
-      config_path: /opt/homelab/config/gokapi
-      data_path: /opt/homelab/data/gokapi
-      logs_path: /opt/homelab/logs/gokapi
-
   stability-agent:
     role: node-watchdog          # read-only docker.sock watchdog, emits filesystem events
     # Deploys via its own deploy-local.sh, outside the declarative pipeline

services/control-plane/env.example

@@ -1,6 +1,6 @@
 # Copy to .env next to docker-compose.yml (gitignored); docker compose picks
 # it up automatically. Same convention as services/fleet-prometheus,
-# services/llm-gateway, services/gokapi, services/ollama.
+# services/llm-gateway, services/ollama.
 
 # Tailscale IP of the VPS node. The operator-ui port (18180) is published as
 # ${TAILSCALE_BIND_IP}:18180:8080 so the mesh-facing bind is never 0.0.0.0.

services/control-plane/src/supervisor.py

@@ -16,6 +16,23 @@ def _atomic_write_json(path: Path, data) -> None:
         os.fsync(f.fileno())
     os.replace(tmp, path)
 
+
+def _parse_ts(ts) -> float:
+    """Return a Unix timestamp float from ts (int/float, or an ISO-8601 string
+    as stability-agent / events.py write it). Mirrors observer.observer._parse_ts
+    — duplicated rather than imported to keep supervisor.py's dependency on the
+    observer module (an unrelated component with its own import-time sys.path
+    manipulation) at zero. Returns 0.0 on None/unparseable input."""
+    if ts is None:
+        return 0.0
+    if isinstance(ts, (int, float)):
+        return float(ts)
+    try:
+        from datetime import datetime
+        return datetime.fromisoformat(str(ts).replace("Z", "+00:00")).timestamp()
+    except Exception:
+        return 0.0
+
 # Constants and Paths
 RUNTIME_PATH = os.getenv("RUNTIME_PATH", "/opt/homelab")
 WORLD_DIR = Path(RUNTIME_PATH) / "world"
@@ -433,13 +450,51 @@ class Supervisor:
         service = drift["service"]
         trigger_type = drift.get("trigger_type")
 
-        # Choose action type first so we can build the stable, deterministic ID.
-        # Stable IDs mean reconcile is truly idempotent: the same drift always
-        # produces the same filename, so we never create duplicates even across
-        # restarts of the supervisor.
+        # Choose action type first so we can build the ID.
+        #
+        # container_restart IDs carry a suffix so two DIFFERENT incidents for
+        # the same node+service never collide in cancelled/completed/failed
+        # (2026-08-26: a generic containers_not_running restart and a later
+        # ha-diag-agent shadow-mode restart both used the bare
+        # container-restart-piha-homeassistant id and overwrote each other's
+        # history — worked around manually that session, see
+        # docs/sessions/2026-08-26.md).
+        #
+        # The suffix is the triggering incident's started_at, NOT time.time()
+        # at generation time: reconcile() calls _generate_recommendation on
+        # every loop iteration while the drift persists, and the pending/
+        # approved/running existence check below is what makes that
+        # idempotent — it only works if repeated calls for the SAME ongoing
+        # incident produce the SAME action_id. started_at stays fixed for the
+        # life of one incident (observer._handle_incident only bumps
+        # last_occurrence/occurrence_count on repeat occurrences) and changes
+        # only when a new incident is opened for that service — exactly the
+        # cases we want "same id" and "different id" for, respectively.
+        #
+        # No incident record → bare id (the pre-fix format, no suffix at
+        # all), NOT time.time(). A missing/unlinked incident_id is not just
+        # a malformed-data edge case: observer._prune_stale_world Case 3
+        # (commit 71a7af5) clears service.incident_id after 24h of event
+        # silence even while the underlying drift is still ongoing, so this
+        # path is hit by a live, still-restarting service. time.time() would
+        # mint a new action_id — and a new pending file — on every single
+        # reconcile() tick, defeating the dedup check below entirely. The
+        # bare id has no incident to distinguish "same" from "different"
+        # occurrences by, but it is at least stable across calls, which is
+        # what idempotency here actually requires.
         if trigger_type in CONTAINER_RESTART_TRIGGERS:
-            action_id = f"container-restart-{node}-{service}"
+            incident_id = self.actual_state["services"].get(drift["svc_key"], {}).get("incident_id")
+            incident = self.actual_state["incidents"].get(incident_id, {}) if incident_id else {}
+            started_ts = int(_parse_ts(incident.get("started_at")))
+            if started_ts:
+                action_id = f"container-restart-{node}-{service}-{started_ts}"
+            else:
+                action_id = f"container-restart-{node}-{service}"
         else:
+            # redeploy IDs stay bare (node-service) — out of scope for this
+            # fix (see commit message: no observed collision here yet), and
+            # _cancel_resolved_pending_actions/_ha_action_recently_completed
+            # do not key off redeploy ids so nothing here depends on it.
             action_id = f"redeploy-{node}-{service}"
 
         # Skip if an action for this ID is already live in any active state

services/control-plane/tests/test_observer_container_events.py

@@ -178,10 +178,15 @@ def test_supervisor_generates_container_restart_from_observer_output(
     sup = Supervisor()
     sup.reconcile()
 
-    action_id = "container-restart-piha-paperless"
-    action_path = actions / "pending" / f"{action_id}.json"
-    assert action_path.exists(), "supervisor did not generate a container_restart"
+    # action_id carries a per-incident timestamp suffix (2026-08-26 fix: two
+    # different incidents for the same node/service must never collide in
+    # cancelled/completed/failed — see supervisor._generate_recommendation),
+    # so match by prefix rather than an exact id.
+    matches = list((actions / "pending").glob("container-restart-piha-paperless-*.json"))
+    assert len(matches) == 1, "supervisor did not generate exactly one container_restart"
+    action_path = matches[0]
     action = json.loads(action_path.read_text())
+    assert action["action_id"] == action_path.stem
     assert action["type"] == "container_restart"
     assert action["node"] == "piha"
     assert action["service"] == "paperless"

services/control-plane/tests/test_supervisor_action_id_uniqueness.py

@@ -0,0 +1,181 @@
+"""action_id uniqueness for container_restart (2026-08-26 fix).
+
+Before this fix, _generate_recommendation() built container_restart action
+ids as the bare `container-restart-<node>-<service>` — no timestamp, no
+incident reference. Two DIFFERENT incidents for the same node+service (e.g. a
+generic containers_not_running restart followed, after recovery and a later
+recurrence, by another restart) produced the identical id. Once the first
+action reached cancelled/completed/failed, the second action's move into that
+same directory overwrote the first one's history file outright — this is
+exactly what happened on 2026-08-26 to a shadow-mode HA-websocket restart
+colliding with an unrelated 08-06 entry (docs/sessions/2026-08-26.md), worked
+around manually that session by hand-renaming the file.
+
+The fix anchors the id's suffix to the triggering incident's `started_at`
+instead of wall-clock time-of-generation, so:
+  - repeated reconcile() calls for the SAME ongoing incident keep producing
+    the SAME action_id (required for the pending/approved/running dedup
+    check in _generate_recommendation to still work — see its comment), and
+  - a NEW incident (after the old one resolved) gets a NEW id, so its
+    eventual cancelled/completed/failed file cannot collide with the old
+    incident's.
+"""
+from __future__ import annotations
+
+import json
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
+import supervisor as supervisor_module  # noqa: E402
+from supervisor import Supervisor, CONTAINER_RESTART_TRIGGERS  # noqa: E402
+
+
+@pytest.fixture
+def sup(tmp_path, monkeypatch):
+    actions = tmp_path / "actions"
+    events = tmp_path / "events"
+    world = tmp_path / "world"
+    repo = tmp_path / "repo"
+    for d in (actions, events, world, repo / "inventory", repo / "hosts"):
+        d.mkdir(parents=True, exist_ok=True)
+    monkeypatch.setattr(supervisor_module, "ACTIONS_DIR", actions)
+    monkeypatch.setattr(supervisor_module, "EVENTS_DIR", events)
+    monkeypatch.setattr(supervisor_module, "WORLD_DIR", world)
+    monkeypatch.setattr(supervisor_module, "REPO_ROOT", repo)
+    s = Supervisor()
+    # _generate_recommendation only reads actual_state — bypass disk-backed
+    # _load_desired_state()/_load_actual_state() and set it directly.
+    s.actual_state = {"services": {}, "nodes": {}, "incidents": {}}
+    return s
+
+
+def _drift(node, service, trigger_type="containers_not_running"):
+    return {
+        "type": "unhealthy_service",
+        "svc_key": f"{node}/{service}",
+        "node": node,
+        "service": service,
+        "status": "unhealthy",
+        "trigger_type": trigger_type,
+    }
+
+
+def _seed_incident(sup, node, service, incident_id, started_at, status="active"):
+    svc_key = f"{node}/{service}"
+    sup.actual_state["services"][svc_key] = {
+        "node": node, "service": service, "status": "unhealthy",
+        "incident_id": incident_id,
+    }
+    sup.actual_state["incidents"][incident_id] = {
+        "id": incident_id, "status": status,
+        "trigger_type": "containers_not_running",
+        "started_at": started_at,
+        "last_occurrence": started_at,
+    }
+
+
+def _pending(tmp_path):
+    return list((tmp_path / "actions" / "pending").glob("*.json"))
+
+
+def test_repeated_calls_for_same_incident_produce_same_action_id_no_duplicate(sup, tmp_path):
+    """Simulates reconcile() calling _generate_recommendation() on every loop
+    iteration while the drift persists: must not spam a new pending action
+    each time."""
+    _seed_incident(sup, "piha", "paperless", "inc-1000-piha-paperless", started_at=1000)
+    drift = _drift("piha", "paperless")
+
+    sup._generate_recommendation(drift)
+    sup._generate_recommendation(drift)
+    sup._generate_recommendation(drift)
+
+    pending = _pending(tmp_path)
+    assert len(pending) == 1, f"expected exactly one pending action, got {[p.name for p in pending]}"
+    assert pending[0].name == "container-restart-piha-paperless-1000.json"
+
+
+def test_new_incident_after_old_completed_gets_different_action_id(sup, tmp_path):
+    """A second, later incident for the same node/service must not collide
+    with the first incident's already-completed action file."""
+    _seed_incident(sup, "piha", "paperless", "inc-1000-piha-paperless", started_at=1000)
+    drift = _drift("piha", "paperless")
+    sup._generate_recommendation(drift)
+
+    first_action_path = tmp_path / "actions" / "pending" / "container-restart-piha-paperless-1000.json"
+    assert first_action_path.exists()
+
+    # First incident's action ran to completion (moved out of pending/ by the
+    # executor in real life; simulate that here).
+    completed_dir = tmp_path / "actions" / "completed"
+    completed_dir.mkdir(parents=True, exist_ok=True)
+    first_action = json.loads(first_action_path.read_text())
+    first_action["status"] = "completed"
+    (completed_dir / first_action_path.name).write_text(json.dumps(first_action))
+    first_action_path.unlink()
+
+    # A NEW incident recurs later for the same node+service (old one resolved
+    # in between — new incident_id, new started_at).
+    _seed_incident(sup, "piha", "paperless", "inc-2000-piha-paperless", started_at=2000)
+    sup._generate_recommendation(drift)
+
+    second_action_path = tmp_path / "actions" / "pending" / "container-restart-piha-paperless-2000.json"
+    assert second_action_path.exists()
+    # The first incident's completed record must be untouched — not overwritten.
+    assert json.loads((completed_dir / "container-restart-piha-paperless-1000.json").read_text())["status"] == "completed"
+
+
+def test_fallback_to_bare_id_when_incident_record_missing(sup, tmp_path):
+    """Malformed/missing incident data (incident_id set on the service but no
+    matching record in incidents.json) must still produce a usable
+    (non-crashing) action_id, not block remediation. The fallback must be the
+    bare pre-fix id — NOT a time.time() suffix — since this path is hit
+    naturally (not just on malformed data): observer._prune_stale_world
+    Case 3 (commit 71a7af5) clears service.incident_id after 24h of event
+    silence while the drift is still ongoing, so a time.time() suffix would
+    mint a new id on every reconcile() tick forever."""
+    svc_key = "piha/paperless"
+    sup.actual_state["services"][svc_key] = {
+        "node": "piha", "service": "paperless", "status": "unhealthy",
+        "incident_id": "inc-missing",
+    }
+    # Deliberately no matching entry in sup.actual_state["incidents"].
+    drift = _drift("piha", "paperless")
+
+    sup._generate_recommendation(drift)
+
+    pending = _pending(tmp_path)
+    assert len(pending) == 1
+    assert pending[0].name == "container-restart-piha-paperless.json"
+
+
+def test_fallback_bare_id_stable_across_repeated_calls(sup, tmp_path):
+    """Same missing-incident-record scenario, but simulating reconcile()
+    calling _generate_recommendation() on every loop iteration while the
+    drift persists: must not spam a new pending action each time, exactly
+    like the has-an-incident-record case above."""
+    svc_key = "piha/paperless"
+    sup.actual_state["services"][svc_key] = {
+        "node": "piha", "service": "paperless", "status": "unhealthy",
+        "incident_id": "inc-missing",
+    }
+    drift = _drift("piha", "paperless")
+
+    sup._generate_recommendation(drift)
+    sup._generate_recommendation(drift)
+    sup._generate_recommendation(drift)
+
+    pending = _pending(tmp_path)
+    assert len(pending) == 1, f"expected exactly one pending action, got {[p.name for p in pending]}"
+    assert pending[0].name == "container-restart-piha-paperless.json"
+
+
+def test_redeploy_action_id_stays_bare(sup, tmp_path):
+    """Non-container_restart drift (redeploy path) is out of scope for this
+    fix and keeps its existing bare node-service id."""
+    drift = _drift("piha", "outline", trigger_type="service_unhealthy")
+    sup._generate_recommendation(drift)
+
+    assert (tmp_path / "actions" / "pending" / "redeploy-piha-outline.json").exists()

services/gokapi/README.md

@@ -1,5 +0,0 @@
-# gokapi
-
-Lekki self-hosted "Firefox Send" alternative — link do jednego pliku, na zewnątrz, z limitem pobrań/czasu. **Osobny serwis od Nextclouda, celowo.**.
-
-Dokumentacja: [kb/services/gokapi.md](../../kb/services/gokapi.md)

services/gokapi/docker-compose.yml

@@ -1,43 +0,0 @@
-# Gokapi — public file-share (Firefox Send-style), separate from Nextcloud.
-#
-# Nextcloud stays the private "twierdza" (mesh/kapala.org only). Gokapi is
-# the OPPOSITE end of the spectrum on purpose: a small, single-container,
-# genuinely public link-share for sending someone outside the mesh a file.
-# Runs on VPS (Hetzner, public host) — home nodes stay untouched.
-services:
-  gokapi:
-    # Pinned to the current stable release (v2.2.4, verified against
-    # github.com/Forceu/Gokapi releases + Docker Hub f0rc3/gokapi tags on
-    # 2026-07-09) — never :latest, so upgrades are a deliberate git diff,
-    # same convention as nextcloud.
-    image: f0rc3/gokapi:v2.2.4
-    container_name: gokapi
-    restart: unless-stopped
-    env_file:
-      - .env
-    volumes:
-      # /opt/homelab/data convention. `data` holds uploaded files (churns
-      # constantly — links expire and get deleted); `config` holds
-      # config.json + the E2E encryption master key. See README backup note:
-      # config is the part that actually needs backing up.
-      - /opt/homelab/data/gokapi/data:/app/data
-      - /opt/homelab/data/gokapi/config:/app/config
-    ports:
-      # PUBLIC reachability is via npm@VPS + share.okit.pl, NOT this bind.
-      # Bound ONLY to the VPS Tailscale interface (TAILSCALE_BIND_IP), never
-      # 0.0.0.0 — same defense-in-depth convention as fleet-prometheus: the
-      # raw port does not exist on the public Hetzner IP (135.181.153.108)
-      # at all. npm, running as its own container on the same host, reaches
-      # gokapi via Docker hairpin NAT through this real interface IP
-      # (loopback would NOT work for that trick — see fleet-prometheus /
-      # nextcloud for the same pattern). Requires .env (from env.example)
-      # next to this file at deploy.
-      - "${TAILSCALE_BIND_IP}:53842:53842"
-    # Image ships curl (alpine-based), so an in-container check works here —
-    # unlike vikunja's image, which has neither curl nor wget.
-    healthcheck:
-      test: ["CMD", "curl", "-fs", "--max-time", "5", "http://127.0.0.1:53842/"]
-      interval: 30s
-      timeout: 10s
-      retries: 5
-      start_period: 15s

services/gokapi/env.example

@@ -1,47 +0,0 @@
-# Gokapi host-local binds + disk-protection tuning — copy to .env (gitignored)
-# next to docker-compose.yml and fill in real values. Never commit .env.
-# No auth secrets here: Gokapi has no headless admin env vars — the admin
-# account is created in the one-time /setup web wizard (see README).
-
-# Tailscale IP of the VPS node (ubuntu-4gb-hel1-1). Bind the gokapi port
-# ONLY to this interface — never 0.0.0.0. The public internet never reaches
-# port 53842 directly; share.okit.pl reaches gokapi only through npm@VPS
-# (which runs on the same host and connects to this IP via Docker hairpin
-# NAT — see docker-compose.yml). Verify when rebuilding the host:
-# tailscale ip -4.
-TAILSCALE_BIND_IP=100.95.58.48
-
-TZ=Europe/Warsaw
-
-# --- Disk protection -------------------------------------------------------
-# VPS has an 80 GB disk shared with npm, outline, joplin, ai-cluster,
-# fleet-prometheus's TSDB, etc. — gokapi is NOT the only tenant, and it has
-# no S3 backend (local disk only, by design). Keep uploads small and leave
-# headroom so expiring links actually reclaim space before the disk fills.
-
-# Per-file upload cap in MB. 5120 = 5 GB. Upstream default is 102400
-# (100 GB), which would let a single upload fill the whole disk.
-GOKAPI_MAX_FILESIZE=5120
-
-# Refuse new uploads once free disk space drops below this many MB. Upstream
-# default is 400; raised here for the same reason as above.
-GOKAPI_MIN_FREE_SPACE=2048
-
-# npm@VPS and gokapi run on the same host; npm reaches gokapi's Tailscale-
-# bound port via Docker hairpin NAT, so the connection arrives from the
-# docker bridge gateway, not the real client IP. Trust that range so
-# X-Forwarded-For / IP logging work correctly — same lesson as nextcloud's
-# TRUSTED_PROXIES. Narrow to the exact /24 once the stack is up on the live
-# host: docker network inspect gokapi_default.
-GOKAPI_TRUSTED_PROXIES=172.16.0.0/12
-
-# --- NOT env vars — Gokapi has no headless config for these. Set them in
-# the one-time /setup wizard on first boot (see README Cutover checklist):
-# - Admin username/password                       (Authentication step)
-# - Storage backend: Local                         (Storage step — NOT S3,
-#                                                    per Oskar's decision)
-# - Encryption level: End-to-End (Level 3)          (Encryption step —
-#                                                    Oskar wants E2E ON)
-# - Default expiry / max downloads: there is NO global default in Gokapi —
-#   it's chosen per-upload in the web form. Use conservative values (e.g.
-#   7 days / 10 downloads) each time to keep the disk from filling.

services/gokapi/healthcheck.sh

@@ -1,27 +0,0 @@
-#!/bin/bash
-# Healthcheck for gokapi (public file-share, https://share.okit.pl via npm@VPS)
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-
-# The port is bound to the Tailscale interface only, so localhost won't answer.
-# Read the bind IP from .env (same file compose uses for the port mapping).
-if [ -f "$SCRIPT_DIR/.env" ]; then
-    # shellcheck disable=SC1091
-    source "$SCRIPT_DIR/.env"
-fi
-BIND_IP="${TAILSCALE_BIND_IP:-127.0.0.1}"
-
-# Container must be running
-if ! docker ps --filter "name=gokapi" --filter "status=running" | grep -qw "gokapi"; then
-    echo "[FAIL] gokapi container is not running"
-    exit 1
-fi
-
-# Web server must answer (setup wizard or login page, either is fine)
-if ! curl -sf --max-time 5 "http://${BIND_IP}:53842/" > /dev/null; then
-    echo "[FAIL] gokapi is not responding on ${BIND_IP}:53842"
-    exit 1
-fi
-
-echo "[OK] gokapi is healthy"
-exit 0

services/gokapi/service.yaml

@@ -1,34 +0,0 @@
-service:
-  name: gokapi
-  owner_node: vps
-  role: public-file-share       # Firefox Send-style link sharing, deliberately separate from Nextcloud
-  exposure: public               # public via npm@VPS (share.okit.pl). The container's own port binds
-                                  # to TAILSCALE_BIND_IP only, never 0.0.0.0 — npm is the sole public
-                                  # entry point (see docker-compose.yml). Same pattern as vikunja.
-  dependencies: []                # standalone; npm@VPS proxies to it but gokapi has no upstream deps
-  ports:
-    - container: 53842
-      host: 53842
-      protocol: tcp
-  healthcheck:
-    type: http
-    endpoint: http://localhost:53842/     # setup wizard until first admin login, then the login page
-    interval: 30s
-    timeout: 10s
-    retries: 5
-  restart_policy: unless-stopped
-  persistence:
-    paths:
-      - /opt/homelab/data/gokapi/data      # uploaded files — ephemeral by design, links expire
-      - /opt/homelab/data/gokapi/config    # config.json + E2E encryption master key — BACK THIS UP
-  runtime:
-    config_files:
-      - .env                    # host-local binds + disk-protection tuning (gitignored, from env.example)
-    env_vars:
-      - TAILSCALE_BIND_IP        # required — mesh-only port bind; npm@VPS reaches it via hairpin NAT
-      - GOKAPI_MAX_FILESIZE      # per-file cap in MB — keeps the shared VPS disk from filling
-      - GOKAPI_MIN_FREE_SPACE    # MB headroom before uploads are refused
-      - GOKAPI_TRUSTED_PROXIES   # npm@VPS as seen through Docker hairpin NAT (docker bridge subnet)
-      # No admin/auth env vars: Gokapi has no headless setup — admin account,
-      # storage backend, encryption level are all set in the /setup web
-      # wizard on first boot (deploy-time step, see README).