Per-node checkpoint now stores the last-processed event TIMESTAMP (int epoch)
instead of a file path compared lexically. A file is "new" iff its timestamp
(parsed from evt-<node>-<unixts>-<type>-<svc>.json, mtime fallback) exceeds the
node's checkpoint; processing is ordered by timestamp, not path.
Root cause (PIHA dead ~34d, 2026-07-12): a stray evt-unknown-<ts>-… file landed
in events/piha/, lexically greater than every evt-piha-… name. The lexical
checkpoint pinned there, so every genuinely newer piha event sorted "before" it
and was skipped forever. Event backlog grew to 7344 files, last_seen frozen,
shadow-read logged false SHADOW_LIVENESS_MISMATCH event=dead prom=up.
- _event_ts_from_path: filename epoch, mtime fallback; NEVER returns 0 for an
existing file (0 == "older than checkpoint" == the poison).
- _checkpoint_ts_from_value: graceful migration of pre-fix path-string
checkpoints (and the older last_processed_file format) to int epochs;
unparseable → 0 (reprocess all — safe, process_event is idempotent on
last_seen/world_state; bias to reprocess, never to skip).
- Preserved: quarantine of bad events, observer-source re-ingest guard.
- Regression tests (test_incident_lifecycle.py section 9): lexically-smaller-
but-newer processed, unparseable name falls back to mtime (not wedged),
ts-not-path ordering, both checkpoint-format migrations, helper units.
Separate bug filed in backlog (not fixed here): ha-diag-agent emits node=
"unknown" events (config.py node_name default) into another node's dir when
NODE_NAME reaches the compose volume path but not the app env — the source of
the poison file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two config bugs found on the already-deployed split-host OCR worker
(module 3): (1) `command: celery ...` was routed through manage.py by
the image entrypoint because it didn't start with "/" — fixed with an
absolute gosu+celery path. (2) SCRATCH_DIR (/tmp/paperless) was not
shared over NFS like data/media/consume, so tasks picked up by
worker@SOLARIA instead of worker@PIHA failed with "File not found" —
fixed by adding a paperless_scratch NFS volume/bind mount on both
sides.
Verified live on PIHA + SOLARIA: test PDFs dropped into consume/ were
split across both workers, the SOLARIA-picked task completed OCR with
zero File not found errors, test documents cleaned up afterward.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
token/list-hosts/list-certs/set-cert/create-host, dry-run domyslny dla
zmian (--apply wymagane), stdlib urllib (zero-dep). Adresy npm@VPS
przez Tailscale (100.95.58.48:81), NIE public IP.
+ docs/backlog.md: npm@VPS admin panel :81 publicznie osiagalny —
brak override ograniczajacego bind do mesh/localhost.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Faza 2 modułu 0 po review Oskara: elasticsearch+diskover usunięte (compose down,
dane esdata zostawione na dysku), available 2.8Gi -> 3.8Gi, kryterium >=1.5Gi
spełnione. llm-gateway udokumentowany (własny router LLM -> Ollama@SOLARIA,
źródło tylko w /opt/llm-gateway — archiwizacja w backlogu); immich zostaje na
PIHA na stałe (24/7, SOLARIA sesyjna).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Swap 2-4GB na VPS -> Zamkniete (2026-06-22). Nowa sekcja planu monitoringu
floty (Prometheus pull up{}, osobny instance, bez Alertmanagera) z krokami.
Zastepuje szkic blackbox+Alertmanager z 2026-06-17.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Oskar Kapala