Commit graph

3 commits

Author SHA1 Message Date
Oskar Kapala d4170004e7 feat(fleet-prometheus): add liveness alert rules for always-on nodes (vps, piha)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 16:42:52 +02:00
oskar 43c47a0a55 fix(fleet-prometheus): bind listen socket to Tailscale IP, not 0.0.0.0
Previously published "9090:9090" → Docker bound to 0.0.0.0 (all host interfaces,
including the public Hetzner IP), leaving tailscale-internal enforced only by the
VPS firewall. Now bind explicitly to the VPS Tailscale interface for
defense-in-depth: the port does not exist on the public IP at all.

- ports -> "${TAILSCALE_BIND_IP}:9090:9090"
- env.example: add TAILSCALE_BIND_IP (VPS Tailscale IP, verify via `tailscale ip -4`)
- README: deploy section — .env is mandatory; a missing .env makes Compose
  silently bind 0.0.0.0 (warns, does not fail), so use --env-file and verify host_ip

Smoke: config with --env-file and with co-located .env both resolve
host_ip=100.95.58.48; with .env absent Compose warns and falls back to 0.0.0.0
(documented). .env is gitignored (global *.env rule).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:07:18 +02:00
oskar 4518b15f98 feat(fleet-prometheus): scaffold fleet liveness Prometheus (VPS)
Clean scaffold of a Prometheus instance dedicated to fleet liveness, deliberately
separate from the home `prom` on PIHA. Scrapes only itself + the VPS-local
node_exporter for now.

- image pinned to prom/prometheus:v3.5.0 (LTS) — no :latest anti-pattern
- TSDB retention 15d / 2GB on the tight VPS; --web.enable-lifecycle for reloads
- mem_limit 512m, oom_score_adj +200 (sacrificial OOM victim before control-plane)
- tailscale-internal exposure via plain published 9090, mirroring control-plane
- node_exporter scraped via host.docker.internal:9100 (it runs network_mode: host)
- secret-free prometheus.yml; alerting/rule_files left as commented placeholders
- in-container healthcheck via busybox wget (present in the image; curl is not)

Smoke: docker compose config OK; promtool check config SUCCESS; up -d ->
/-/healthy + /-/ready OK; both targets (prometheus, fleet-node) report up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:07:18 +02:00