homelab-codex-ws/kb/runbooks/fleet-prometheus-deploy.md
oskar 0a8a668b05 feat(kb): SPLIT fleet-prometheus -> service + decision + runbook
kb/services/fleet-prometheus.md (Stack, Placement & exposure, Data, Next steps)
kb/decisions/fleet-prometheus-osobny-od-prom.md ("Why separate from the home prom")
kb/runbooks/fleet-prometheus-deploy.md (Configuration, Verify)

Wzajemne links. Tresc sekcji nietknieta; kontrola: multizbior niepustych
linii czesci == oryginal z HEAD.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 16:58:46 +02:00

2.4 KiB

okf type visibility status updated links
0.1 runbook private active 2026-06-24
../services/fleet-prometheus.md
../decisions/fleet-prometheus-osobny-od-prom.md

fleet-prometheus — konfiguracja i weryfikacja

Configuration

  • prometheus.yml — committed, secret-free. Global config + two scrape jobs.
  • env.example → copy to .env (gitignored). Holds TAILSCALE_BIND_IP, the VPS Tailscale interface IP the listen socket is bound to.

Deploy — .env is mandatory

The port is published as ${TAILSCALE_BIND_IP}:9090:9090 so the socket is bound only to the Tailscale interface, never 0.0.0.0. Compose auto-loads a co-located services/fleet-prometheus/.env, but if that file is missing the variable resolves to a blank string and Compose silently binds to 0.0.0.0 (publicly reachable on the Hetzner IP) — it only warns, it does not fail.

Therefore, before up -d:

  1. Create services/fleet-prometheus/.env from env.example with the real TAILSCALE_BIND_IP (verify with tailscale ip -4 on the VPS).

  2. Deploy with the env file explicit, to be safe:

    docker compose -f services/fleet-prometheus/docker-compose.yml \
      --env-file services/fleet-prometheus/.env up -d
    
  3. Confirm the bind is Tailscale-only (must NOT show 0.0.0.0):

    docker compose -f services/fleet-prometheus/docker-compose.yml \
      --env-file services/fleet-prometheus/.env config | grep host_ip
    

Scrape targets (scaffold)

Job Target Notes
prometheus localhost:9090 self
fleet-node host.docker.internal:9100 VPS node_exporter (runs network_mode: host, listens on host :9100); reached via host-gateway

Verify

# Config is syntactically valid (uses promtool inside the image):
docker run --rm -v "$PWD/services/fleet-prometheus/prometheus.yml":/etc/prometheus/prometheus.yml:ro \
  prom/prometheus:v3.5.0 promtool check config /etc/prometheus/prometheus.yml

# Compose renders:
docker compose -f services/fleet-prometheus/docker-compose.yml config

# After `up -d`:
curl -sf http://localhost:9090/-/healthy
curl -s http://localhost:9090/api/v1/targets | jq '.data.activeTargets[].health'
./services/fleet-prometheus/healthcheck.sh