# fleet-prometheus kb-mail-sync rules — faza mailowa Krok 7
# (recon kb/audits/mail-sync-2026-08-06.md §3.4).
#
# Same delivery convention as kb-ingest.yml and liveness.yml: no Alertmanager. These rules
# only make alerts FIRING (visible at GET /api/v1/alerts on this Prometheus instance);
# brain-watchdog@PIHA polls that API and forwards to Telegram. Do NOT add Alertmanager here.
#
# Metrics come from jobs/mail-imap-sync's textfile output
# (/opt/homelab/state/node-exporter/kb-mail-sync.prom), scraped by node_exporter on PIHA —
# every series from that target already carries node="piha" (fleet-node job's static label).
#
# Until the operator installs and enables kb-mail-sync.timer the metric does not exist, and
# `time() - <missing series>` produces no result — so this file is inert on a fleet where the
# job has not been set up. It does not need to be deployed in lockstep with the job.

groups:
  - name: kb-mail-sync
    rules:
      - alert: KbMailSyncStale
        # 6 missed hourly ticks. This is the ONE alert this job gets, and deliberately so:
        # it answers "is the poller working at all", which has no false positives — unlike
        # any threshold on how much mail arrived.
        #
        # "No new mail for X days" was considered and REJECTED (recon §3.4). Zero new mail is
        # a legal state of a mailbox — a holiday, a weekend, traffic moved to the other
        # account. An alert that fires on a healthy system gets muted, and a muted alert is
        # not there on the day it is needed.
        expr: time() - kb_mail_sync_last_success_timestamp{node="piha"} > 21600
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "kb-mail-sync stale on {{ $labels.node }}"
          description: "kb-mail-sync.timer has not completed a successful run in over 6h on {{ $labels.node }} — the mail corpus has stopped growing. Check `journalctl -u kb-mail-sync.service` and the latest /opt/homelab/logs/kb-mail-sync/run-*.log; an expired or revoked IMAP app password is the most likely cause."

      # Deliberately NOT defined here, pending a month of observation (recon §3.4):
      # a per-account warning on kb_mail_sync_last_message_ts (age of the newest mail).
      # It needs a per-account threshold — gmail is the junk account and days of silence
      # there would be odd, while for fastmail it would not be — and that threshold should
      # come from measured behaviour, not from a guess made before the first run.