docs(backlog): follow-upy z etapu 0 — 40-register.sh stary schemat topologii, komentarz mqtt_unreachable w observerze

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

CLAUDE.md

@@ -108,7 +108,6 @@ Normalized event types: `deployment_started/completed/failed`, `service_unhealth
 | Event type | Source | Action generated | Cooldown |
 |---|---|---|---|
 | `containers_not_running` | stability-agent | `container_restart` | dedup via stable ID |
-| `mqtt_unreachable` | stability-agent | `container_restart` | dedup via stable ID |
 | `service_unhealthy` / other | stability-agent | `redeploy` | dedup via stable ID |
 | `disk_pressure` (high) | stability-agent | `disk_cleanup` | dedup via stable ID |
 | `ha_websocket_dead` | ha-diag-agent | `container_restart` (homeassistant) | 30 min after completion |

docs/architecture/ARCHITEKTURA.md

@@ -0,0 +1,42 @@
+# Architektura — decyzje obowiązujące
+
+Stan decyzji na 2026-07-28. Podstawa dowodowa:
+[RECON-multiagent-2026-07-27.md](RECON-multiagent-2026-07-27.md); plan wykonawczy:
+[PLAN-subsystem-a-2026-07-28.md](PLAN-subsystem-a-2026-07-28.md). Zmiana którejkolwiek
+decyzji wymaga aktualizacji tego pliku z nową datą.
+
+## Dwa subsystemy (2026-07-28)
+
+- **A — keep-it-running**: control-plane (observer, supervisor, executor, operator-ui),
+  node-agenty, stability-agenty, self-healing floty. **To jest fokus tego repo.**
+- **B — do-the-work**: dyspozytor zadań (agent) + nogi KB / Home Assistant /
+  homelab-ops. Osobny wysiłek, osobny projekt. Z ai-clustra dziedziczy **wzorce**
+  (bus zadań, role workerów, allowlisty komend) — **nie kod**.
+
+## Role node'ów (2026-07-28)
+
+| Node | Rola | Status |
+|---|---|---|
+| **vps** | ingress publiczny + control-plane, nic więcej (4 GB RAM, publiczny) | active |
+| **piha** | serwer domowy 24/7 — dom wszystkiego, co musi zawsze działać; przyszły dom dyspozytora subsystemu B | active |
+| **solaria** | desktop obliczeniowy on-demand; wyłączany ~16 h/dobę **z założenia** — dobowe eventy offline/online to oczekiwany cykl pracy, nie awaria; nic wymagającego 24/7 nie może tu mieszkać | active |
+| **lustro** | Raspberry Pi z MagicMirror; pełnoprawny monitorowany node (node-agent, eventy) | active |
+| **chelsty-infra** | druga lokalizacja: hypervisor LTE, Zigbee/MQTT | dormant (sprzęt padł ~2026-06-01; reanimacja później) |
+| **chelsty-ha** | druga lokalizacja: VM Home Assistant | dormant (jak wyżej) |
+
+Node'y `dormant`: observer trzyma ostatni znany stan, ale nie emituje eventów
+liveness; supervisor nie generuje dla nich żadnych akcji.
+
+## ai-cluster = LEGACY (2026-07-28)
+
+Stack ai-cluster na vps (openclaw, codex-worker, planner-worker, service-ops-worker,
+redis, mosquitto) jest **wygaszany, nie migrowany**. Bus `codex/*` martwy od
+2026-06-09 (zero nowych połączeń). Branch `task/ai-cluster-solaria` zostaje
+**niezmergowany** — pełni rolę dokumentacji. Kontenery na vps zostaną zatrzymane w
+osobnej, nadzorowanej sesji. Szczegóły: [ai-cluster-LEGACY.md](ai-cluster-LEGACY.md).
+
+## Approvale zostają HITL (2026-07-28)
+
+Human-in-the-loop obowiązuje. Restarty kontenerów mogą w przyszłości przejść na
+autonomię, ale `redeploy` i `disk_cleanup` zawsze wymagają zgody człowieka.
+Kanałem powiadomień i approvali będzie **Telegram** (etap 2 planu).

docs/architecture/ai-cluster-LEGACY.md

@@ -0,0 +1,43 @@
+# ai-cluster — LEGACY, wygaszany (decyzja 2026-07-28)
+
+## Decyzja
+
+Stack **ai-cluster** działający na vps (`ai-cluster-openclaw-1`, `codex-worker`,
+`planner-worker`, `service-ops-worker`, `redis`, `mosquitto`) jest **wygaszany,
+nie migrowany**. Podstawa (recon
+[RECON-multiagent-2026-07-27.md](RECON-multiagent-2026-07-27.md), C9):
+bus `codex/*` jest martwy od **2026-06-09** — zero nowych połączeń przez ~7 tygodni,
+workery trzymają tylko puste długożyjące połączenia.
+
+Konsekwencje:
+
+- **Migracja na solarię NIE wchodzi.** Branch `task/ai-cluster-solaria`
+  (@ `b124e54`) zostaje **niezmergowany** — pełni rolę dokumentacji stanu prac
+  i reconu; nie kasować.
+- ai-cluster nie ma katalogu w `services/` na masterze i **nie dostanie go** —
+  nie wciągamy legacy do GitOps.
+- `planner-agent` na solarii (ta sama rodzina) ma w
+  `hosts/solaria/services.yaml` `monitor: false` — decyzja o jego losie osobno.
+- Kontenery na vps **zostaną zatrzymane w osobnej, nadzorowanej sesji**
+  (stop stacka, obserwacja `free -m`, rollback = start) — poza tym taskiem;
+  do tego czasu działają dalej pod hard mem_limitami.
+
+## Co przeżywa w subsystemie B (wzorce, nie kod)
+
+Subsystem B (dyspozytor + KB + HA + homelab-ops; osobny projekt) dziedziczy
+z ai-clustra **wzorce projektowe**:
+
+- bus zadań z routingiem po polu `target` (`role:*` / `AGENT_ID`) i wąskim ACL,
+- podział na wyspecjalizowane workery-role (dev / planner / service-ops),
+- allowlisty kształtu komend przed wykonaniem czegokolwiek na hoście — z lekcją
+  z reconu (A2): allowlist musi być **egzekwowana i celowana** (martwa stała
+  `SERVICE_NAMES` + substring-match po `docker ps` to antywzorzec),
+- tryb preview/diagnose przed wykonaniem.
+
+Kod ai-clustra **nie jest** przenoszony.
+
+## Czego NIE robić
+
+- Nie deployować, nie restartować i nie "naprawiać" stacka ai-cluster na vps.
+- Nie mergować `task/ai-cluster-solaria`.
+- Nie podpinać nowych klientów pod `codex/*` na brokerze ai-clustra.

docs/backlog.md

@@ -1207,3 +1207,16 @@ Pomiar throughput GPU vs CPU baseline (0.79s/chunk) — patrz
   cicho no-opuje się przez istniejący constraint. Schema change do zrobienia
   przy fazie 3 (patrz `jobs/documents-ingest/README.md`, sekcja "Idempotency"
   kroku 6 embed).
+
+## Follow-upy z etapu 0 (truth cleanup, 2026-07-29)
+
+**Źródło**: sesja merge `task/etap0-truth` (topologia: status active|dormant,
+listy serwisów usunięte z topology.yaml — node-level truth only).
+
+- **40-register.sh emituje stary schemat topologii** — szablon bloku node'a
+  w `scripts/onboard/steps/40-register.sh` nadal zawiera listę `services:`
+  i nie ma pola `status:`; wyrównać z nowym schematem node-level-only
+  (`inventory/topology.yaml`).
+- **Historyczny komentarz mqtt_unreachable w observerze** — `scripts/observer/
+  observer.py:866` wspomina routing `mqtt_unreachable -> container_restart`
+  usunięty z supervisora (recon D15); sprzątnąć przy najbliższej edycji pliku.

hosts/piha/runtime/ollama-piha/docker-compose.override.yml

@@ -0,0 +1,18 @@
+# PIHA-specific overrides for ollama-piha (KB module 5, phase 4 — embed fallback).
+#
+# RESOURCE CONTEXT: PIHA is the RAM-bound 8 GB box shared with Home Assistant.
+# bge-m3 is ~1.2 GB on disk, estimated 1.5–2 GB resident during an embed burst
+# (plan §2 decision 2). OLLAMA_KEEP_ALIVE=0 (base compose) makes that a short
+# per-call spike, not a resident cost — but the spike lands exactly when other
+# SOLARIA-fallback consumers (e.g. paperless fallback OCR) may also be awake,
+# so it gets a hard ceiling.
+services:
+  ollama-piha:
+    # Plan §2 D2 starting value — the cgroup OOM killer restarts this container
+    # on breach instead of the host OOM killer picking a victim (which could be
+    # Home Assistant). Confirm or trim after live calibration (plan §5 step 4:
+    # a few embeds + docker stats at a normal-load hour).
+    mem_limit: 2560m
+    # Deliberately no mem_reservation: the working set is a transient spike and
+    # the idle daemon is ~100 MB — soft-reserving gigabytes would permanently
+    # take them from HA for a backend that mostly sleeps.

hosts/piha/services.yaml

@@ -105,14 +105,37 @@ services:
       # data is in Docker named volume kb_postgres_data — must land on the NVMe
       # (Docker data-root on /home); see hosts/piha/runtime/kb-postgres override.
 
+  ollama-piha:
+    role: embed-fallback         # module 5 phase 4 (plan §2 D2/§5): local CPU bge-m3 for kb-query while SOLARIA sleeps
+    deployment_model: docker-compose
+    exposure: private            # 127.0.0.1 + LAN bind (LAN_BIND_IP) only; no public/Tailscale exposure
+    offline_required: false
+    depends_on:
+      local: []
+      external: []
+    ports:
+      - name: http
+        container_port: 11434
+        host_port: 11434
+        protocol: tcp
+    runtime:
+      # .env (LAN_BIND_IP) lives alongside the compose file; models live in the
+      # Docker named volume ollama_piha_models (NVMe data-root, like kb-postgres).
+      # Deploy note: `docker exec ollama-piha ollama pull bge-m3` is a manual
+      # post-deploy step — the image ships no models.
+      config_path: services/ollama-piha
+
   kb-query:
     role: kb-search-api          # module 5 phase 4: FastAPI wrapper over kb_retrieval cascade/flat query
     deployment_model: docker-compose
     exposure: private            # LAN bind (LAN_BIND_IP); npm@PIHA vhost + OIDC is a later step
     offline_required: false
     depends_on:
-      local: [kb-postgres]
-      external: [ollama]         # SOLARIA may be offline -> /search 502/503, /healthz stays ok
+      # ollama-piha is a soft local dependency: the embed fallback while SOLARIA
+      # sleeps — kb-query starts and serves /healthz without it, /search degrades
+      # to 503 only when BOTH embed backends are unreachable.
+      local: [kb-postgres, ollama-piha]
+      external: [ollama]         # primary embed backend @ SOLARIA; may be offline -> fallback to ollama-piha
     ports:
       - name: http
         container_port: 8080

hosts/solaria/services.yaml

@@ -14,6 +14,38 @@ services:
       data_path: /opt/homelab/state
       logs_path: /opt/homelab/events
 
+  stability-agent:
+    role: node-watchdog          # read-only docker.sock watchdog, emits filesystem events
+    deployment_model: docker-compose
+    exposure: local-only
+    offline_required: true
+    depends_on:
+      local: []
+      external: []
+    runtime:
+      data_path: /opt/homelab/state
+      logs_path: /opt/homelab/events
+
+  node_exporter:
+    role: metrics-exporter
+    deployment_model: docker-compose
+    exposure: local-only
+    offline_required: true
+    depends_on:
+      local: []
+      external: []
+
+  planner-agent:
+    # legacy ai-cluster family — decision pending, do not remediate
+    monitor: false
+    role: ai-planner
+    deployment_model: docker-compose
+    exposure: local-only
+    offline_required: false
+    depends_on:
+      local: []
+      external: []
+
   ollama:
     role: llm-inference          # GPU-backed inference (RTX 4070 Ti SUPER, driver restored 2026-07-16): embeddings (bge-m3) + coder models
     deployment_model: docker-compose

inventory/topology.yaml

@@ -3,7 +3,9 @@ topology:
   git_provider: forgejo
 
 deployment:
-  mode: pull
+  # Every deploy script SSH-pushes from saturn to the target node (recon
+  # F20.10); "pull" never matched reality.
+  mode: push
   orchestrator: saturn
 
 # Ingress dla usług domowych przez NPM @ PIHA + wildcard cert kapala.org.
@@ -31,54 +33,56 @@ ingress:
       access: tailscale
       cert: wildcard-kapala
 
+# Node-level truth ONLY. Per-node service lists live in hosts/<node>/services.yaml
+# (authoritative — the lists formerly duplicated here contradicted them, recon F20.1).
+#
+# status:
+#   active  — node is monitored and remediated normally.
+#   dormant — node exists but is expected offline: the observer keeps its
+#             last-known world state and emits NO liveness events for it; the
+#             supervisor generates NO actions for it. See ARCHITEKTURA.md.
 nodes:
 
   saturn:
+    status: active
     roles:
       - control
       - development
 
   piha:
+    status: active
     roles:
       - infra
       - monitoring
-    services:
-      - node-agent
-      - ha-diag-agent
-      - brain-watchdog
-      - vikunja             # Task management (vikunja + postgres), public via npm
-      - kb-postgres         # KB spine: Postgres 16 + pgvector, port 5433 (always-on)
-      - llm-gateway         # FastAPI router -> Ollama @ SOLARIA (Tailscale-internal :8080)
-      - node_exporter       # per-host (also vps) — textfile collector for kb-ingest metrics
 
   solaria:
+    # Powered off ~16 h/day BY DESIGN (on-demand compute desktop): one daily
+    # offline/online liveness cycle is expected, not a fault. Nothing that
+    # must run 24/7 may live here.
+    status: active
     roles:
       - compute
       - ai
-    services:
-      - node-agent
 
   vps:
+    status: active
     roles:
       - edge
       - ingress
       - control-plane
-    services:
-      # Repo-managed GitOps services (hosts/vps/services.yaml is authoritative)
-      - node-agent
-      - control-plane       # executor, observer, supervisor, operator-ui
-      - node_exporter
-      - fleet-prometheus    # Fleet liveness source of truth (Tailscale-internal)
-      - stability-agent
-      - npm                 # Nginx Proxy Manager — public ingress, TLS termination
-      - outline             # Team wiki (outline + postgres + redis)
-      - joplin              # Note sync server (joplin-server + postgres)
-      - ai-cluster          # AI workers: codex-worker, openclaw, planner-worker,
-                            # service-ops-worker, redis, mosquitto
-      - gokapi              # Public file-share (Firefox Send alt), share.okit.pl via npm@VPS,
-                            # local disk storage, E2E encryption ON — separate from private Nextcloud
+
+  lustro:
+    # MagicMirror Raspberry Pi — full monitored node: runs node-agent and
+    # ships events to the control plane (recon F20.11).
+    status: active
+    roles:
+      - edge
+      - display
 
   chelsty-infra:
+    status: dormant
+    dormant_reason: "site hardware down since ~2026-06-01; will be revived later"
+    dormant_since: "2026-06-01"
     site: chelsty
     roles:
       - remote
@@ -90,15 +94,15 @@ nodes:
       intermittent: true
     home_automation:
       offline_operation_required: true
-      services:
-        - zigbee2mqtt
-        - mosquitto
       coordinator:
         model: SLZB-06U
         connection: network
         usb: false
 
   chelsty-ha:
+    status: dormant
+    dormant_reason: "site hardware down since ~2026-06-01; will be revived later"
+    dormant_since: "2026-06-01"
     site: chelsty
     roles:
       - remote
@@ -108,11 +112,3 @@ nodes:
       intermittent: true
     home_automation:
       offline_operation_required: true
-      services:
-        - homeassistant
-
-  lustro:
-    roles:
-      - edge
-    services:
-      - node-agent

scripts/deploy/deploy-role.sh

@@ -1,21 +0,0 @@
-#!/usr/bin/env bash
-
-set -e
-
-ROLE=$1
-HOST=$2
-
-if [ -z "$ROLE" ] || [ -z "$HOST" ]; then
-    echo "Usage:"
-    echo "./deploy-role.sh <role> <host>"
-    exit 1
-fi
-
-echo "Deploying role: $ROLE to host: $HOST"
-
-ssh $HOST "
-    mkdir -p ~/homelab-codex-ws &&
-    cd ~/homelab-codex-ws &&
-    git pull &&
-    docker compose -f roles/$ROLE/compose/docker-compose.yml up -d
-"

scripts/observer/observer.py

@@ -235,6 +235,18 @@ class Observer:
             }
         }
         self.inventory = self._load_inventory()
+        # Nodes declared `status: dormant` in inventory/topology.yaml: expected
+        # offline (e.g. chelsty site, hardware down since ~2026-06-01).  The
+        # observer keeps their last-known world state but never reclassifies
+        # liveness for them, so no node_offline/node_stale/node_online events
+        # are emitted while a node is dormant.  See docs/architecture/ARCHITEKTURA.md.
+        self.dormant_nodes = {
+            name for name, info in self.inventory["nodes"].items()
+            if info.get("status") == "dormant"
+        }
+        if self.dormant_nodes:
+            logger.info("Dormant nodes (no liveness tracking): %s",
+                        sorted(self.dormant_nodes))
         self._ensure_dirs()
         self._load_checkpoint()
         # Persistent SHADOW_LIVENESS_MISMATCH sink (survives container recreate).
@@ -274,9 +286,12 @@ class Observer:
                 with open(INVENTORY_TOPOLOGY, "r") as f:
                     topo = yaml.safe_load(f)
                     for node_name, node_info in topo.get("nodes", {}).items():
+                        node_info = node_info or {}
                         inventory["nodes"][node_name] = {
                             "roles": node_info.get("roles", []),
-                            "connectivity": node_info.get("connectivity", {})
+                            "connectivity": node_info.get("connectivity", {}),
+                            # topology node status: active (default) | dormant
+                            "status": node_info.get("status", "active"),
                         }
             
             # Load service assignments from hosts files
@@ -558,6 +573,9 @@ class Observer:
         # when disabled/unreachable — fail-open, event liveness is authoritative.
         prom_liveness_map = self._query_prometheus_liveness()
         for node_name, node_info in self.world_state["nodes"].items():
+            if node_name in self.dormant_nodes:
+                # Dormant node: keep last-known state frozen, emit nothing.
+                continue
             roles = (node_info.get("roles")
                      or self.inventory["nodes"].get(node_name, {}).get("roles", []))
             liveness = compute_liveness(

services/control-plane/src/supervisor.py

@@ -33,10 +33,13 @@ try:
 except Exception:
     NODE_ALIAS_MAP = {}
 
-# Event trigger types that should result in a lightweight container_restart
-# rather than a full redeploy. The container is present but not running,
-# or a dependency (MQTT) is unreachable — a restart is the right first step.
-CONTAINER_RESTART_TRIGGERS = {"containers_not_running", "mqtt_unreachable"}
+# Incident trigger types that should result in a lightweight container_restart
+# rather than a full redeploy: the container is present but not running.
+# mqtt_unreachable was removed 2026-07-28: the observer never creates incidents
+# with that trigger_type, so the branch was dead code (recon
+# docs/architecture/RECON-multiagent-2026-07-27.md, D15). stability-agent still
+# emits the mqtt_unreachable *event*; it just never becomes an incident.
+CONTAINER_RESTART_TRIGGERS = {"containers_not_running"}
 
 # Nodes where automatic disk_cleanup actions must NOT be generated.
 # On chelsty nodes disk fullness is overwhelmingly caused by Frigate recordings
@@ -129,6 +132,13 @@ class Supervisor:
     def __init__(self):
         self.desired_state = {"services": {}}
         self.actual_state = {"services": {}, "nodes": {}, "incidents": {}}
+        # Nodes declared `status: dormant` in inventory/topology.yaml (e.g. the
+        # chelsty site, hardware down since ~2026-06-01).  The supervisor
+        # generates NO actions of any kind for them: their services are excluded
+        # from desired state (which also auto-cancels their stale pending
+        # actions), and node/HA events from them are not routed to alerts.
+        # See docs/architecture/ARCHITEKTURA.md.
+        self.dormant_nodes: set = set()
         # In-memory set of already-routed HA event IDs; prevents re-processing
         # on each reconcile cycle. Grows to at most ~hundreds of entries/day.
         self._ha_processed_event_ids: set = set()
@@ -179,6 +189,21 @@ class Supervisor:
     # State loading
     # ------------------------------------------------------------------
 
+    def _load_dormant_nodes(self):
+        """Refresh the set of dormant nodes from inventory/topology.yaml."""
+        dormant = set()
+        topo_file = REPO_ROOT / "inventory" / "topology.yaml"
+        try:
+            if topo_file.exists():
+                with open(topo_file, "r") as f:
+                    topo = yaml.safe_load(f) or {}
+                for name, info in (topo.get("nodes") or {}).items():
+                    if (info or {}).get("status") == "dormant":
+                        dormant.add(name)
+        except Exception as e:
+            logger.error(f"Failed to load dormant nodes from {topo_file}: {e}")
+        self.dormant_nodes = dormant
+
     def _load_desired_state(self):
         services = {}
         hosts_dir = REPO_ROOT / "hosts"
@@ -194,6 +219,11 @@ class Supervisor:
                         with open(svc_file, "r") as f:
                             data = yaml.safe_load(f)
                             host_name = data.get("host")
+                            if host_name in self.dormant_nodes:
+                                logger.debug(
+                                    f"Skipping desired services of {host_name}: node dormant"
+                                )
+                                continue
                             for svc_name, svc_info in data.get("services", {}).items():
                                 svc_info = svc_info or {}
                                 # monitor: false — service is documented as desired but
@@ -276,7 +306,7 @@ class Supervisor:
         """
         Return the trigger_type of the active incident for a service, or None.
         trigger_type is set by the observer when it creates an incident from
-        a specific event type (e.g. 'containers_not_running', 'mqtt_unreachable').
+        a specific event type (e.g. 'containers_not_running').
         """
         svc_info = self.actual_state["services"].get(svc_key, {})
         incident_id = svc_info.get("incident_id")
@@ -299,6 +329,7 @@ class Supervisor:
         except Exception as e:
             logger.error(f"Failed to touch heartbeat file: {e}")
 
+        self._load_dormant_nodes()
         self._load_desired_state()
         if not self._load_actual_state():
             return  # world state unreadable this cycle — skip to avoid false drift
@@ -334,7 +365,7 @@ class Supervisor:
 
         # 3. Generate node-level recommendations (disk pressure)
         for node_name, node_info in self.actual_state["nodes"].items():
-            if node_name in NO_DISK_CLEANUP_NODES:
+            if node_name in NO_DISK_CLEANUP_NODES or node_name in self.dormant_nodes:
                 continue
             if node_info.get("disk_pressure") == "high":
                 self._generate_disk_cleanup_recommendation(node_name)
@@ -381,8 +412,7 @@ class Supervisor:
 
         if trigger_type in CONTAINER_RESTART_TRIGGERS:
             # Lightweight remediation: the container exists but is not running
-            # (containers_not_running) or its MQTT dependency is unreachable
-            # (mqtt_unreachable). A docker restart is sufficient and low-risk.
+            # (containers_not_running). A docker restart is sufficient and low-risk.
             container_name = self._get_container_name(service)
             action = {
                 "action_id": action_id,
@@ -582,6 +612,9 @@ class Supervisor:
         node = event.get("node", "")
         if not node:
             return
+        if node in self.dormant_nodes:
+            logger.debug(f"Suppressing {event_type} on {node}: node dormant")
+            return
 
         if event_type in HA_CONTAINER_RESTART_EVENTS:
             if self._is_ha_in_transition(node):
@@ -772,6 +805,9 @@ class Supervisor:
         node = event.get("node") or event.get("payload", {}).get("affected_node")
         if not node:
             return
+        if node in self.dormant_nodes:
+            logger.debug(f"Suppressing {event_type} on {node}: node dormant")
+            return
 
         action_id = f"alert-{event_type.replace('_', '-')}-{node}"
 

services/control-plane/tests/test_dormant_nodes.py

@@ -0,0 +1,197 @@
+"""Dormant-node handling (topology `status: dormant`, etap 0 truth cleanup).
+
+Contract (docs/architecture/ARCHITEKTURA.md): a dormant node keeps its
+last-known world state, the observer emits NO liveness events for it, and the
+supervisor generates NO actions of any kind for it.
+"""
+from __future__ import annotations
+
+import json
+import sys
+import time
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
+sys.path.insert(0, str(Path(__file__).parent.parent.parent.parent / "scripts"))
+
+import observer.observer as obs_mod
+from observer.observer import Observer
+import supervisor as supervisor_module
+from supervisor import Supervisor
+
+TOPOLOGY_YAML = (
+    "nodes:\n"
+    "  piha:\n"
+    "    status: active\n"
+    "    roles: [infra]\n"
+    "  oldnode:\n"
+    "    status: dormant\n"
+    "    dormant_reason: \"hardware down\"\n"
+    "    roles: [remote]\n"
+)
+
+
+# ---------------------------------------------------------------------------
+# Observer
+# ---------------------------------------------------------------------------
+
+@pytest.fixture
+def obs_paths(tmp_path, monkeypatch):
+    world = tmp_path / "world"
+    state = tmp_path / "state"
+    events = tmp_path / "events"
+    logs = tmp_path / "logs"
+    repo = tmp_path / "repo"
+    for d in (world, state, events, logs, repo / "inventory", repo / "hosts"):
+        d.mkdir(parents=True, exist_ok=True)
+    (repo / "inventory" / "topology.yaml").write_text(TOPOLOGY_YAML)
+    monkeypatch.setattr(obs_mod, "WORLD_DIR", world)
+    monkeypatch.setattr(obs_mod, "STATE_DIR", state)
+    monkeypatch.setattr(obs_mod, "EVENTS_DIR", events)
+    monkeypatch.setattr(obs_mod, "LOGS_DIR", logs)
+    monkeypatch.setattr(obs_mod, "INVENTORY_TOPOLOGY", repo / "inventory" / "topology.yaml")
+    monkeypatch.setattr(obs_mod, "REPO_ROOT", repo)
+    monkeypatch.setattr(obs_mod, "FAILED_EVENTS_DIR", state / "observer_failed_events")
+    monkeypatch.setattr(obs_mod, "OBSERVER_STATE_FILE", state / "observer_checkpoint.json")
+    monkeypatch.setattr(obs_mod, "PROM_SHADOW_URL", "")
+    monkeypatch.setattr(obs_mod, "SHADOW_LOG_DIR", logs / "observer")
+    return tmp_path
+
+
+def test_observer_loads_dormant_set_from_topology(obs_paths):
+    obs = Observer()
+    assert obs.dormant_nodes == {"oldnode"}
+
+
+def test_observer_freezes_dormant_node_and_emits_nothing(obs_paths):
+    """A dormant node whose heartbeats stopped long ago must NOT be
+    reclassified (stale→dead would normally fire node_offline)."""
+    obs = Observer()
+    stale_age = int(time.time()) - 999999
+    obs.world_state["nodes"] = {
+        "oldnode": {"status": "stale", "liveness": "stale", "last_seen": stale_age},
+        "piha": {"status": "unknown", "last_seen": int(time.time())},
+    }
+
+    obs._prune_stale_world()
+
+    frozen = obs.world_state["nodes"]["oldnode"]
+    assert frozen["liveness"] == "stale"      # unchanged, despite dead-level age
+    assert frozen["status"] == "stale"        # unchanged
+    # No liveness event file was written for the dormant node.
+    assert not list((obs_paths / "events").glob("**/evt-oldnode-*"))
+    # The active node is still processed normally.
+    assert obs.world_state["nodes"]["piha"]["liveness"] == "fresh"
+
+
+def test_observer_keeps_dormant_node_in_world_state(obs_paths):
+    """Dormant nodes are in the inventory, so pruning must not remove them."""
+    obs = Observer()
+    obs.world_state["nodes"] = {"oldnode": {"status": "offline", "last_seen": 1}}
+
+    obs._prune_stale_world()
+
+    assert "oldnode" in obs.world_state["nodes"]
+
+
+# ---------------------------------------------------------------------------
+# Supervisor
+# ---------------------------------------------------------------------------
+
+@pytest.fixture
+def sup(tmp_path, monkeypatch):
+    actions = tmp_path / "actions"
+    events = tmp_path / "events"
+    world = tmp_path / "world"
+    state = tmp_path / "state"
+    repo = tmp_path / "repo"
+    for d in (actions, events, world, state, repo / "inventory", repo / "hosts" / "oldnode"):
+        d.mkdir(parents=True, exist_ok=True)
+    (repo / "inventory" / "topology.yaml").write_text(TOPOLOGY_YAML)
+    (repo / "hosts" / "oldnode" / "services.yaml").write_text(
+        "host: oldnode\nservices:\n  some-svc:\n    role: x\n"
+    )
+    monkeypatch.setattr(supervisor_module, "ACTIONS_DIR", actions)
+    monkeypatch.setattr(supervisor_module, "EVENTS_DIR", events)
+    monkeypatch.setattr(supervisor_module, "WORLD_DIR", world)
+    monkeypatch.setattr(supervisor_module, "REPO_ROOT", repo)
+    s = Supervisor()
+    return s
+
+
+def _pending(tmp_path):
+    return list((tmp_path / "actions" / "pending").glob("*.json"))
+
+
+def test_supervisor_loads_dormant_set(sup):
+    sup._load_dormant_nodes()
+    assert sup.dormant_nodes == {"oldnode"}
+
+
+def test_dormant_services_excluded_from_desired_state(sup):
+    sup._load_dormant_nodes()
+    sup._load_desired_state()
+    assert "oldnode/some-svc" not in sup.desired_state["services"]
+
+
+def test_reconcile_generates_no_actions_for_dormant_node(sup, tmp_path):
+    """Full reconcile: missing service + high disk pressure on a dormant node
+    must produce zero pending actions."""
+    world = tmp_path / "world"
+    (world / "nodes.json").write_text(json.dumps(
+        {"oldnode": {"status": "offline", "disk_pressure": "high"}}
+    ))
+    (world / "services.json").write_text("{}")
+    (world / "incidents.json").write_text("{}")
+
+    sup.reconcile()
+
+    assert _pending(tmp_path) == []
+
+
+def test_node_event_from_dormant_node_not_routed(sup, tmp_path):
+    sup._load_dormant_nodes()
+    event = {
+        "id": "evt-oldnode-1-node_offline-node",
+        "type": "node_offline",
+        "node": "oldnode",
+        "payload": {"affected_node": "oldnode"},
+    }
+    (tmp_path / "events" / f"{event['id']}.json").write_text(json.dumps(event))
+
+    sup._process_ha_events()
+
+    assert _pending(tmp_path) == []
+
+
+def test_ha_event_from_dormant_node_not_routed(sup, tmp_path):
+    sup._load_dormant_nodes()
+    event = {
+        "id": "evt-oldnode-2-ha_entity_unavailable_long-homeassistant",
+        "type": "ha_entity_unavailable_long",
+        "node": "oldnode",
+        "payload": {},
+    }
+    (tmp_path / "events" / f"{event['id']}.json").write_text(json.dumps(event))
+
+    sup._process_ha_events()
+
+    assert _pending(tmp_path) == []
+
+
+def test_active_node_events_still_routed(sup, tmp_path):
+    """Sanity: the dormant guard must not swallow active-node events."""
+    sup._load_dormant_nodes()
+    event = {
+        "id": "evt-piha-3-node_stale-node",
+        "type": "node_stale",
+        "node": "piha",
+        "payload": {"affected_node": "piha"},
+    }
+    (tmp_path / "events" / f"{event['id']}.json").write_text(json.dumps(event))
+
+    sup._process_ha_events()
+
+    assert len(_pending(tmp_path)) == 1

services/kb-query/README.md

@@ -12,7 +12,7 @@ that's phase 5.
 |---|---|---|
 | `/` | GET | Search UI (Jinja2 shell + `/static/app.js`, no login yet — plan §8 OIDC is a later step) |
 | `/static/*` | GET | UI assets (`app.js`, `style.css`) |
-| `/healthz` | GET | `{"status": "ok", "sol_status": "up"\|"down"}` — `sol_status` is a live probe of Ollama@SOLARIA, no auth required (monitoring must reach it) |
+| `/healthz` | GET | `{"status": "ok", "sol_status": "up"\|"down", "fallback_status": "up"\|"down"\|"unconfigured"}` — `sol_status` comes from the embed router's ~30 s-cached SOLARIA probe (same world view `/search` routes by), `fallback_status` is a live cheap probe of ollama-piha; no auth required (monitoring must reach it) |
 | `/search?q=<text>&mode=cascade\|flat` | GET | `query_text -> embed -> cascade_query/flat_query -> results`, `mode` defaults to `cascade` |
 
 `/search` response shape (module 5 phase 4 plan §4, `summary`/`summary_tags`
@@ -21,7 +21,7 @@ change any field the plan §4 shape already defined):
 
 ```json
 {
-  "query": "...", "mode": "cascade", "sol_status": "up",
+  "query": "...", "mode": "cascade", "sol_status": "up", "embed_backend": "solaria",
   "results": [
     {"envelope_id": "paperless:119", "source": "paperless", "dist": 0.34,
      "chunk_index": 2, "text": "...", "link": "https://paper.kapala.org/documents/119/details",
@@ -64,13 +64,39 @@ container, no node build step (plan §2 decision 4): `app/templates/index.html`
 - Stopka pokazuje `sol_status` dyskretnie (odświeżane z `/healthz` przy
   starcie strony i po każdym wyszukiwaniu).
 
-## Embed path (current step)
+## Embed path — active SOLARIA→PIHA fallback (Krok 2, plan §2 D2/§5)
 
-Calls Ollama on SOLARIA directly per request — no cache, no circuit breaker,
-no local-PIHA fallback yet (that state machine, plan §2 decision 2/§5, is a
-separate later step). If SOLARIA is unreachable, `/search` returns **503**;
-`/healthz` still answers (`sol_status: "down"`), same tolerance pattern as
-`llm-gateway`.
+Query embeddings go through `app/embed_router.py` (`EmbedRouter`, one instance
+per process):
+
+1. SOLARIA's health verdict (`GET /api/tags`, 1.5 s timeout) is **cached for
+   30 s** — no per-request probing.
+2. Verdict `up` → embed on `EMBED_PRIMARY_URL` (SOLARIA GPU, ~207 ms) under a
+   hard 3 s timeout.
+3. A failure **during a real embed** flips the verdict to `down` for one TTL
+   window and the **same request** is served from `EMBED_FALLBACK_URL`
+   (`ollama-piha`, CPU, `OLLAMA_KEEP_ALIVE=0` — slower, single seconds, but
+   alive) — the user never sees a SOLARIA error while a fallback exists.
+4. Verdict `down` → straight to the fallback until the TTL expires; when
+   SOLARIA wakes up, traffic returns to the GPU within ≤30 s.
+
+Every `/search` response and log line says which backend embedded the query
+(`embed_backend: "solaria"|"piha"`, log `backend=…` in `kb-query.embed`) —
+needed to debug result quality per backend. `sol_status` in the response is
+the router's world view; the UI footer renders `down` as
+„SOLARIA: offline (fallback embed)".
+
+**Model invariant, both halves:** at startup kb-query pins `EMBED_MODEL`
+(bge-m3) to `document_chunk.model`/`document_summary.embedding_model` (below);
+additionally each backend is verified once, at its first use, that its
+`/api/tags` actually lists `EMBED_MODEL`. A backend serving the wrong model is
+a **loud ERROR + 500** — never a silent distance computation across two
+different embedding spaces. Verification is lazy because SOLARIA may be asleep
+at boot and must not block startup.
+
+`/search` returns **503** only when BOTH backends are unreachable (or
+`EMBED_FALLBACK_URL` is unset — then the pre-fallback behaviour applies);
+`/healthz` still answers, same tolerance pattern as `llm-gateway`.
 
 ## Startup invariant (hard-fail)
 
@@ -86,13 +112,23 @@ that *wrote* the summary, e.g. `claude-haiku-4-5`, not the embedder).
 ## Configuration
 
 `.env` — **gitignored**, copy from `env.example`. Required: `LAN_BIND_IP`,
-`KB_DSN`. Optional: `OLLAMA_URL`, `EMBED_MODEL`, `SUMMARY_MODEL`.
+`KB_DSN`. On PIHA also set `EMBED_FALLBACK_URL=http://192.168.31.5:11434`
+(ollama-piha). Optional (defaults in parentheses): `EMBED_PRIMARY_URL`
+(`http://solaria:11434`), `EMBED_PRIMARY_NAME`/`EMBED_FALLBACK_NAME`
+(`solaria`/`piha`), `EMBED_HEALTH_TTL_S` (30), `EMBED_HEALTH_TIMEOUT_S` (1.5),
+`EMBED_PRIMARY_TIMEOUT_S` (3), `EMBED_MODEL` (`bge-m3`), `SUMMARY_MODEL`
+(`claude-haiku-4-5`). `OLLAMA_URL` was **renamed** to `EMBED_PRIMARY_URL` in
+Krok 2 — if an old `.env` sets `OLLAMA_URL`, it is ignored.
 
 ## Deploy (PIHA)
 
+0. Prerequisite: `ollama-piha` deployed and `bge-m3` pulled — see
+   `services/ollama-piha/README.md` (the pull is a **manual** deploy step).
 1. `git pull` on PIHA (`~/homelab-codex-ws`).
 2. `cp services/kb-query/env.example services/kb-query/.env` and fill in the
-   real `KB_DSN` password.
+   real `KB_DSN` password (the template already sets `EMBED_FALLBACK_URL`).
+   On an existing install: add `EMBED_FALLBACK_URL=http://192.168.31.5:11434`
+   to the existing `.env`.
 3. ```
    docker compose -f services/kb-query/docker-compose.yml \
      -f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build
@@ -101,6 +137,22 @@ that *wrote* the summary, e.g. `claude-haiku-4-5`, not the embedder).
    `curl "http://192.168.31.5:8230/search?q=test"` and open
    `http://192.168.31.5:8230/` in a browser.
 
+## Fallback verification (execution: operator, after deploy)
+
+- **Test A — SOLARIA online**: query via UI/`curl`; response has
+  `"embed_backend": "solaria"`, `docker logs kb-query` shows
+  `backend=solaria`, latency ~sub-second.
+- **Test B — SOLARIA offline**: either wait for the nightly power-off, or
+  simulate: set `EMBED_PRIMARY_URL=http://192.0.2.1:11434` (TEST-NET, always
+  unreachable) in `.env` and `docker compose … up -d` again. Query still
+  works; response has `"embed_backend": "piha"`, log shows `backend=piha`
+  plus a `circuit open for 30s` warning on the first hit; latency visibly
+  higher (CPU + cold model load each time, `OLLAMA_KEEP_ALIVE=0`). Revert
+  `.env` afterwards if simulated.
+- **Test C — SOLARIA returns**: after it is back up, within ≤30 s (one
+  health-cache TTL) responses show `"embed_backend": "solaria"` again, no
+  restart needed.
+
 ## Tests
 
 ```
@@ -150,5 +202,8 @@ existed.
 
 ## Out of scope for this step
 
-- Local-PIHA embed fallback / circuit breaker (plan §2 decision 2, §5).
 - OIDC login (see above) — separate session, needs `authlib` + Forgejo OAuth2 app.
+- Calibration verdict for the fallback (plan §5 steps 4–5: live PIHA
+  measurements → keep/tune/degrade decision) — the mechanism is built and
+  default-on when `EMBED_FALLBACK_URL` is set; the measurements are the
+  operator's post-deploy step (see `services/ollama-piha/README.md`).

services/kb-query/app/embed_router.py

@@ -0,0 +1,185 @@
+"""Embed-backend router -- module 5 phase 4, Krok 2 (docs/kb/modules/05-faza4-plan.md §2
+Decyzja 2, §5): the active SOLARIA->PIHA fallback state machine `/search` embeds through.
+One instance per process (`app.state.embed_router`), holding the single global `sol_status`
+cache the plan describes:
+
+| state           | behaviour                                                            |
+|-----------------|----------------------------------------------------------------------|
+| cache valid     | use remembered up/down, no probe                                     |
+| cache expired   | GET {primary}/api/tags, short timeout -> up/down, cached for the TTL |
+| up -> embed     | POST {primary}/api/embeddings under a hard timeout                   |
+| embed fails     | one-shot flip (plan step 3b): mark down for one TTL window and serve |
+|                 | THE SAME query from the fallback -- the user never sees a SOLARIA    |
+|                 | error unless there is no fallback configured                         |
+| down -> embed   | straight to the fallback, no primary attempt until the TTL expires   |
+
+Both legs call `kb_retrieval.embed.embed_chunk(..., model=embed_model)` with the same
+configured model (the plan's "twardy inwariant"). On top of the DB-side startup check
+(app/startup.py pins EMBED_MODEL to `document_chunk.model`/`document_summary.embedding_model`),
+each backend is additionally verified ONCE per process lifetime, lazily at its first use:
+its `/api/tags` must list `embed_model`. A backend that answers but lacks bge-m3 raises
+`ModelMismatchError` (ERROR log, 500 upstream) -- never a silent embed in a vector space
+that doesn't match the index. Verification is lazy rather than at startup because either
+backend may legitimately be offline when kb-query boots (SOLARIA sleeps ~16 h/day, plan
+§1) and must not block the service.
+
+Timeouts: probe 1.5 s (task spec "krótki, np. 1-2 s"; the plan's 500 ms proposal was for
+LAN-adjacent Tailscale -- 1.5 s absorbs LTE-grade jitter without hurting the 30 s cache
+economics), primary embed 3 s hard (plan step 3). The fallback embed gets NO hard timeout:
+Pi-5 CPU embed plus a cold model load (OLLAMA_KEEP_ALIVE=0 on ollama-piha) is legitimately
+slow (single seconds to low tens) and it is the last resort -- cutting it off would turn
+"slow but alive" back into "dead".
+"""
+from __future__ import annotations
+
+import asyncio
+import logging
+import time
+from dataclasses import dataclass, field
+
+import aiohttp
+
+from kb_retrieval.embed import check_ollama_health, embed_chunk
+
+logger = logging.getLogger("kb-query.embed")
+
+
+class EmbedBackendError(RuntimeError):
+    """No configured backend could embed the query (primary down and the fallback is
+    unconfigured or unreachable). Maps to 503 in the HTTP layer."""
+
+
+class ModelMismatchError(RuntimeError):
+    """A backend is reachable but its /api/tags does not list the configured embed model.
+    Config/ops error (wrong URL, model not pulled) -- deliberately loud, maps to 500."""
+
+
+@dataclass
+class _Backend:
+    name: str
+    url: str
+    model_verified: bool = field(default=False)
+
+
+class EmbedRouter:
+    def __init__(
+        self,
+        primary_url: str,
+        fallback_url: str | None = None,
+        *,
+        embed_model: str,
+        primary_name: str = "solaria",
+        fallback_name: str = "piha",
+        health_ttl_s: float = 30.0,
+        health_timeout_s: float = 1.5,
+        primary_embed_timeout_s: float = 3.0,
+        clock=time.monotonic,
+    ):
+        self.primary = _Backend(primary_name, primary_url.rstrip("/"))
+        self.fallback = _Backend(fallback_name, fallback_url.rstrip("/")) if fallback_url else None
+        self.embed_model = embed_model
+        self.health_ttl_s = health_ttl_s
+        self.health_timeout_s = health_timeout_s
+        self.primary_embed_timeout_s = primary_embed_timeout_s
+        self._clock = clock
+        self._sol_status: str | None = None  # "up" | "down" | None (never probed yet)
+        self._checked_at: float = 0.0
+
+    def _cache_valid(self) -> bool:
+        return self._sol_status is not None and (self._clock() - self._checked_at) < self.health_ttl_s
+
+    def _set_status(self, status: str) -> None:
+        self._sol_status = status
+        self._checked_at = self._clock()
+
+    async def primary_status(self, session: aiohttp.ClientSession) -> str:
+        """Cached primary liveness ("up"/"down"), probing /api/tags only when the TTL expired.
+        Shared by /search routing and /healthz so both report the same world view."""
+        if not self._cache_valid():
+            up = await check_ollama_health(session, self.primary.url, self.health_timeout_s)
+            self._set_status("up" if up else "down")
+        return self._sol_status
+
+    async def fallback_status(self, session: aiohttp.ClientSession) -> str:
+        """Live fallback liveness for /healthz: "up"/"down"/"unconfigured". Uncached -- the
+        probe is a cheap /api/tags (does not load the model on ollama-piha) and monitoring
+        calls this every 30 s anyway."""
+        if self.fallback is None:
+            return "unconfigured"
+        up = await check_ollama_health(session, self.fallback.url, self.health_timeout_s)
+        return "up" if up else "down"
+
+    async def _verify_model(self, session: aiohttp.ClientSession, backend: _Backend) -> None:
+        """First-use guard: `/api/tags` of `backend` must list `embed_model` (exact name or
+        same base, so a pulled "bge-m3:latest" satisfies EMBED_MODEL=bge-m3). Raises
+        ModelMismatchError when reachable-but-missing; connection errors propagate to the
+        caller's failover handling. Passed once -> never re-checked for this process."""
+        if backend.model_verified:
+            return
+        async with session.get(
+            f"{backend.url}/api/tags",
+            timeout=aiohttp.ClientTimeout(total=self.health_timeout_s),
+        ) as resp:
+            resp.raise_for_status()
+            data = await resp.json()
+        names = {m.get("name", "") for m in data.get("models", [])}
+        base = self.embed_model.split(":")[0]
+        if not any(n == self.embed_model or n.split(":")[0] == base for n in names):
+            raise ModelMismatchError(
+                f"embed backend {backend.name} ({backend.url}) does not serve "
+                f"model {self.embed_model!r} (available: {sorted(names) or 'none'}) -- "
+                f"refusing to embed in a vector space that doesn't match document_chunk"
+            )
+        backend.model_verified = True
+
+    async def embed(self, session: aiohttp.ClientSession, text: str) -> tuple[list[float], str]:
+        """Embed one query text, returning (embedding, backend_name). Implements the state
+        machine from the module docstring; raises EmbedBackendError (503) when no backend
+        can serve, ModelMismatchError (500) when the fallback serves the wrong model."""
+        if await self.primary_status(session) == "up":
+            try:
+                await self._verify_model(session, self.primary)
+                embedding, elapsed = await asyncio.wait_for(
+                    embed_chunk(session, self.primary.url, self.embed_model, text),
+                    timeout=self.primary_embed_timeout_s,
+                )
+                logger.info(
+                    "query embedded backend=%s url=%s elapsed_ms=%.0f",
+                    self.primary.name, self.primary.url, elapsed * 1000,
+                )
+                return embedding, self.primary.name
+            except ModelMismatchError as exc:
+                # Reachable but serving the wrong model set -- loud, then try the fallback
+                # (which is verified independently below, so this cannot cascade silently).
+                logger.error("primary embed backend rejected: %s", exc)
+                self._set_status("down")
+            except (aiohttp.ClientError, TimeoutError, ValueError) as exc:
+                # Plan step 3b: one-shot flip -- the SAME query continues to the fallback.
+                logger.warning(
+                    "primary embed failed backend=%s url=%s (%s: %s); circuit open for %.0fs",
+                    self.primary.name, self.primary.url, type(exc).__name__, exc, self.health_ttl_s,
+                )
+                self._set_status("down")
+
+        if self.fallback is None:
+            raise EmbedBackendError(
+                f"primary embed backend {self.primary.name} ({self.primary.url}) is down "
+                f"and no EMBED_FALLBACK_URL is configured"
+            )
+
+        try:
+            await self._verify_model(session, self.fallback)
+        except ModelMismatchError:
+            raise  # loud config error -- never silently embed in a mismatched space
+        except (aiohttp.ClientError, TimeoutError) as exc:
+            raise EmbedBackendError(
+                f"primary embed backend {self.primary.name} is down and fallback "
+                f"{self.fallback.name} ({self.fallback.url}) is unreachable: {exc}"
+            ) from exc
+
+        embedding, elapsed = await embed_chunk(session, self.fallback.url, self.embed_model, text)
+        logger.info(
+            "query embedded backend=%s url=%s elapsed_ms=%.0f (fallback; primary=%s down)",
+            self.fallback.name, self.fallback.url, elapsed * 1000, self.primary.name,
+        )
+        return embedding, self.fallback.name

services/kb-query/app/main.py

@@ -1,12 +1,15 @@
 """kb-query -- module 5 phase 4 (docs/kb/modules/05-faza4-plan.md §4): first user-facing HTTP
-entry point to the KB. Wraps `kb_retrieval.cascade_query`/`flat_query` (module 5 phase 3,
-already gated PASS -- docs/sessions/2026-07-21.md) in FastAPI. This is a search API, not chat:
-no answer synthesis, no LLM call over the results (that is phase 5, out of scope here).
+entry point to the KB. Wraps `kb_retrieval` retrieval (module 5 phase 3, gated PASS --
+docs/sessions/2026-07-21.md) in FastAPI. This is a search API, not chat: no answer synthesis,
+no LLM call over the results (that is phase 5, out of scope here).
 
-Embed path is deliberately simple for this step: calls Ollama on SOLARIA directly, no
-cache/circuit-breaker/local-PIHA-fallback (plan §2 decision 2, §5) -- that state machine is a
-later, separate step. A failed embed call (SOLARIA unreachable) surfaces as 503 to the caller
-rather than a bare 500.
+Embed path (Krok 2, plan §2 decision 2 / §5 -- the active fallback): queries embed through
+`app.embed_router.EmbedRouter` -- SOLARIA's GPU Ollama (EMBED_PRIMARY_URL) when its cached
+~30 s health-check says up, the local CPU `ollama-piha` (EMBED_FALLBACK_URL) when SOLARIA
+sleeps. A mid-query primary failure fails over the SAME request instead of surfacing an
+error. Only when BOTH backends are unavailable does `/search` return 503; a backend serving
+the wrong model (vs the bge-m3 space `document_chunk` is indexed in) is a loud 500, never a
+silent cross-space distance computation.
 
 `GET /` (Krok 4, plan §7) serves the search UI from this same FastAPI process -- one image, one
 container (plan §2 decision 4): a Jinja2 shell + a static vanilla-JS file, no node build step.
@@ -18,6 +21,7 @@ available explicitly starting here, but the default stays `cascade` until the qu
 """
 from __future__ import annotations
 
+import logging
 import os
 import pathlib
 from contextlib import asynccontextmanager
@@ -28,18 +32,30 @@ from fastapi.responses import HTMLResponse
 from fastapi.staticfiles import StaticFiles
 from fastapi.templating import Jinja2Templates
 
-from kb_retrieval.embed import check_ollama_health
-
 from app.db import create_pool
+from app.embed_router import EmbedBackendError, EmbedRouter, ModelMismatchError
 from app.search import run_search
 from app.startup import validate_embed_model
 
+# uvicorn only configures its own loggers; without this the router's
+# backend=solaria/piha lines (kb-query.embed) would never reach docker logs.
+logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s")
+
 BASE_DIR = pathlib.Path(__file__).resolve().parent
 KB_DSN = os.environ.get("KB_DSN")
-OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://solaria:11434")
+# EMBED_PRIMARY_URL replaced OLLAMA_URL in Krok 2 (aktywny fallback) -- one naming scheme
+# for both legs. Same default upstream as before: Ollama @ SOLARIA over Tailscale.
+EMBED_PRIMARY_URL = os.environ.get("EMBED_PRIMARY_URL", "http://solaria:11434")
+# Unset/empty -> no fallback leg: /search 503s when SOLARIA is down (pre-Krok-2 behaviour).
+EMBED_FALLBACK_URL = os.environ.get("EMBED_FALLBACK_URL") or None
+EMBED_PRIMARY_NAME = os.environ.get("EMBED_PRIMARY_NAME", "solaria")
+EMBED_FALLBACK_NAME = os.environ.get("EMBED_FALLBACK_NAME", "piha")
 EMBED_MODEL = os.environ.get("EMBED_MODEL", "bge-m3")
 SUMMARY_MODEL = os.environ.get("SUMMARY_MODEL", "claude-haiku-4-5")
-OLLAMA_HEALTH_TIMEOUT_S = 3.0
+# State-machine parameters (plan §2 D2 table; timeouts rationale in app/embed_router.py).
+EMBED_HEALTH_TTL_S = float(os.environ.get("EMBED_HEALTH_TTL_S", "30"))
+EMBED_HEALTH_TIMEOUT_S = float(os.environ.get("EMBED_HEALTH_TIMEOUT_S", "1.5"))
+EMBED_PRIMARY_TIMEOUT_S = float(os.environ.get("EMBED_PRIMARY_TIMEOUT_S", "3"))
 
 
 @asynccontextmanager
@@ -50,11 +66,23 @@ async def lifespan(app: FastAPI):
     pool = await create_pool(KB_DSN)
     async with pool.acquire() as conn:
         # Hard invariant (plan §2 decision 2): refuse to start rather than silently serve
-        # queries against a mismatched embedding space.
+        # queries against a mismatched embedding space. The per-backend half of the same
+        # invariant (does the Ollama actually serve EMBED_MODEL?) lives in EmbedRouter and
+        # runs lazily at each backend's first use -- a sleeping SOLARIA must not block boot.
         await validate_embed_model(conn, EMBED_MODEL)
 
     app.state.pool = pool
     app.state.http = aiohttp.ClientSession()
+    app.state.embed_router = EmbedRouter(
+        EMBED_PRIMARY_URL,
+        EMBED_FALLBACK_URL,
+        embed_model=EMBED_MODEL,
+        primary_name=EMBED_PRIMARY_NAME,
+        fallback_name=EMBED_FALLBACK_NAME,
+        health_ttl_s=EMBED_HEALTH_TTL_S,
+        health_timeout_s=EMBED_HEALTH_TIMEOUT_S,
+        primary_embed_timeout_s=EMBED_PRIMARY_TIMEOUT_S,
+    )
     try:
         yield
     finally:
@@ -74,8 +102,15 @@ async def index(request: Request):
 
 @app.get("/healthz")
 async def healthz() -> dict:
-    sol_up = await check_ollama_health(app.state.http, OLLAMA_URL, OLLAMA_HEALTH_TIMEOUT_S)
-    return {"status": "ok", "sol_status": "up" if sol_up else "down"}
+    # sol_status goes through the router's 30 s cache -- /healthz and /search routing
+    # deliberately share one world view (and monitoring no longer re-probes a sleeping
+    # SOLARIA on every scrape). fallback_status is a live cheap /api/tags probe.
+    router = app.state.embed_router
+    return {
+        "status": "ok",
+        "sol_status": await router.primary_status(app.state.http),
+        "fallback_status": await router.fallback_status(app.state.http),
+    }
 
 
 @app.get("/search")
@@ -86,7 +121,10 @@ async def search(
     try:
         async with app.state.pool.acquire() as conn:
             return await run_search(
-                conn, app.state.http, OLLAMA_URL, q, mode, EMBED_MODEL, SUMMARY_MODEL
+                conn, app.state.http, app.state.embed_router, q, mode, SUMMARY_MODEL
             )
-    except aiohttp.ClientError as exc:
+    except ModelMismatchError as exc:
+        # Config/ops error (backend up but wrong model set) -- 500, deliberately loud.
+        raise HTTPException(status_code=500, detail=str(exc)) from exc
+    except (EmbedBackendError, aiohttp.ClientError) as exc:
         raise HTTPException(status_code=503, detail=f"embed backend unavailable: {exc}") from exc

services/kb-query/app/search.py

@@ -1,6 +1,6 @@
 """`/search` core -- module 5 phase 4 (docs/kb/modules/05-faza4-plan.md §4). Kept decoupled
-from FastAPI so it can be unit-tested with fake `conn`/`session` objects, the same style as
-`kb_retrieval`'s own tests, instead of needing a live DB/Ollama behind a TestClient.
+from FastAPI so it can be unit-tested with fake `conn`/`session`/`router` objects, the same
+style as `kb_retrieval`'s own tests, instead of needing a live DB/Ollama behind a TestClient.
 
 Response shape (plan §4 exactly): `{"query", "mode", "sol_status", "results": [...]}`, each
 result carrying `dist` un-filtered -- the 0.45/0.55 colour thresholds (plan §7) are a frontend
@@ -8,43 +8,48 @@ concern (Krok 4, out of this step's scope), never applied server-side.
 
 `summary`/`summary_tags` (document_summary, haiku track) are an additive field added in Krok 4
 for the frontend's per-envelope result header (plan §7) -- `None`/`[]` when the envelope has no
-summary for `summary_model` yet. Purely additive: does not change any field already covered by
-the phase-4 gate's HTTP-equivalence check (plan §9).
+summary for `summary_model` yet.
+
+Krok 2 (aktywny fallback, plan §5): the query embedding no longer happens inside
+`kb_retrieval.cascade_query`/`flat_query`/`hybrid_query` -- it goes through
+`app.embed_router.EmbedRouter` (SOLARIA primary -> PIHA fallback state machine) and the
+resulting vector feeds the same `*_retrieve` functions those wrappers call. Two additive
+consequences for the response: `sol_status` now reports the router's real world view ("down"
+while serving from the fallback -- the frontend already renders that as "offline (fallback
+embed)"), and `embed_backend` names which backend actually embedded THIS query (task spec:
+needed to debug result quality per backend).
 """
 from __future__ import annotations
 
 import aiohttp
 import asyncpg
 
-from kb_retrieval.retrieval import cascade_query, flat_query, hybrid_query
+from kb_retrieval.embed import _vector_literal  # same private-import convention as kb_retrieval.retrieval
+from kb_retrieval.retrieval import cascade_retrieve, flat_retrieve, hybrid_retrieve
 
 from app.db import fetch_envelopes, fetch_summaries
+from app.embed_router import EmbedRouter
 from app.links import build_result
 
 
 async def run_search(
     conn: asyncpg.Connection,
     session: aiohttp.ClientSession,
-    ollama_url: str,
+    router: EmbedRouter,
     query_text: str,
     mode: str,
-    embed_model: str,
     summary_model: str,
 ) -> dict:
-    if mode == "flat":
-        retrieval = await flat_query(conn, session, ollama_url, query_text, embed_model=embed_model)
-    elif mode == "hybrid":
-        retrieval = await hybrid_query(
-            conn, session, ollama_url, query_text,
-            summary_model=summary_model, embed_model=embed_model,
-        )
-    else:
-        retrieval = await cascade_query(
-            conn, session, ollama_url, query_text,
-            summary_model=summary_model, embed_model=embed_model,
-        )
+    embedding, backend = await router.embed(session, query_text)
+    query_vector = _vector_literal(embedding)
+
+    if mode == "flat":
+        chunks = await flat_retrieve(conn, query_vector)
+    elif mode == "hybrid":
+        chunks = (await hybrid_retrieve(conn, query_vector, summary_model))["chunks"]
+    else:
+        chunks = (await cascade_retrieve(conn, query_vector, summary_model))["chunks"]
 
-    chunks = retrieval["chunks"]
     envelope_ids = sorted({c["envelope_id"] for c in chunks})
     envelopes = await fetch_envelopes(conn, envelope_ids)
     summaries = await fetch_summaries(conn, envelope_ids, summary_model)
@@ -60,6 +65,9 @@ async def run_search(
     return {
         "query": query_text,
         "mode": mode,
-        "sol_status": "up",  # reaching this point means the embed call above succeeded
+        # "up" iff the primary (SOLARIA) embedded this very query -- the fallback path
+        # implies the router just observed the primary down (probe or mid-embed failure).
+        "sol_status": "up" if backend == router.primary.name else "down",
+        "embed_backend": backend,
         "results": results,
     }

services/kb-query/docker-compose.yml

@@ -17,7 +17,17 @@ services:
       - "${LAN_BIND_IP}:8230:8080"
     environment:
       - KB_DSN=${KB_DSN}
-      - OLLAMA_URL=${OLLAMA_URL:-http://solaria:11434}
+      # Embed backends (Krok 2, aktywny fallback — app/embed_router.py): primary is
+      # Ollama @ SOLARIA (GPU, duty-cycled), fallback is ollama-piha on this host
+      # (CPU, OLLAMA_KEEP_ALIVE=0). Empty EMBED_FALLBACK_URL disables the fallback
+      # leg (pre-Krok-2 behaviour: /search 503s while SOLARIA sleeps).
+      - EMBED_PRIMARY_URL=${EMBED_PRIMARY_URL:-http://solaria:11434}
+      - EMBED_FALLBACK_URL=${EMBED_FALLBACK_URL:-}
+      - EMBED_PRIMARY_NAME=${EMBED_PRIMARY_NAME:-solaria}
+      - EMBED_FALLBACK_NAME=${EMBED_FALLBACK_NAME:-piha}
+      - EMBED_HEALTH_TTL_S=${EMBED_HEALTH_TTL_S:-30}
+      - EMBED_HEALTH_TIMEOUT_S=${EMBED_HEALTH_TIMEOUT_S:-1.5}
+      - EMBED_PRIMARY_TIMEOUT_S=${EMBED_PRIMARY_TIMEOUT_S:-3}
       - EMBED_MODEL=${EMBED_MODEL:-bge-m3}
       - SUMMARY_MODEL=${SUMMARY_MODEL:-claude-haiku-4-5}
     # python:3.12-slim has no curl/wget; same in-container check pattern as llm-gateway.

services/kb-query/env.example

@@ -11,9 +11,25 @@ LAN_BIND_IP=192.168.31.5
 # reasoning as paperless-worker@SOLARIA reaching paperless@PIHA over LAN.
 KB_DSN=postgresql://kb:CHANGE-ME@192.168.31.5:5433/kb
 
-# Ollama upstream (SOLARIA, over Tailscale MagicDNS — same trick as
-# llm-gateway's OLLAMA_URL). Optional: defaults to this value if unset.
-# OLLAMA_URL=http://solaria:11434
+# Primary embed backend: Ollama @ SOLARIA (GPU, over Tailscale MagicDNS —
+# same trick as llm-gateway's OLLAMA_URL). Optional: defaults to this value.
+# EMBED_PRIMARY_URL=http://solaria:11434
+
+# Fallback embed backend: ollama-piha on this host (services/ollama-piha/),
+# reached over PIHA's LAN interface (same pattern as KB_DSN above). SOLARIA is
+# powered off ~16 h/day — without this, /search is dead the whole time. Leave
+# unset/empty to disable the fallback leg entirely.
+EMBED_FALLBACK_URL=http://192.168.31.5:11434
+
+# Backend labels used in logs and the /search "embed_backend" response field.
+# Optional: defaults solaria / piha.
+# EMBED_PRIMARY_NAME=solaria
+# EMBED_FALLBACK_NAME=piha
+
+# Fallback state machine tuning (plan §2 D2). Optional; defaults shown.
+# EMBED_HEALTH_TTL_S=30        # how long an up/down verdict on SOLARIA is cached
+# EMBED_HEALTH_TIMEOUT_S=1.5   # /api/tags probe timeout
+# EMBED_PRIMARY_TIMEOUT_S=3    # hard timeout for a primary embed call
 
 # Embedding model kb-query enforces as a startup invariant (plan §2 decision
 # 2) against document_chunk.model / document_summary.embedding_model.

services/kb-query/service.yaml

@@ -5,10 +5,12 @@ service:
   exposure: private               # LAN/Tailscale only, npm@PIHA vhost (kb.kapala.org) is a later step
   dependencies:
     - kb-postgres
+    - ollama-piha                 # soft: local CPU embed fallback (Krok 2) -- kb-query starts without it
     - forgejo                     # OIDC identity provider, wired in a later step (plan §8); not yet enforced
-  # ollama@SOLARIA is an external, optional runtime dependency (embed calls), not a hard
-  # dependency here -- SOLARIA may be offline; /search then fails per-request with 503,
-  # /healthz still answers (same tolerance pattern as llm-gateway).
+  # ollama@SOLARIA is an external, optional runtime dependency (primary embed backend), not
+  # a hard dependency here -- SOLARIA sleeps ~16 h/day; the embed router (app/embed_router.py)
+  # then falls back to ollama-piha on this host. /search 503s only when BOTH legs are down;
+  # /healthz always answers (same tolerance pattern as llm-gateway).
   ports:
     - container: 8080
       host: 8230                  # LAN_BIND_IP only, never 0.0.0.0
@@ -28,6 +30,12 @@ service:
     env_vars:
       - LAN_BIND_IP                # required — compose port-bind interpolation
       - KB_DSN                     # required — asyncpg DSN for kb-postgres@PIHA
-      - OLLAMA_URL                 # optional — defaults to http://solaria:11434
-      - EMBED_MODEL                 # optional — defaults to bge-m3; startup invariant vs document_chunk/document_summary
-      - SUMMARY_MODEL               # optional — defaults to claude-haiku-4-5; cascade_query's stage-1 model
+      - EMBED_PRIMARY_URL          # optional — primary embed backend, defaults to http://solaria:11434
+      - EMBED_FALLBACK_URL         # optional — local fallback (ollama-piha); empty disables the fallback leg
+      - EMBED_PRIMARY_NAME         # optional — log/response label, defaults to solaria
+      - EMBED_FALLBACK_NAME        # optional — log/response label, defaults to piha
+      - EMBED_HEALTH_TTL_S         # optional — primary health verdict cache, defaults to 30
+      - EMBED_HEALTH_TIMEOUT_S     # optional — /api/tags probe timeout, defaults to 1.5
+      - EMBED_PRIMARY_TIMEOUT_S    # optional — hard timeout for a primary embed call, defaults to 3
+      - EMBED_MODEL                 # optional — defaults to bge-m3; startup invariant vs document_chunk/document_summary + per-backend /api/tags check
+      - SUMMARY_MODEL               # optional — defaults to claude-haiku-4-5; cascade stage-1 model

services/kb-query/tests/test_embed_router.py

@@ -0,0 +1,224 @@
+"""Unit tests for the SOLARIA->PIHA embed fallback state machine (app.embed_router) -- no
+real Ollama on either side. Fake backends are keyed by URL so one fake session can serve
+both legs; the clock is injected so the 30 s TTL is tested without sleeping."""
+from __future__ import annotations
+
+import pathlib
+import sys
+
+import aiohttp
+import pytest
+
+sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
+
+from app.embed_router import EmbedBackendError, EmbedRouter, ModelMismatchError  # noqa: E402
+
+PRIMARY = "http://solaria:11434"
+FALLBACK = "http://192.168.31.5:11434"
+
+
+class _FakeResponse:
+    def __init__(self, payload=None, exc=None, status=200):
+        self._payload = payload
+        self._exc = exc
+        self.status = status  # check_ollama_health reads resp.status directly
+
+    async def __aenter__(self):
+        if self._exc is not None:
+            raise self._exc
+        return self
+
+    async def __aexit__(self, *exc):
+        return False
+
+    def raise_for_status(self):
+        pass
+
+    async def json(self):
+        return self._payload
+
+
+class _FakeBackend:
+    """One fake Ollama: `models` drives /api/tags, `down=True` refuses every connection,
+    `embed_exc` makes /api/embeddings fail while /api/tags still answers (the plan's step-3b
+    scenario: probe says up, the real embed then dies mid-query)."""
+
+    def __init__(self, models=("bge-m3:latest",), down=False, embed_exc=None, vector_value=0.01):
+        self.models = list(models)
+        self.down = down
+        self.embed_exc = embed_exc
+        self.vector_value = vector_value
+        self.tags_calls = 0
+        self.embed_calls = 0
+
+
+class _FakeSession:
+    def __init__(self, backends: dict):
+        self._backends = backends
+
+    def _backend(self, url):
+        for base, backend in self._backends.items():
+            if url.startswith(base):
+                return backend
+        raise AssertionError(f"unexpected url: {url}")
+
+    def get(self, url, timeout=None):
+        backend = self._backend(url)
+        backend.tags_calls += 1
+        if backend.down:
+            return _FakeResponse(exc=aiohttp.ClientConnectionError("connection refused"))
+        return _FakeResponse({"models": [{"name": n} for n in backend.models]})
+
+    def post(self, url, json):
+        backend = self._backend(url)
+        backend.embed_calls += 1
+        if backend.down:
+            return _FakeResponse(exc=aiohttp.ClientConnectionError("connection refused"))
+        if backend.embed_exc is not None:
+            return _FakeResponse(exc=backend.embed_exc)
+        return _FakeResponse({"embedding": [backend.vector_value] * 1024})
+
+
+class _FakeClock:
+    def __init__(self):
+        self.now = 1000.0
+
+    def __call__(self):
+        return self.now
+
+
+def _router(fallback_url=FALLBACK, clock=None, **kwargs):
+    return EmbedRouter(
+        PRIMARY,
+        fallback_url,
+        embed_model="bge-m3",
+        clock=clock or _FakeClock(),
+        **kwargs,
+    )
+
+
+class TestPrimaryHealthy:
+    async def test_embeds_on_primary_and_reports_its_name(self):
+        solaria = _FakeBackend()
+        piha = _FakeBackend()
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        embedding, backend = await _router().embed(session, "q")
+        assert backend == "solaria"
+        assert len(embedding) == 1024
+        assert solaria.embed_calls == 1
+        assert piha.embed_calls == 0
+
+    async def test_health_verdict_is_cached_within_ttl(self):
+        solaria = _FakeBackend()
+        clock = _FakeClock()
+        router = _router(clock=clock)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: _FakeBackend()})
+        await router.embed(session, "q1")
+        probes_after_first = solaria.tags_calls  # probe + first-use model verification
+        clock.now += 10  # inside the 30 s TTL
+        await router.embed(session, "q2")
+        assert solaria.tags_calls == probes_after_first  # no new probe, model check done once
+
+    async def test_model_tag_with_latest_suffix_satisfies_bare_model_name(self):
+        solaria = _FakeBackend(models=["bge-m3:latest"])
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: _FakeBackend()})
+        _, backend = await _router().embed(session, "q")
+        assert backend == "solaria"
+
+
+class TestFailover:
+    async def test_primary_down_at_probe_falls_back_to_piha(self):
+        solaria = _FakeBackend(down=True)
+        piha = _FakeBackend(vector_value=0.02)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        embedding, backend = await _router().embed(session, "q")
+        assert backend == "piha"
+        assert embedding[0] == 0.02
+        assert solaria.embed_calls == 0
+
+    async def test_mid_embed_failure_serves_same_query_from_fallback(self):
+        # Plan §2 D2 step 3b: probe said up, the real embed then dies -- the SAME request
+        # must come back from the fallback, never as an error to the user.
+        solaria = _FakeBackend(embed_exc=aiohttp.ClientConnectionError("died mid-embed"))
+        piha = _FakeBackend(vector_value=0.02)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        router = _router()
+        embedding, backend = await router.embed(session, "q")
+        assert backend == "piha"
+        assert embedding[0] == 0.02
+        assert solaria.embed_calls == 1
+
+    async def test_down_verdict_is_cached_and_skips_primary_until_ttl_expires(self):
+        solaria = _FakeBackend(down=True)
+        piha = _FakeBackend()
+        clock = _FakeClock()
+        router = _router(clock=clock)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        await router.embed(session, "q1")
+        probes = solaria.tags_calls
+        clock.now += 10  # still inside TTL -> no re-probe, straight to fallback
+        _, backend = await router.embed(session, "q2")
+        assert backend == "piha"
+        assert solaria.tags_calls == probes
+
+    async def test_traffic_returns_to_primary_after_ttl_expiry(self):
+        # Test C from the task spec: SOLARIA comes back -> within one TTL window the
+        # router re-probes and routes to the GPU again.
+        solaria = _FakeBackend(down=True)
+        piha = _FakeBackend()
+        clock = _FakeClock()
+        router = _router(clock=clock)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        _, backend = await router.embed(session, "q1")
+        assert backend == "piha"
+        solaria.down = False  # SOLARIA wakes up
+        clock.now += 31  # TTL (30 s) expired
+        _, backend = await router.embed(session, "q2")
+        assert backend == "solaria"
+
+
+class TestNoUsableBackend:
+    async def test_primary_down_without_fallback_raises_backend_error(self):
+        solaria = _FakeBackend(down=True)
+        session = _FakeSession({PRIMARY: solaria})
+        with pytest.raises(EmbedBackendError, match="no EMBED_FALLBACK_URL"):
+            await _router(fallback_url=None).embed(session, "q")
+
+    async def test_both_backends_down_raises_backend_error(self):
+        session = _FakeSession({PRIMARY: _FakeBackend(down=True), FALLBACK: _FakeBackend(down=True)})
+        with pytest.raises(EmbedBackendError, match="unreachable"):
+            await _router().embed(session, "q")
+
+
+class TestModelInvariant:
+    async def test_fallback_without_bge_m3_is_a_loud_error_not_a_silent_embed(self):
+        solaria = _FakeBackend(down=True)
+        piha = _FakeBackend(models=["llama3:8b"])
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        with pytest.raises(ModelMismatchError, match="bge-m3"):
+            await _router().embed(session, "q")
+        assert piha.embed_calls == 0  # never embedded in the wrong vector space
+
+    async def test_primary_without_bge_m3_fails_over_to_verified_fallback(self):
+        solaria = _FakeBackend(models=["llama3:8b"])
+        piha = _FakeBackend()
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        _, backend = await _router().embed(session, "q")
+        assert backend == "piha"
+        assert solaria.embed_calls == 0
+
+
+class TestStatusReporting:
+    async def test_primary_status_up_and_fallback_status_up(self):
+        session = _FakeSession({PRIMARY: _FakeBackend(), FALLBACK: _FakeBackend()})
+        router = _router()
+        assert await router.primary_status(session) == "up"
+        assert await router.fallback_status(session) == "up"
+
+    async def test_fallback_status_unconfigured_without_fallback_url(self):
+        session = _FakeSession({PRIMARY: _FakeBackend()})
+        assert await _router(fallback_url=None).fallback_status(session) == "unconfigured"
+
+    async def test_primary_status_down_when_probe_fails(self):
+        session = _FakeSession({PRIMARY: _FakeBackend(down=True), FALLBACK: _FakeBackend()})
+        assert await _router().primary_status(session) == "down"

services/kb-query/tests/test_search.py

@@ -75,30 +75,23 @@ class _FakeConn:
         raise AssertionError(f"unexpected query: {query}")
 
 
-class _FakeEmbedResponse:
-    def __init__(self, payload):
-        self._payload = payload
-
-    async def __aenter__(self):
-        return self
-
-    async def __aexit__(self, *exc):
-        return False
-
-    def raise_for_status(self):
-        pass
-
-    async def json(self):
-        return self._payload
-
-
 class _FakeSession:
-    def __init__(self):
-        self.post_calls: list[dict] = []
+    """run_search no longer talks HTTP itself -- embedding goes through the router
+    (below), so the session is just passed through untouched."""
 
-    def post(self, url, json):
-        self.post_calls.append({"url": url, "json": json})
-        return _FakeEmbedResponse({"embedding": [0.01] * 1024})
+
+class _FakeRouter:
+    """Stands in for app.embed_router.EmbedRouter: returns a fixed embedding and the name
+    of the backend that 'served' it, mirroring EmbedRouter.embed's contract."""
+
+    def __init__(self, backend="solaria", primary_name="solaria"):
+        self._backend = backend
+        self.primary = type("B", (), {"name": primary_name})()
+        self.embed_calls: list[str] = []
+
+    async def embed(self, session, text):
+        self.embed_calls.append(text)
+        return [0.01] * 1024, self._backend
 
 
 class TestRunSearchHappyPath:
@@ -110,11 +103,12 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
         )
         assert result["query"] == "polisa PZU"
         assert result["mode"] == "cascade"
         assert result["sol_status"] == "up"
+        assert result["embed_backend"] == "solaria"
         assert len(result["results"]) == 1
         hit = result["results"][0]
         assert hit["envelope_id"] == "paperless:119"
@@ -131,7 +125,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "q", "flat", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "q", "flat", "claude-haiku-4-5"
         )
         assert result["mode"] == "flat"
         assert len(result["results"]) == 1
@@ -145,7 +139,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
         )
         hit = result["results"][0]
         assert hit["summary"] == "Polisa OC 2024"
@@ -159,7 +153,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
         )
         hit = result["results"][0]
         assert hit["summary"] is None
@@ -180,7 +174,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "q", "hybrid", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "q", "hybrid", "claude-haiku-4-5"
         )
         assert result["mode"] == "hybrid"
         envelope_ids = [r["envelope_id"] for r in result["results"]]
@@ -201,7 +195,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "q", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "q", "cascade", "claude-haiku-4-5"
         )
         hit = result["results"][0]
         assert hit["source"] == "gmail"
@@ -210,6 +204,24 @@ class TestRunSearchHappyPath:
         assert hit["mail_ui_url"] is None
 
 
+class TestRunSearchEmbedBackendMarking:
+    async def test_fallback_embed_marks_backend_piha_and_sol_status_down(self):
+        # Task spec: the response must say WHICH backend embedded the query (quality
+        # debugging), and sol_status must reflect the router's world view -- the frontend
+        # renders "down" as "offline (fallback embed)".
+        conn = _FakeConn(
+            summaries=[("paperless:119", 0.1)],
+            chunks_by_envelope={"paperless:119": [(2, "hit text", 0.34)]},
+            envelopes={"paperless:119": {"source": "paperless", "entities": []}},
+        )
+        result = await run_search(
+            conn, _FakeSession(), _FakeRouter(backend="piha"), "q", "cascade", "claude-haiku-4-5"
+        )
+        assert result["embed_backend"] == "piha"
+        assert result["sol_status"] == "down"
+        assert len(result["results"]) == 1
+
+
 class TestRunSearchNoGoodResults:
     async def test_results_above_no_answer_threshold_are_still_returned_unfiltered(self):
         # Plan §7: the 0.55 "no answer" colour threshold is a frontend concern -- the API
@@ -222,7 +234,7 @@ class TestRunSearchNoGoodResults:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "unrelated query", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "unrelated query", "cascade", "claude-haiku-4-5"
         )
         assert len(result["results"]) == 1
         assert result["results"][0]["dist"] == 0.62
@@ -231,6 +243,6 @@ class TestRunSearchNoGoodResults:
         conn = _FakeConn(summaries=[], chunks_by_envelope={}, envelopes={})
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "nothing matches", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "nothing matches", "cascade", "claude-haiku-4-5"
         )
         assert result["results"] == []

services/mosquitto/README.md

@@ -1,9 +1,15 @@
-# Mosquitto MQTT Broker
+# Mosquitto MQTT Broker — NOT DEPLOYED / LEGACY
+
+> **Status (2026-07-28, truth cleanup):** this manifest matches **nothing that
+> actually runs** (recon `docs/architecture/RECON-multiagent-2026-07-27.md`, C8).
+> The broker on **vps** is part of the legacy ai-cluster stack (being
+> decommissioned, see `docs/architecture/ai-cluster-LEGACY.md`); the broker on
+> **piha** is a host systemd mosquitto (OS package) with no repo definition;
+> **chelsty** has its own config under `hosts/chelsty-infra/runtime/mosquitto/`.
+> Do not deploy from this directory. Kept until the MQTT topology decision
+> (recon open question 7) is made.
 
 Eclipse Mosquitto is an open source (EPL/EDL licensed) message broker that implements the MQTT protocol versions 5.0, 3.1.1 and 3.1.
 
-## Usage
-Deployed on the `piha` node.
-
 Port 1883 for standard MQTT.
 Port 9001 for WebSockets.

services/mosquitto/service.yaml

@@ -1,6 +1,13 @@
+# NOT DEPLOYED — LEGACY MANIFEST (2026-07-28 truth cleanup, recon C8).
+# This definition matches NOTHING that actually runs: the broker on vps belongs
+# to the legacy ai-cluster stack (being decommissioned), and piha's broker is a
+# host systemd mosquitto (OS package, no repo definition). The chelsty broker
+# has its own config under hosts/chelsty-infra/runtime/mosquitto/. Kept for
+# reference until the MQTT topology decision (recon open question 7) is made —
+# do not deploy from this directory.
 service:
   name: mosquitto
-  owner_node: vps
+  owner_node: vps  # WRONG in practice — see header; README used to say piha
   exposure: private
   dependencies: []
   ports:

services/ollama-piha/README.md

@@ -0,0 +1,76 @@
+# ollama-piha
+
+Local CPU Ollama on **PIHA**, serving exactly one purpose: the **fallback embed
+backend** for `kb-query` while SOLARIA (the GPU node, ~16 h/day powered off)
+sleeps. Model: `bge-m3` — the **same** model as SOLARIA's Ollama, because query
+embeddings must live in the same vector space as the pgvector index
+(`document_chunk.embedding VECTOR(1024)`); a different/smaller model is not an
+option (module 5 phase 4 plan §2 decision 2).
+
+Expected latency: bge-m3 embeds in ~207 ms on SOLARIA's GPU vs ~790 ms on x86
+CPU; on the Pi 5 expect single seconds per query (plus model load, since the
+model is never resident — see below). Slower but alive beats fast but dead.
+
+## Design constraints
+
+- **`OLLAMA_KEEP_ALIVE=0`** (pinned in compose): PIHA is the RAM-bound 8 GB box
+  shared with Home Assistant. The model is unloaded immediately after every
+  call — a transient ~1.5–2 GB spike per embed, ~100 MB idle daemon, never a
+  resident cost.
+- **`mem_limit: 2560m`** (host override, `hosts/piha/runtime/ollama-piha/`):
+  hard cgroup ceiling, plan §2 D2 starting value. The cgroup OOM killer
+  restarts this container instead of the host OOM killer picking a victim
+  (which could be Home Assistant). Confirm/trim after live calibration.
+- **Bind**: `127.0.0.1` + `LAN_BIND_IP` (192.168.31.5) only — kb-query calls it
+  over the host LAN interface (same pattern as kb-query → kb-postgres:5433).
+  Never `0.0.0.0`, never a Tailscale bind, no public ingress.
+- **Storage**: Docker named volume `ollama_piha_models` (NVMe data-root), not a
+  bind mount — the ollama image runs as in-container root and would break
+  PIHA's uid pattern (host oskar=1004, containers uid 1000, setgid group pi)
+  if it wrote to a shared bind directory.
+
+## Deploy (PIHA, master, after merge)
+
+```bash
+cd ~/homelab-codex-ws && git pull
+cp services/ollama-piha/env.example services/ollama-piha/.env   # LAN_BIND_IP
+docker compose -f services/ollama-piha/docker-compose.yml \
+  -f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
+  --env-file services/ollama-piha/.env up -d
+```
+
+**Then pull the model — this does NOT happen automatically:**
+
+```bash
+docker exec ollama-piha ollama pull bge-m3
+```
+
+Verify:
+
+```bash
+services/ollama-piha/healthcheck.sh          # checks container + API + bge-m3 present
+time curl -s http://127.0.0.1:11434/api/embeddings \
+  -d '{"model":"bge-m3","prompt":"test kalibracyjny"}' | head -c 80
+```
+
+(`deploy-node.sh` on PIHA also picks this service up from
+`hosts/piha/services.yaml` once `.env` exists — the `ollama pull bge-m3` step
+stays manual either way.)
+
+## Calibration (plan §5 step 4 — gate, not formality)
+
+Before trusting the fallback under load, on live PIHA at a normal (not
+night-quiet) hour: run a few embeds as above while watching
+`docker stats ollama-piha`, note peak RAM and wall time. Verdict per plan §5
+step 5: keep as default fallback / tune `mem_limit` / fall back to explicit
+503 degradation.
+
+## Relation to kb-query
+
+kb-query's router (`services/kb-query/app/embed_router.py`) health-checks
+SOLARIA with a ~30 s cache and only sends embeds here while SOLARIA is down.
+kb-query verifies at first use that this backend actually serves `bge-m3`
+(`/api/tags`) and refuses to embed against a mismatched model. Configuration:
+`EMBED_FALLBACK_URL=http://192.168.31.5:11434` in `services/kb-query/.env`.
+See `services/kb-query/README.md` for the fallback verification plan (tests
+A/B/C).

services/ollama-piha/docker-compose.yml

@@ -0,0 +1,33 @@
+services:
+  ollama-piha:
+    # Multi-arch image with a published linux/arm64 manifest — runs natively on
+    # the Pi 5, no emulation (same mechanism as pgvector/pgvector:pg16 on PIHA).
+    image: ollama/ollama:latest
+    container_name: ollama-piha
+    restart: unless-stopped
+    environment:
+      # PIHA is the RAM-bound 8 GB box shared with Home Assistant. bge-m3 must be
+      # unloaded from RAM immediately after every call — a short spike per embed
+      # (idle daemon ~100 MB), never a resident ~2 GB cost. Fallback traffic is
+      # rare by design: kb-query only embeds here while SOLARIA sleeps.
+      - OLLAMA_KEEP_ALIVE=0
+    ports:
+      # Loopback: healthcheck.sh + manual curl on the node itself.
+      # LAN IP: kb-query runs in a separate compose project/network and reaches
+      # this backend over the host's LAN interface — same reasoning as
+      # kb-query -> kb-postgres (:5433). No 0.0.0.0, no Tailscale bind: nothing
+      # off-node ever embeds here (SOLARIA-side callers use SOLARIA's own Ollama).
+      # Requires .env (from env.example) next to this file at deploy.
+      - "127.0.0.1:11434:11434"
+      - "${LAN_BIND_IP}:11434:11434"
+    volumes:
+      # Named volume, NOT a bind mount: the ollama image runs as in-container
+      # root and would fill a bind dir with root-owned files, breaking PIHA's
+      # uid pattern (host oskar=1004, containers uid 1000 + setgid group pi).
+      # Docker-managed volumes live under the data-root on the NVMe — same
+      # convention as kb_postgres_data.
+      - ollama_piha_models:/root/.ollama
+
+volumes:
+  ollama_piha_models:
+    name: ollama_piha_models

services/ollama-piha/env.example

@@ -0,0 +1,7 @@
+# ollama-piha — copy to .env (gitignored) next to docker-compose.yml; docker
+# compose picks it up automatically. Same convention as services/kb-query.
+
+# LAN IP of PIHA. The published port (11434) binds ONLY to 127.0.0.1 and this
+# interface — never 0.0.0.0. kb-query's EMBED_FALLBACK_URL points at
+# http://<LAN_BIND_IP>:11434. Verify after host rebuilds: ip -4 addr.
+LAN_BIND_IP=192.168.31.5

services/ollama-piha/healthcheck.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+# Healthcheck for ollama-piha (local CPU embed fallback for kb-query)
+
+# Container must be running
+if ! docker ps --filter "name=ollama-piha" --filter "status=running" | grep -qw "ollama-piha"; then
+    echo "[FAIL] ollama-piha container is not running"
+    exit 1
+fi
+
+# API must answer on loopback
+TAGS=$(curl -sf http://127.0.0.1:11434/api/tags)
+if [ $? -ne 0 ]; then
+    echo "[FAIL] ollama-piha API is not responding on 127.0.0.1:11434"
+    exit 1
+fi
+
+# bge-m3 must be pulled — without it the API answers /api/tags but every kb-query
+# fallback embed fails the model check (deploy step 3 in README.md).
+if ! echo "$TAGS" | grep -q '"bge-m3'; then
+    echo "[FAIL] model bge-m3 missing (run: docker exec ollama-piha ollama pull bge-m3)"
+    exit 1
+fi
+
+echo "[OK] ollama-piha is healthy (bge-m3 present)"
+exit 0

services/ollama-piha/service.yaml

@@ -0,0 +1,26 @@
+service:
+  name: ollama-piha
+  owner_node: piha
+  role: embed-fallback            # module 5 phase 4 (plan §2 D2/§5): local CPU bge-m3 embed backend for kb-query while SOLARIA sleeps
+  exposure: private               # 127.0.0.1 + LAN bind only — never 0.0.0.0, never public ingress, no Tailscale bind
+  dependencies: []
+  ports:
+    - container: 11434
+      host: 11434                 # bound to 127.0.0.1 and LAN_BIND_IP only
+      protocol: tcp
+  healthcheck:
+    type: http
+    endpoint: http://127.0.0.1:11434/api/tags
+    interval: 1m
+    timeout: 10s
+    retries: 3
+  restart_policy: unless-stopped
+  persistence:
+    paths: []                     # models live in the Docker named volume ollama_piha_models (NVMe data-root)
+  runtime:
+    config_files:
+      - .env                      # LAN_BIND_IP (gitignored, from env.example)
+    env_vars:
+      - LAN_BIND_IP               # required — compose port-bind interpolation
+    # OLLAMA_KEEP_ALIVE is pinned to 0 in docker-compose.yml (not operator-tunable):
+    # the model must never stay resident on the RAM-bound, HA-sharing Pi 5.

services/stability-agent/service.yaml

@@ -1,6 +1,10 @@
 service:
   name: stability-agent
-  owner_node: chelsty
+  # Deployed per-host: runs on vps, piha and solaria (and chelsty-infra once the
+  # site is revived). Was owner_node: chelsty — not a real node name; fixed during
+  # the 2026-07 truth cleanup (recon B7/F20.5), closing the docs/backlog.md
+  # "stability-agent / node_exporter owner_node single" item for stability-agent.
+  owner_node: per-host
   exposure: private
   dependencies: []
   healthcheck: