feat(kb): aktywny fallback embeddingów SOLARIA→PIHA dla kb-query (faza 4 Krok 2)

Ostatni krok fazy 4 KB (plan §2 Decyzja 2, §5): kb-query przestaje być martwe
przez ~16 h/dobę, gdy SOLARIA (GPU) śpi — zapytania embeduje wtedy lokalna
Ollama CPU na PIHA (wolniej: ~790 ms+ vs ~207 ms na GPU, ale działa).

Nowy serwis services/ollama-piha (GitOps, owner_node: piha):
- ollama/ollama:latest (arm64 natywnie), OLLAMA_KEEP_ALIVE=0 — model zwalnia
  RAM natychmiast po każdym wywołaniu (spike, nie rezydent; PIHA dzieli 8 GB z HA)
- bind wyłącznie 127.0.0.1 + LAN_BIND_IP (192.168.31.5), nigdy 0.0.0.0/Tailscale
- named volume ollama_piha_models (NVMe data-root) zamiast bind-mounta — obraz
  biega jako root w kontenerze i bind łamałby wzorzec uid PIHA (oskar=1004,
  kontenery uid 1000, setgid pi)
- override hosts/piha/runtime/ollama-piha: mem_limit 2560m (wartość startowa
  z planu, do potwierdzenia kalibracją na żywo), świadomie bez mem_reservation
- pull bge-m3 to jawny, ręczny krok deployu (README) — obraz nie ma modeli

kb-query — maszyna stanów fallbacku (app/embed_router.py):
- health-check SOLARII (GET /api/tags, timeout 1.5 s) z cache 30 s — zero
  sondowania per request; po powrocie SOLARII ruch wraca na GPU w ≤30 s
- primary up → embed na SOLARII z twardym timeoutem 3 s; błąd W TRAKCIE
  zapytania = jednorazowe przełączenie (krok 3b planu): status down na 30 s
  i TO SAMO zapytanie leci na fallback — user nie widzi błędu SOLARII
- primary down → embed prosto na ollama-piha (bez twardego timeoutu: CPU +
  zimny load modelu to legalnie pojedyncze sekundy)
- 503 tylko gdy oba backendy padłe (lub fallback nieskonfigurowany)
- inwariant modelu, druga połowa: każdy backend weryfikowany raz, leniwie przy
  pierwszym użyciu, że /api/tags zawiera EMBED_MODEL (bge-m3 — ta sama wartość
  co startowy check przeciw document_chunk.model/document_summary.embedding_model);
  niezgodność = ERROR log + 500, nigdy ciche liczenie dystansów między
  różnymi przestrzeniami embeddingów; leniwie, bo śpiąca SOLARIA nie może
  blokować startu serwisu
- odpowiedź /search: nowe pole embed_backend ("solaria"|"piha") + sol_status
  wg realnego świata routera (UI już renderuje down jako "offline (fallback
  embed)"); log INFO backend=... elapsed_ms=... per zapytanie
- /healthz: sol_status przez cache routera (spójny widok z routingiem) +
  fallback_status (żywa, tania sonda /api/tags)

Konfiguracja spójnie przez env (compose + env.example + service.yaml + README):
EMBED_PRIMARY_URL (zastępuje OLLAMA_URL), EMBED_FALLBACK_URL (pusty = brak
fallbacku, zachowanie sprzed kroku 2), EMBED_{PRIMARY,FALLBACK}_NAME,
EMBED_HEALTH_TTL_S/EMBED_HEALTH_TIMEOUT_S/EMBED_PRIMARY_TIMEOUT_S.

Testy: 39 pass (14 nowych w test_embed_router.py: cache TTL, failover w trakcie
zapytania, powrót po TTL, oba padłe, mismatch modelu na primary i fallbacku,
tag "bge-m3:latest" vs "bge-m3"); docker build + smoke (importy + uvicorn do
guardu KB_DSN) OK; compose config OK dla obu stacków.

Deploy (Oskar, na PIHA z mastera po merge):
  cd ~/homelab-codex-ws && git pull
  # 1. ollama-piha
  cp services/ollama-piha/env.example services/ollama-piha/.env
  docker compose -f services/ollama-piha/docker-compose.yml \
    -f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
    --env-file services/ollama-piha/.env up -d
  docker exec ollama-piha ollama pull bge-m3     # ręczny krok, obowiązkowy
  services/ollama-piha/healthcheck.sh
  # 2. kb-query (dopisać fallback do istniejącego .env)
  echo 'EMBED_FALLBACK_URL=http://192.168.31.5:11434' >> services/kb-query/.env
  docker compose -f services/kb-query/docker-compose.yml \
    -f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build
  services/kb-query/healthcheck.sh
  # (deploy-node.sh też podniesie oba serwisy z hosts/piha/services.yaml,
  #  ale pull bge-m3 i .env pozostają ręczne)
Weryfikacja: testy A/B/C w services/kb-query/README.md (backend=solaria przy
SOLARII online; backend=piha przy symulacji offline; powrót na GPU w ≤30 s).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

hosts/piha/runtime/ollama-piha/docker-compose.override.yml

@@ -0,0 +1,18 @@
+# PIHA-specific overrides for ollama-piha (KB module 5, phase 4 — embed fallback).
+#
+# RESOURCE CONTEXT: PIHA is the RAM-bound 8 GB box shared with Home Assistant.
+# bge-m3 is ~1.2 GB on disk, estimated 1.5–2 GB resident during an embed burst
+# (plan §2 decision 2). OLLAMA_KEEP_ALIVE=0 (base compose) makes that a short
+# per-call spike, not a resident cost — but the spike lands exactly when other
+# SOLARIA-fallback consumers (e.g. paperless fallback OCR) may also be awake,
+# so it gets a hard ceiling.
+services:
+  ollama-piha:
+    # Plan §2 D2 starting value — the cgroup OOM killer restarts this container
+    # on breach instead of the host OOM killer picking a victim (which could be
+    # Home Assistant). Confirm or trim after live calibration (plan §5 step 4:
+    # a few embeds + docker stats at a normal-load hour).
+    mem_limit: 2560m
+    # Deliberately no mem_reservation: the working set is a transient spike and
+    # the idle daemon is ~100 MB — soft-reserving gigabytes would permanently
+    # take them from HA for a backend that mostly sleeps.

hosts/piha/services.yaml

@@ -105,14 +105,37 @@ services:
       # data is in Docker named volume kb_postgres_data — must land on the NVMe
       # (Docker data-root on /home); see hosts/piha/runtime/kb-postgres override.
 
+  ollama-piha:
+    role: embed-fallback         # module 5 phase 4 (plan §2 D2/§5): local CPU bge-m3 for kb-query while SOLARIA sleeps
+    deployment_model: docker-compose
+    exposure: private            # 127.0.0.1 + LAN bind (LAN_BIND_IP) only; no public/Tailscale exposure
+    offline_required: false
+    depends_on:
+      local: []
+      external: []
+    ports:
+      - name: http
+        container_port: 11434
+        host_port: 11434
+        protocol: tcp
+    runtime:
+      # .env (LAN_BIND_IP) lives alongside the compose file; models live in the
+      # Docker named volume ollama_piha_models (NVMe data-root, like kb-postgres).
+      # Deploy note: `docker exec ollama-piha ollama pull bge-m3` is a manual
+      # post-deploy step — the image ships no models.
+      config_path: services/ollama-piha
+
   kb-query:
     role: kb-search-api          # module 5 phase 4: FastAPI wrapper over kb_retrieval cascade/flat query
     deployment_model: docker-compose
     exposure: private            # LAN bind (LAN_BIND_IP); npm@PIHA vhost + OIDC is a later step
     offline_required: false
     depends_on:
-      local: [kb-postgres]
+      # ollama-piha is a soft local dependency: the embed fallback while SOLARIA
-      external: [ollama]         # SOLARIA may be offline -> /search 502/503, /healthz stays ok
+      # sleeps — kb-query starts and serves /healthz without it, /search degrades
+      # to 503 only when BOTH embed backends are unreachable.
+      local: [kb-postgres, ollama-piha]
+      external: [ollama]         # primary embed backend @ SOLARIA; may be offline -> fallback to ollama-piha
     ports:
       - name: http
         container_port: 8080

services/kb-query/README.md

@@ -12,7 +12,7 @@ that's phase 5.
 |---|---|---|
 | `/` | GET | Search UI (Jinja2 shell + `/static/app.js`, no login yet — plan §8 OIDC is a later step) |
 | `/static/*` | GET | UI assets (`app.js`, `style.css`) |
-| `/healthz` | GET | `{"status": "ok", "sol_status": "up"\|"down"}` — `sol_status` is a live probe of Ollama@SOLARIA, no auth required (monitoring must reach it) |
+| `/healthz` | GET | `{"status": "ok", "sol_status": "up"\|"down", "fallback_status": "up"\|"down"\|"unconfigured"}` — `sol_status` comes from the embed router's ~30 s-cached SOLARIA probe (same world view `/search` routes by), `fallback_status` is a live cheap probe of ollama-piha; no auth required (monitoring must reach it) |
 | `/search?q=<text>&mode=cascade\|flat` | GET | `query_text -> embed -> cascade_query/flat_query -> results`, `mode` defaults to `cascade` |
 
 `/search` response shape (module 5 phase 4 plan §4, `summary`/`summary_tags`
@@ -21,7 +21,7 @@ change any field the plan §4 shape already defined):
 
 ```json
 {
-  "query": "...", "mode": "cascade", "sol_status": "up",
+  "query": "...", "mode": "cascade", "sol_status": "up", "embed_backend": "solaria",
   "results": [
     {"envelope_id": "paperless:119", "source": "paperless", "dist": 0.34,
      "chunk_index": 2, "text": "...", "link": "https://paper.kapala.org/documents/119/details",
@@ -64,13 +64,39 @@ container, no node build step (plan §2 decision 4): `app/templates/index.html`
 - Stopka pokazuje `sol_status` dyskretnie (odświeżane z `/healthz` przy
   starcie strony i po każdym wyszukiwaniu).
 
-## Embed path (current step)
+## Embed path — active SOLARIA→PIHA fallback (Krok 2, plan §2 D2/§5)
 
-Calls Ollama on SOLARIA directly per request — no cache, no circuit breaker,
+Query embeddings go through `app/embed_router.py` (`EmbedRouter`, one instance
-no local-PIHA fallback yet (that state machine, plan §2 decision 2/§5, is a
+per process):
-separate later step). If SOLARIA is unreachable, `/search` returns **503**;
+
-`/healthz` still answers (`sol_status: "down"`), same tolerance pattern as
+1. SOLARIA's health verdict (`GET /api/tags`, 1.5 s timeout) is **cached for
-`llm-gateway`.
+   30 s** — no per-request probing.
+2. Verdict `up` → embed on `EMBED_PRIMARY_URL` (SOLARIA GPU, ~207 ms) under a
+   hard 3 s timeout.
+3. A failure **during a real embed** flips the verdict to `down` for one TTL
+   window and the **same request** is served from `EMBED_FALLBACK_URL`
+   (`ollama-piha`, CPU, `OLLAMA_KEEP_ALIVE=0` — slower, single seconds, but
+   alive) — the user never sees a SOLARIA error while a fallback exists.
+4. Verdict `down` → straight to the fallback until the TTL expires; when
+   SOLARIA wakes up, traffic returns to the GPU within ≤30 s.
+
+Every `/search` response and log line says which backend embedded the query
+(`embed_backend: "solaria"|"piha"`, log `backend=…` in `kb-query.embed`) —
+needed to debug result quality per backend. `sol_status` in the response is
+the router's world view; the UI footer renders `down` as
+„SOLARIA: offline (fallback embed)".
+
+**Model invariant, both halves:** at startup kb-query pins `EMBED_MODEL`
+(bge-m3) to `document_chunk.model`/`document_summary.embedding_model` (below);
+additionally each backend is verified once, at its first use, that its
+`/api/tags` actually lists `EMBED_MODEL`. A backend serving the wrong model is
+a **loud ERROR + 500** — never a silent distance computation across two
+different embedding spaces. Verification is lazy because SOLARIA may be asleep
+at boot and must not block startup.
+
+`/search` returns **503** only when BOTH backends are unreachable (or
+`EMBED_FALLBACK_URL` is unset — then the pre-fallback behaviour applies);
+`/healthz` still answers, same tolerance pattern as `llm-gateway`.
 
 ## Startup invariant (hard-fail)
 
@@ -86,13 +112,23 @@ that *wrote* the summary, e.g. `claude-haiku-4-5`, not the embedder).
 ## Configuration
 
 `.env` — **gitignored**, copy from `env.example`. Required: `LAN_BIND_IP`,
-`KB_DSN`. Optional: `OLLAMA_URL`, `EMBED_MODEL`, `SUMMARY_MODEL`.
+`KB_DSN`. On PIHA also set `EMBED_FALLBACK_URL=http://192.168.31.5:11434`
+(ollama-piha). Optional (defaults in parentheses): `EMBED_PRIMARY_URL`
+(`http://solaria:11434`), `EMBED_PRIMARY_NAME`/`EMBED_FALLBACK_NAME`
+(`solaria`/`piha`), `EMBED_HEALTH_TTL_S` (30), `EMBED_HEALTH_TIMEOUT_S` (1.5),
+`EMBED_PRIMARY_TIMEOUT_S` (3), `EMBED_MODEL` (`bge-m3`), `SUMMARY_MODEL`
+(`claude-haiku-4-5`). `OLLAMA_URL` was **renamed** to `EMBED_PRIMARY_URL` in
+Krok 2 — if an old `.env` sets `OLLAMA_URL`, it is ignored.
 
 ## Deploy (PIHA)
 
+0. Prerequisite: `ollama-piha` deployed and `bge-m3` pulled — see
+   `services/ollama-piha/README.md` (the pull is a **manual** deploy step).
 1. `git pull` on PIHA (`~/homelab-codex-ws`).
 2. `cp services/kb-query/env.example services/kb-query/.env` and fill in the
-   real `KB_DSN` password.
+   real `KB_DSN` password (the template already sets `EMBED_FALLBACK_URL`).
+   On an existing install: add `EMBED_FALLBACK_URL=http://192.168.31.5:11434`
+   to the existing `.env`.
 3. ```
    docker compose -f services/kb-query/docker-compose.yml \
      -f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build
@@ -101,6 +137,22 @@ that *wrote* the summary, e.g. `claude-haiku-4-5`, not the embedder).
    `curl "http://192.168.31.5:8230/search?q=test"` and open
    `http://192.168.31.5:8230/` in a browser.
 
+## Fallback verification (execution: operator, after deploy)
+
+- **Test A — SOLARIA online**: query via UI/`curl`; response has
+  `"embed_backend": "solaria"`, `docker logs kb-query` shows
+  `backend=solaria`, latency ~sub-second.
+- **Test B — SOLARIA offline**: either wait for the nightly power-off, or
+  simulate: set `EMBED_PRIMARY_URL=http://192.0.2.1:11434` (TEST-NET, always
+  unreachable) in `.env` and `docker compose … up -d` again. Query still
+  works; response has `"embed_backend": "piha"`, log shows `backend=piha`
+  plus a `circuit open for 30s` warning on the first hit; latency visibly
+  higher (CPU + cold model load each time, `OLLAMA_KEEP_ALIVE=0`). Revert
+  `.env` afterwards if simulated.
+- **Test C — SOLARIA returns**: after it is back up, within ≤30 s (one
+  health-cache TTL) responses show `"embed_backend": "solaria"` again, no
+  restart needed.
+
 ## Tests
 
 ```
@@ -150,5 +202,8 @@ existed.
 
 ## Out of scope for this step
 
-- Local-PIHA embed fallback / circuit breaker (plan §2 decision 2, §5).
 - OIDC login (see above) — separate session, needs `authlib` + Forgejo OAuth2 app.
+- Calibration verdict for the fallback (plan §5 steps 4–5: live PIHA
+  measurements → keep/tune/degrade decision) — the mechanism is built and
+  default-on when `EMBED_FALLBACK_URL` is set; the measurements are the
+  operator's post-deploy step (see `services/ollama-piha/README.md`).

services/kb-query/app/embed_router.py

@@ -0,0 +1,185 @@
+"""Embed-backend router -- module 5 phase 4, Krok 2 (docs/kb/modules/05-faza4-plan.md §2
+Decyzja 2, §5): the active SOLARIA->PIHA fallback state machine `/search` embeds through.
+One instance per process (`app.state.embed_router`), holding the single global `sol_status`
+cache the plan describes:
+
+| state           | behaviour                                                            |
+|-----------------|----------------------------------------------------------------------|
+| cache valid     | use remembered up/down, no probe                                     |
+| cache expired   | GET {primary}/api/tags, short timeout -> up/down, cached for the TTL |
+| up -> embed     | POST {primary}/api/embeddings under a hard timeout                   |
+| embed fails     | one-shot flip (plan step 3b): mark down for one TTL window and serve |
+|                 | THE SAME query from the fallback -- the user never sees a SOLARIA    |
+|                 | error unless there is no fallback configured                         |
+| down -> embed   | straight to the fallback, no primary attempt until the TTL expires   |
+
+Both legs call `kb_retrieval.embed.embed_chunk(..., model=embed_model)` with the same
+configured model (the plan's "twardy inwariant"). On top of the DB-side startup check
+(app/startup.py pins EMBED_MODEL to `document_chunk.model`/`document_summary.embedding_model`),
+each backend is additionally verified ONCE per process lifetime, lazily at its first use:
+its `/api/tags` must list `embed_model`. A backend that answers but lacks bge-m3 raises
+`ModelMismatchError` (ERROR log, 500 upstream) -- never a silent embed in a vector space
+that doesn't match the index. Verification is lazy rather than at startup because either
+backend may legitimately be offline when kb-query boots (SOLARIA sleeps ~16 h/day, plan
+§1) and must not block the service.
+
+Timeouts: probe 1.5 s (task spec "krótki, np. 1-2 s"; the plan's 500 ms proposal was for
+LAN-adjacent Tailscale -- 1.5 s absorbs LTE-grade jitter without hurting the 30 s cache
+economics), primary embed 3 s hard (plan step 3). The fallback embed gets NO hard timeout:
+Pi-5 CPU embed plus a cold model load (OLLAMA_KEEP_ALIVE=0 on ollama-piha) is legitimately
+slow (single seconds to low tens) and it is the last resort -- cutting it off would turn
+"slow but alive" back into "dead".
+"""
+from __future__ import annotations
+
+import asyncio
+import logging
+import time
+from dataclasses import dataclass, field
+
+import aiohttp
+
+from kb_retrieval.embed import check_ollama_health, embed_chunk
+
+logger = logging.getLogger("kb-query.embed")
+
+
+class EmbedBackendError(RuntimeError):
+    """No configured backend could embed the query (primary down and the fallback is
+    unconfigured or unreachable). Maps to 503 in the HTTP layer."""
+
+
+class ModelMismatchError(RuntimeError):
+    """A backend is reachable but its /api/tags does not list the configured embed model.
+    Config/ops error (wrong URL, model not pulled) -- deliberately loud, maps to 500."""
+
+
+@dataclass
+class _Backend:
+    name: str
+    url: str
+    model_verified: bool = field(default=False)
+
+
+class EmbedRouter:
+    def __init__(
+        self,
+        primary_url: str,
+        fallback_url: str | None = None,
+        *,
+        embed_model: str,
+        primary_name: str = "solaria",
+        fallback_name: str = "piha",
+        health_ttl_s: float = 30.0,
+        health_timeout_s: float = 1.5,
+        primary_embed_timeout_s: float = 3.0,
+        clock=time.monotonic,
+    ):
+        self.primary = _Backend(primary_name, primary_url.rstrip("/"))
+        self.fallback = _Backend(fallback_name, fallback_url.rstrip("/")) if fallback_url else None
+        self.embed_model = embed_model
+        self.health_ttl_s = health_ttl_s
+        self.health_timeout_s = health_timeout_s
+        self.primary_embed_timeout_s = primary_embed_timeout_s
+        self._clock = clock
+        self._sol_status: str | None = None  # "up" | "down" | None (never probed yet)
+        self._checked_at: float = 0.0
+
+    def _cache_valid(self) -> bool:
+        return self._sol_status is not None and (self._clock() - self._checked_at) < self.health_ttl_s
+
+    def _set_status(self, status: str) -> None:
+        self._sol_status = status
+        self._checked_at = self._clock()
+
+    async def primary_status(self, session: aiohttp.ClientSession) -> str:
+        """Cached primary liveness ("up"/"down"), probing /api/tags only when the TTL expired.
+        Shared by /search routing and /healthz so both report the same world view."""
+        if not self._cache_valid():
+            up = await check_ollama_health(session, self.primary.url, self.health_timeout_s)
+            self._set_status("up" if up else "down")
+        return self._sol_status
+
+    async def fallback_status(self, session: aiohttp.ClientSession) -> str:
+        """Live fallback liveness for /healthz: "up"/"down"/"unconfigured". Uncached -- the
+        probe is a cheap /api/tags (does not load the model on ollama-piha) and monitoring
+        calls this every 30 s anyway."""
+        if self.fallback is None:
+            return "unconfigured"
+        up = await check_ollama_health(session, self.fallback.url, self.health_timeout_s)
+        return "up" if up else "down"
+
+    async def _verify_model(self, session: aiohttp.ClientSession, backend: _Backend) -> None:
+        """First-use guard: `/api/tags` of `backend` must list `embed_model` (exact name or
+        same base, so a pulled "bge-m3:latest" satisfies EMBED_MODEL=bge-m3). Raises
+        ModelMismatchError when reachable-but-missing; connection errors propagate to the
+        caller's failover handling. Passed once -> never re-checked for this process."""
+        if backend.model_verified:
+            return
+        async with session.get(
+            f"{backend.url}/api/tags",
+            timeout=aiohttp.ClientTimeout(total=self.health_timeout_s),
+        ) as resp:
+            resp.raise_for_status()
+            data = await resp.json()
+        names = {m.get("name", "") for m in data.get("models", [])}
+        base = self.embed_model.split(":")[0]
+        if not any(n == self.embed_model or n.split(":")[0] == base for n in names):
+            raise ModelMismatchError(
+                f"embed backend {backend.name} ({backend.url}) does not serve "
+                f"model {self.embed_model!r} (available: {sorted(names) or 'none'}) -- "
+                f"refusing to embed in a vector space that doesn't match document_chunk"
+            )
+        backend.model_verified = True
+
+    async def embed(self, session: aiohttp.ClientSession, text: str) -> tuple[list[float], str]:
+        """Embed one query text, returning (embedding, backend_name). Implements the state
+        machine from the module docstring; raises EmbedBackendError (503) when no backend
+        can serve, ModelMismatchError (500) when the fallback serves the wrong model."""
+        if await self.primary_status(session) == "up":
+            try:
+                await self._verify_model(session, self.primary)
+                embedding, elapsed = await asyncio.wait_for(
+                    embed_chunk(session, self.primary.url, self.embed_model, text),
+                    timeout=self.primary_embed_timeout_s,
+                )
+                logger.info(
+                    "query embedded backend=%s url=%s elapsed_ms=%.0f",
+                    self.primary.name, self.primary.url, elapsed * 1000,
+                )
+                return embedding, self.primary.name
+            except ModelMismatchError as exc:
+                # Reachable but serving the wrong model set -- loud, then try the fallback
+                # (which is verified independently below, so this cannot cascade silently).
+                logger.error("primary embed backend rejected: %s", exc)
+                self._set_status("down")
+            except (aiohttp.ClientError, TimeoutError, ValueError) as exc:
+                # Plan step 3b: one-shot flip -- the SAME query continues to the fallback.
+                logger.warning(
+                    "primary embed failed backend=%s url=%s (%s: %s); circuit open for %.0fs",
+                    self.primary.name, self.primary.url, type(exc).__name__, exc, self.health_ttl_s,
+                )
+                self._set_status("down")
+
+        if self.fallback is None:
+            raise EmbedBackendError(
+                f"primary embed backend {self.primary.name} ({self.primary.url}) is down "
+                f"and no EMBED_FALLBACK_URL is configured"
+            )
+
+        try:
+            await self._verify_model(session, self.fallback)
+        except ModelMismatchError:
+            raise  # loud config error -- never silently embed in a mismatched space
+        except (aiohttp.ClientError, TimeoutError) as exc:
+            raise EmbedBackendError(
+                f"primary embed backend {self.primary.name} is down and fallback "
+                f"{self.fallback.name} ({self.fallback.url}) is unreachable: {exc}"
+            ) from exc
+
+        embedding, elapsed = await embed_chunk(session, self.fallback.url, self.embed_model, text)
+        logger.info(
+            "query embedded backend=%s url=%s elapsed_ms=%.0f (fallback; primary=%s down)",
+            self.fallback.name, self.fallback.url, elapsed * 1000, self.primary.name,
+        )
+        return embedding, self.fallback.name

services/kb-query/app/main.py

@@ -1,12 +1,15 @@
 """kb-query -- module 5 phase 4 (docs/kb/modules/05-faza4-plan.md §4): first user-facing HTTP
-entry point to the KB. Wraps `kb_retrieval.cascade_query`/`flat_query` (module 5 phase 3,
+entry point to the KB. Wraps `kb_retrieval` retrieval (module 5 phase 3, gated PASS --
-already gated PASS -- docs/sessions/2026-07-21.md) in FastAPI. This is a search API, not chat:
+docs/sessions/2026-07-21.md) in FastAPI. This is a search API, not chat: no answer synthesis,
-no answer synthesis, no LLM call over the results (that is phase 5, out of scope here).
+no LLM call over the results (that is phase 5, out of scope here).
 
-Embed path is deliberately simple for this step: calls Ollama on SOLARIA directly, no
+Embed path (Krok 2, plan §2 decision 2 / §5 -- the active fallback): queries embed through
-cache/circuit-breaker/local-PIHA-fallback (plan §2 decision 2, §5) -- that state machine is a
+`app.embed_router.EmbedRouter` -- SOLARIA's GPU Ollama (EMBED_PRIMARY_URL) when its cached
-later, separate step. A failed embed call (SOLARIA unreachable) surfaces as 503 to the caller
+~30 s health-check says up, the local CPU `ollama-piha` (EMBED_FALLBACK_URL) when SOLARIA
-rather than a bare 500.
+sleeps. A mid-query primary failure fails over the SAME request instead of surfacing an
+error. Only when BOTH backends are unavailable does `/search` return 503; a backend serving
+the wrong model (vs the bge-m3 space `document_chunk` is indexed in) is a loud 500, never a
+silent cross-space distance computation.
 
 `GET /` (Krok 4, plan §7) serves the search UI from this same FastAPI process -- one image, one
 container (plan §2 decision 4): a Jinja2 shell + a static vanilla-JS file, no node build step.
@@ -18,6 +21,7 @@ available explicitly starting here, but the default stays `cascade` until the qu
 """
 from __future__ import annotations
 
+import logging
 import os
 import pathlib
 from contextlib import asynccontextmanager
@@ -28,18 +32,30 @@ from fastapi.responses import HTMLResponse
 from fastapi.staticfiles import StaticFiles
 from fastapi.templating import Jinja2Templates
 
-from kb_retrieval.embed import check_ollama_health
-
 from app.db import create_pool
+from app.embed_router import EmbedBackendError, EmbedRouter, ModelMismatchError
 from app.search import run_search
 from app.startup import validate_embed_model
 
+# uvicorn only configures its own loggers; without this the router's
+# backend=solaria/piha lines (kb-query.embed) would never reach docker logs.
+logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s")
+
 BASE_DIR = pathlib.Path(__file__).resolve().parent
 KB_DSN = os.environ.get("KB_DSN")
-OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://solaria:11434")
+# EMBED_PRIMARY_URL replaced OLLAMA_URL in Krok 2 (aktywny fallback) -- one naming scheme
+# for both legs. Same default upstream as before: Ollama @ SOLARIA over Tailscale.
+EMBED_PRIMARY_URL = os.environ.get("EMBED_PRIMARY_URL", "http://solaria:11434")
+# Unset/empty -> no fallback leg: /search 503s when SOLARIA is down (pre-Krok-2 behaviour).
+EMBED_FALLBACK_URL = os.environ.get("EMBED_FALLBACK_URL") or None
+EMBED_PRIMARY_NAME = os.environ.get("EMBED_PRIMARY_NAME", "solaria")
+EMBED_FALLBACK_NAME = os.environ.get("EMBED_FALLBACK_NAME", "piha")
 EMBED_MODEL = os.environ.get("EMBED_MODEL", "bge-m3")
 SUMMARY_MODEL = os.environ.get("SUMMARY_MODEL", "claude-haiku-4-5")
-OLLAMA_HEALTH_TIMEOUT_S = 3.0
+# State-machine parameters (plan §2 D2 table; timeouts rationale in app/embed_router.py).
+EMBED_HEALTH_TTL_S = float(os.environ.get("EMBED_HEALTH_TTL_S", "30"))
+EMBED_HEALTH_TIMEOUT_S = float(os.environ.get("EMBED_HEALTH_TIMEOUT_S", "1.5"))
+EMBED_PRIMARY_TIMEOUT_S = float(os.environ.get("EMBED_PRIMARY_TIMEOUT_S", "3"))
 
 
 @asynccontextmanager
@@ -50,11 +66,23 @@ async def lifespan(app: FastAPI):
     pool = await create_pool(KB_DSN)
     async with pool.acquire() as conn:
         # Hard invariant (plan §2 decision 2): refuse to start rather than silently serve
-        # queries against a mismatched embedding space.
+        # queries against a mismatched embedding space. The per-backend half of the same
+        # invariant (does the Ollama actually serve EMBED_MODEL?) lives in EmbedRouter and
+        # runs lazily at each backend's first use -- a sleeping SOLARIA must not block boot.
         await validate_embed_model(conn, EMBED_MODEL)
 
     app.state.pool = pool
     app.state.http = aiohttp.ClientSession()
+    app.state.embed_router = EmbedRouter(
+        EMBED_PRIMARY_URL,
+        EMBED_FALLBACK_URL,
+        embed_model=EMBED_MODEL,
+        primary_name=EMBED_PRIMARY_NAME,
+        fallback_name=EMBED_FALLBACK_NAME,
+        health_ttl_s=EMBED_HEALTH_TTL_S,
+        health_timeout_s=EMBED_HEALTH_TIMEOUT_S,
+        primary_embed_timeout_s=EMBED_PRIMARY_TIMEOUT_S,
+    )
     try:
         yield
     finally:
@@ -74,8 +102,15 @@ async def index(request: Request):
 
 @app.get("/healthz")
 async def healthz() -> dict:
-    sol_up = await check_ollama_health(app.state.http, OLLAMA_URL, OLLAMA_HEALTH_TIMEOUT_S)
+    # sol_status goes through the router's 30 s cache -- /healthz and /search routing
-    return {"status": "ok", "sol_status": "up" if sol_up else "down"}
+    # deliberately share one world view (and monitoring no longer re-probes a sleeping
+    # SOLARIA on every scrape). fallback_status is a live cheap /api/tags probe.
+    router = app.state.embed_router
+    return {
+        "status": "ok",
+        "sol_status": await router.primary_status(app.state.http),
+        "fallback_status": await router.fallback_status(app.state.http),
+    }
 
 
 @app.get("/search")
@@ -86,7 +121,10 @@ async def search(
     try:
         async with app.state.pool.acquire() as conn:
             return await run_search(
-                conn, app.state.http, OLLAMA_URL, q, mode, EMBED_MODEL, SUMMARY_MODEL
+                conn, app.state.http, app.state.embed_router, q, mode, SUMMARY_MODEL
             )
-    except aiohttp.ClientError as exc:
+    except ModelMismatchError as exc:
+        # Config/ops error (backend up but wrong model set) -- 500, deliberately loud.
+        raise HTTPException(status_code=500, detail=str(exc)) from exc
+    except (EmbedBackendError, aiohttp.ClientError) as exc:
         raise HTTPException(status_code=503, detail=f"embed backend unavailable: {exc}") from exc

services/kb-query/app/search.py

@@ -1,6 +1,6 @@
 """`/search` core -- module 5 phase 4 (docs/kb/modules/05-faza4-plan.md §4). Kept decoupled
-from FastAPI so it can be unit-tested with fake `conn`/`session` objects, the same style as
+from FastAPI so it can be unit-tested with fake `conn`/`session`/`router` objects, the same
-`kb_retrieval`'s own tests, instead of needing a live DB/Ollama behind a TestClient.
+style as `kb_retrieval`'s own tests, instead of needing a live DB/Ollama behind a TestClient.
 
 Response shape (plan §4 exactly): `{"query", "mode", "sol_status", "results": [...]}`, each
 result carrying `dist` un-filtered -- the 0.45/0.55 colour thresholds (plan §7) are a frontend
@@ -8,43 +8,48 @@ concern (Krok 4, out of this step's scope), never applied server-side.
 
 `summary`/`summary_tags` (document_summary, haiku track) are an additive field added in Krok 4
 for the frontend's per-envelope result header (plan §7) -- `None`/`[]` when the envelope has no
-summary for `summary_model` yet. Purely additive: does not change any field already covered by
+summary for `summary_model` yet.
-the phase-4 gate's HTTP-equivalence check (plan §9).
+
+Krok 2 (aktywny fallback, plan §5): the query embedding no longer happens inside
+`kb_retrieval.cascade_query`/`flat_query`/`hybrid_query` -- it goes through
+`app.embed_router.EmbedRouter` (SOLARIA primary -> PIHA fallback state machine) and the
+resulting vector feeds the same `*_retrieve` functions those wrappers call. Two additive
+consequences for the response: `sol_status` now reports the router's real world view ("down"
+while serving from the fallback -- the frontend already renders that as "offline (fallback
+embed)"), and `embed_backend` names which backend actually embedded THIS query (task spec:
+needed to debug result quality per backend).
 """
 from __future__ import annotations
 
 import aiohttp
 import asyncpg
 
-from kb_retrieval.retrieval import cascade_query, flat_query, hybrid_query
+from kb_retrieval.embed import _vector_literal  # same private-import convention as kb_retrieval.retrieval
+from kb_retrieval.retrieval import cascade_retrieve, flat_retrieve, hybrid_retrieve
 
 from app.db import fetch_envelopes, fetch_summaries
+from app.embed_router import EmbedRouter
 from app.links import build_result
 
 
 async def run_search(
     conn: asyncpg.Connection,
     session: aiohttp.ClientSession,
-    ollama_url: str,
+    router: EmbedRouter,
     query_text: str,
     mode: str,
-    embed_model: str,
     summary_model: str,
 ) -> dict:
-    if mode == "flat":
+    embedding, backend = await router.embed(session, query_text)
-        retrieval = await flat_query(conn, session, ollama_url, query_text, embed_model=embed_model)
+    query_vector = _vector_literal(embedding)
-    elif mode == "hybrid":
+
-        retrieval = await hybrid_query(
+    if mode == "flat":
-            conn, session, ollama_url, query_text,
+        chunks = await flat_retrieve(conn, query_vector)
-            summary_model=summary_model, embed_model=embed_model,
+    elif mode == "hybrid":
-        )
+        chunks = (await hybrid_retrieve(conn, query_vector, summary_model))["chunks"]
-    else:
+    else:
-        retrieval = await cascade_query(
+        chunks = (await cascade_retrieve(conn, query_vector, summary_model))["chunks"]
-            conn, session, ollama_url, query_text,
-            summary_model=summary_model, embed_model=embed_model,
-        )
 
-    chunks = retrieval["chunks"]
     envelope_ids = sorted({c["envelope_id"] for c in chunks})
     envelopes = await fetch_envelopes(conn, envelope_ids)
     summaries = await fetch_summaries(conn, envelope_ids, summary_model)
@@ -60,6 +65,9 @@ async def run_search(
     return {
         "query": query_text,
         "mode": mode,
-        "sol_status": "up",  # reaching this point means the embed call above succeeded
+        # "up" iff the primary (SOLARIA) embedded this very query -- the fallback path
+        # implies the router just observed the primary down (probe or mid-embed failure).
+        "sol_status": "up" if backend == router.primary.name else "down",
+        "embed_backend": backend,
         "results": results,
     }

services/kb-query/docker-compose.yml

@@ -17,7 +17,17 @@ services:
       - "${LAN_BIND_IP}:8230:8080"
     environment:
       - KB_DSN=${KB_DSN}
-      - OLLAMA_URL=${OLLAMA_URL:-http://solaria:11434}
+      # Embed backends (Krok 2, aktywny fallback — app/embed_router.py): primary is
+      # Ollama @ SOLARIA (GPU, duty-cycled), fallback is ollama-piha on this host
+      # (CPU, OLLAMA_KEEP_ALIVE=0). Empty EMBED_FALLBACK_URL disables the fallback
+      # leg (pre-Krok-2 behaviour: /search 503s while SOLARIA sleeps).
+      - EMBED_PRIMARY_URL=${EMBED_PRIMARY_URL:-http://solaria:11434}
+      - EMBED_FALLBACK_URL=${EMBED_FALLBACK_URL:-}
+      - EMBED_PRIMARY_NAME=${EMBED_PRIMARY_NAME:-solaria}
+      - EMBED_FALLBACK_NAME=${EMBED_FALLBACK_NAME:-piha}
+      - EMBED_HEALTH_TTL_S=${EMBED_HEALTH_TTL_S:-30}
+      - EMBED_HEALTH_TIMEOUT_S=${EMBED_HEALTH_TIMEOUT_S:-1.5}
+      - EMBED_PRIMARY_TIMEOUT_S=${EMBED_PRIMARY_TIMEOUT_S:-3}
       - EMBED_MODEL=${EMBED_MODEL:-bge-m3}
       - SUMMARY_MODEL=${SUMMARY_MODEL:-claude-haiku-4-5}
     # python:3.12-slim has no curl/wget; same in-container check pattern as llm-gateway.

services/kb-query/env.example

@@ -11,9 +11,25 @@ LAN_BIND_IP=192.168.31.5
 # reasoning as paperless-worker@SOLARIA reaching paperless@PIHA over LAN.
 KB_DSN=postgresql://kb:CHANGE-ME@192.168.31.5:5433/kb
 
-# Ollama upstream (SOLARIA, over Tailscale MagicDNS — same trick as
+# Primary embed backend: Ollama @ SOLARIA (GPU, over Tailscale MagicDNS —
-# llm-gateway's OLLAMA_URL). Optional: defaults to this value if unset.
+# same trick as llm-gateway's OLLAMA_URL). Optional: defaults to this value.
-# OLLAMA_URL=http://solaria:11434
+# EMBED_PRIMARY_URL=http://solaria:11434
+
+# Fallback embed backend: ollama-piha on this host (services/ollama-piha/),
+# reached over PIHA's LAN interface (same pattern as KB_DSN above). SOLARIA is
+# powered off ~16 h/day — without this, /search is dead the whole time. Leave
+# unset/empty to disable the fallback leg entirely.
+EMBED_FALLBACK_URL=http://192.168.31.5:11434
+
+# Backend labels used in logs and the /search "embed_backend" response field.
+# Optional: defaults solaria / piha.
+# EMBED_PRIMARY_NAME=solaria
+# EMBED_FALLBACK_NAME=piha
+
+# Fallback state machine tuning (plan §2 D2). Optional; defaults shown.
+# EMBED_HEALTH_TTL_S=30        # how long an up/down verdict on SOLARIA is cached
+# EMBED_HEALTH_TIMEOUT_S=1.5   # /api/tags probe timeout
+# EMBED_PRIMARY_TIMEOUT_S=3    # hard timeout for a primary embed call
 
 # Embedding model kb-query enforces as a startup invariant (plan §2 decision
 # 2) against document_chunk.model / document_summary.embedding_model.

services/kb-query/service.yaml

@@ -5,10 +5,12 @@ service:
   exposure: private               # LAN/Tailscale only, npm@PIHA vhost (kb.kapala.org) is a later step
   dependencies:
     - kb-postgres
+    - ollama-piha                 # soft: local CPU embed fallback (Krok 2) -- kb-query starts without it
     - forgejo                     # OIDC identity provider, wired in a later step (plan §8); not yet enforced
-  # ollama@SOLARIA is an external, optional runtime dependency (embed calls), not a hard
+  # ollama@SOLARIA is an external, optional runtime dependency (primary embed backend), not
-  # dependency here -- SOLARIA may be offline; /search then fails per-request with 503,
+  # a hard dependency here -- SOLARIA sleeps ~16 h/day; the embed router (app/embed_router.py)
-  # /healthz still answers (same tolerance pattern as llm-gateway).
+  # then falls back to ollama-piha on this host. /search 503s only when BOTH legs are down;
+  # /healthz always answers (same tolerance pattern as llm-gateway).
   ports:
     - container: 8080
       host: 8230                  # LAN_BIND_IP only, never 0.0.0.0
@@ -28,6 +30,12 @@ service:
     env_vars:
       - LAN_BIND_IP                # required — compose port-bind interpolation
       - KB_DSN                     # required — asyncpg DSN for kb-postgres@PIHA
-      - OLLAMA_URL                 # optional — defaults to http://solaria:11434
+      - EMBED_PRIMARY_URL          # optional — primary embed backend, defaults to http://solaria:11434
-      - EMBED_MODEL                 # optional — defaults to bge-m3; startup invariant vs document_chunk/document_summary
+      - EMBED_FALLBACK_URL         # optional — local fallback (ollama-piha); empty disables the fallback leg
-      - SUMMARY_MODEL               # optional — defaults to claude-haiku-4-5; cascade_query's stage-1 model
+      - EMBED_PRIMARY_NAME         # optional — log/response label, defaults to solaria
+      - EMBED_FALLBACK_NAME        # optional — log/response label, defaults to piha
+      - EMBED_HEALTH_TTL_S         # optional — primary health verdict cache, defaults to 30
+      - EMBED_HEALTH_TIMEOUT_S     # optional — /api/tags probe timeout, defaults to 1.5
+      - EMBED_PRIMARY_TIMEOUT_S    # optional — hard timeout for a primary embed call, defaults to 3
+      - EMBED_MODEL                 # optional — defaults to bge-m3; startup invariant vs document_chunk/document_summary + per-backend /api/tags check
+      - SUMMARY_MODEL               # optional — defaults to claude-haiku-4-5; cascade stage-1 model

services/kb-query/tests/test_embed_router.py

@@ -0,0 +1,224 @@
+"""Unit tests for the SOLARIA->PIHA embed fallback state machine (app.embed_router) -- no
+real Ollama on either side. Fake backends are keyed by URL so one fake session can serve
+both legs; the clock is injected so the 30 s TTL is tested without sleeping."""
+from __future__ import annotations
+
+import pathlib
+import sys
+
+import aiohttp
+import pytest
+
+sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
+
+from app.embed_router import EmbedBackendError, EmbedRouter, ModelMismatchError  # noqa: E402
+
+PRIMARY = "http://solaria:11434"
+FALLBACK = "http://192.168.31.5:11434"
+
+
+class _FakeResponse:
+    def __init__(self, payload=None, exc=None, status=200):
+        self._payload = payload
+        self._exc = exc
+        self.status = status  # check_ollama_health reads resp.status directly
+
+    async def __aenter__(self):
+        if self._exc is not None:
+            raise self._exc
+        return self
+
+    async def __aexit__(self, *exc):
+        return False
+
+    def raise_for_status(self):
+        pass
+
+    async def json(self):
+        return self._payload
+
+
+class _FakeBackend:
+    """One fake Ollama: `models` drives /api/tags, `down=True` refuses every connection,
+    `embed_exc` makes /api/embeddings fail while /api/tags still answers (the plan's step-3b
+    scenario: probe says up, the real embed then dies mid-query)."""
+
+    def __init__(self, models=("bge-m3:latest",), down=False, embed_exc=None, vector_value=0.01):
+        self.models = list(models)
+        self.down = down
+        self.embed_exc = embed_exc
+        self.vector_value = vector_value
+        self.tags_calls = 0
+        self.embed_calls = 0
+
+
+class _FakeSession:
+    def __init__(self, backends: dict):
+        self._backends = backends
+
+    def _backend(self, url):
+        for base, backend in self._backends.items():
+            if url.startswith(base):
+                return backend
+        raise AssertionError(f"unexpected url: {url}")
+
+    def get(self, url, timeout=None):
+        backend = self._backend(url)
+        backend.tags_calls += 1
+        if backend.down:
+            return _FakeResponse(exc=aiohttp.ClientConnectionError("connection refused"))
+        return _FakeResponse({"models": [{"name": n} for n in backend.models]})
+
+    def post(self, url, json):
+        backend = self._backend(url)
+        backend.embed_calls += 1
+        if backend.down:
+            return _FakeResponse(exc=aiohttp.ClientConnectionError("connection refused"))
+        if backend.embed_exc is not None:
+            return _FakeResponse(exc=backend.embed_exc)
+        return _FakeResponse({"embedding": [backend.vector_value] * 1024})
+
+
+class _FakeClock:
+    def __init__(self):
+        self.now = 1000.0
+
+    def __call__(self):
+        return self.now
+
+
+def _router(fallback_url=FALLBACK, clock=None, **kwargs):
+    return EmbedRouter(
+        PRIMARY,
+        fallback_url,
+        embed_model="bge-m3",
+        clock=clock or _FakeClock(),
+        **kwargs,
+    )
+
+
+class TestPrimaryHealthy:
+    async def test_embeds_on_primary_and_reports_its_name(self):
+        solaria = _FakeBackend()
+        piha = _FakeBackend()
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        embedding, backend = await _router().embed(session, "q")
+        assert backend == "solaria"
+        assert len(embedding) == 1024
+        assert solaria.embed_calls == 1
+        assert piha.embed_calls == 0
+
+    async def test_health_verdict_is_cached_within_ttl(self):
+        solaria = _FakeBackend()
+        clock = _FakeClock()
+        router = _router(clock=clock)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: _FakeBackend()})
+        await router.embed(session, "q1")
+        probes_after_first = solaria.tags_calls  # probe + first-use model verification
+        clock.now += 10  # inside the 30 s TTL
+        await router.embed(session, "q2")
+        assert solaria.tags_calls == probes_after_first  # no new probe, model check done once
+
+    async def test_model_tag_with_latest_suffix_satisfies_bare_model_name(self):
+        solaria = _FakeBackend(models=["bge-m3:latest"])
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: _FakeBackend()})
+        _, backend = await _router().embed(session, "q")
+        assert backend == "solaria"
+
+
+class TestFailover:
+    async def test_primary_down_at_probe_falls_back_to_piha(self):
+        solaria = _FakeBackend(down=True)
+        piha = _FakeBackend(vector_value=0.02)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        embedding, backend = await _router().embed(session, "q")
+        assert backend == "piha"
+        assert embedding[0] == 0.02
+        assert solaria.embed_calls == 0
+
+    async def test_mid_embed_failure_serves_same_query_from_fallback(self):
+        # Plan §2 D2 step 3b: probe said up, the real embed then dies -- the SAME request
+        # must come back from the fallback, never as an error to the user.
+        solaria = _FakeBackend(embed_exc=aiohttp.ClientConnectionError("died mid-embed"))
+        piha = _FakeBackend(vector_value=0.02)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        router = _router()
+        embedding, backend = await router.embed(session, "q")
+        assert backend == "piha"
+        assert embedding[0] == 0.02
+        assert solaria.embed_calls == 1
+
+    async def test_down_verdict_is_cached_and_skips_primary_until_ttl_expires(self):
+        solaria = _FakeBackend(down=True)
+        piha = _FakeBackend()
+        clock = _FakeClock()
+        router = _router(clock=clock)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        await router.embed(session, "q1")
+        probes = solaria.tags_calls
+        clock.now += 10  # still inside TTL -> no re-probe, straight to fallback
+        _, backend = await router.embed(session, "q2")
+        assert backend == "piha"
+        assert solaria.tags_calls == probes
+
+    async def test_traffic_returns_to_primary_after_ttl_expiry(self):
+        # Test C from the task spec: SOLARIA comes back -> within one TTL window the
+        # router re-probes and routes to the GPU again.
+        solaria = _FakeBackend(down=True)
+        piha = _FakeBackend()
+        clock = _FakeClock()
+        router = _router(clock=clock)
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        _, backend = await router.embed(session, "q1")
+        assert backend == "piha"
+        solaria.down = False  # SOLARIA wakes up
+        clock.now += 31  # TTL (30 s) expired
+        _, backend = await router.embed(session, "q2")
+        assert backend == "solaria"
+
+
+class TestNoUsableBackend:
+    async def test_primary_down_without_fallback_raises_backend_error(self):
+        solaria = _FakeBackend(down=True)
+        session = _FakeSession({PRIMARY: solaria})
+        with pytest.raises(EmbedBackendError, match="no EMBED_FALLBACK_URL"):
+            await _router(fallback_url=None).embed(session, "q")
+
+    async def test_both_backends_down_raises_backend_error(self):
+        session = _FakeSession({PRIMARY: _FakeBackend(down=True), FALLBACK: _FakeBackend(down=True)})
+        with pytest.raises(EmbedBackendError, match="unreachable"):
+            await _router().embed(session, "q")
+
+
+class TestModelInvariant:
+    async def test_fallback_without_bge_m3_is_a_loud_error_not_a_silent_embed(self):
+        solaria = _FakeBackend(down=True)
+        piha = _FakeBackend(models=["llama3:8b"])
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        with pytest.raises(ModelMismatchError, match="bge-m3"):
+            await _router().embed(session, "q")
+        assert piha.embed_calls == 0  # never embedded in the wrong vector space
+
+    async def test_primary_without_bge_m3_fails_over_to_verified_fallback(self):
+        solaria = _FakeBackend(models=["llama3:8b"])
+        piha = _FakeBackend()
+        session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
+        _, backend = await _router().embed(session, "q")
+        assert backend == "piha"
+        assert solaria.embed_calls == 0
+
+
+class TestStatusReporting:
+    async def test_primary_status_up_and_fallback_status_up(self):
+        session = _FakeSession({PRIMARY: _FakeBackend(), FALLBACK: _FakeBackend()})
+        router = _router()
+        assert await router.primary_status(session) == "up"
+        assert await router.fallback_status(session) == "up"
+
+    async def test_fallback_status_unconfigured_without_fallback_url(self):
+        session = _FakeSession({PRIMARY: _FakeBackend()})
+        assert await _router(fallback_url=None).fallback_status(session) == "unconfigured"
+
+    async def test_primary_status_down_when_probe_fails(self):
+        session = _FakeSession({PRIMARY: _FakeBackend(down=True), FALLBACK: _FakeBackend()})
+        assert await _router().primary_status(session) == "down"

services/kb-query/tests/test_search.py

@@ -75,30 +75,23 @@ class _FakeConn:
         raise AssertionError(f"unexpected query: {query}")
 
 
-class _FakeEmbedResponse:
-    def __init__(self, payload):
-        self._payload = payload
-
-    async def __aenter__(self):
-        return self
-
-    async def __aexit__(self, *exc):
-        return False
-
-    def raise_for_status(self):
-        pass
-
-    async def json(self):
-        return self._payload
-
-
 class _FakeSession:
-    def __init__(self):
+    """run_search no longer talks HTTP itself -- embedding goes through the router
-        self.post_calls: list[dict] = []
+    (below), so the session is just passed through untouched."""
 
-    def post(self, url, json):
+
-        self.post_calls.append({"url": url, "json": json})
+class _FakeRouter:
-        return _FakeEmbedResponse({"embedding": [0.01] * 1024})
+    """Stands in for app.embed_router.EmbedRouter: returns a fixed embedding and the name
+    of the backend that 'served' it, mirroring EmbedRouter.embed's contract."""
+
+    def __init__(self, backend="solaria", primary_name="solaria"):
+        self._backend = backend
+        self.primary = type("B", (), {"name": primary_name})()
+        self.embed_calls: list[str] = []
+
+    async def embed(self, session, text):
+        self.embed_calls.append(text)
+        return [0.01] * 1024, self._backend
 
 
 class TestRunSearchHappyPath:
@@ -110,11 +103,12 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
         )
         assert result["query"] == "polisa PZU"
         assert result["mode"] == "cascade"
         assert result["sol_status"] == "up"
+        assert result["embed_backend"] == "solaria"
         assert len(result["results"]) == 1
         hit = result["results"][0]
         assert hit["envelope_id"] == "paperless:119"
@@ -131,7 +125,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "q", "flat", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "q", "flat", "claude-haiku-4-5"
         )
         assert result["mode"] == "flat"
         assert len(result["results"]) == 1
@@ -145,7 +139,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
         )
         hit = result["results"][0]
         assert hit["summary"] == "Polisa OC 2024"
@@ -159,7 +153,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
         )
         hit = result["results"][0]
         assert hit["summary"] is None
@@ -180,7 +174,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "q", "hybrid", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "q", "hybrid", "claude-haiku-4-5"
         )
         assert result["mode"] == "hybrid"
         envelope_ids = [r["envelope_id"] for r in result["results"]]
@@ -201,7 +195,7 @@ class TestRunSearchHappyPath:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "q", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "q", "cascade", "claude-haiku-4-5"
         )
         hit = result["results"][0]
         assert hit["source"] == "gmail"
@@ -210,6 +204,24 @@ class TestRunSearchHappyPath:
         assert hit["mail_ui_url"] is None
 
 
+class TestRunSearchEmbedBackendMarking:
+    async def test_fallback_embed_marks_backend_piha_and_sol_status_down(self):
+        # Task spec: the response must say WHICH backend embedded the query (quality
+        # debugging), and sol_status must reflect the router's world view -- the frontend
+        # renders "down" as "offline (fallback embed)".
+        conn = _FakeConn(
+            summaries=[("paperless:119", 0.1)],
+            chunks_by_envelope={"paperless:119": [(2, "hit text", 0.34)]},
+            envelopes={"paperless:119": {"source": "paperless", "entities": []}},
+        )
+        result = await run_search(
+            conn, _FakeSession(), _FakeRouter(backend="piha"), "q", "cascade", "claude-haiku-4-5"
+        )
+        assert result["embed_backend"] == "piha"
+        assert result["sol_status"] == "down"
+        assert len(result["results"]) == 1
+
+
 class TestRunSearchNoGoodResults:
     async def test_results_above_no_answer_threshold_are_still_returned_unfiltered(self):
         # Plan §7: the 0.55 "no answer" colour threshold is a frontend concern -- the API
@@ -222,7 +234,7 @@ class TestRunSearchNoGoodResults:
         )
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "unrelated query", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "unrelated query", "cascade", "claude-haiku-4-5"
         )
         assert len(result["results"]) == 1
         assert result["results"][0]["dist"] == 0.62
@@ -231,6 +243,6 @@ class TestRunSearchNoGoodResults:
         conn = _FakeConn(summaries=[], chunks_by_envelope={}, envelopes={})
         session = _FakeSession()
         result = await run_search(
-            conn, session, "http://fake-ollama", "nothing matches", "cascade", "bge-m3", "claude-haiku-4-5"
+            conn, session, _FakeRouter(), "nothing matches", "cascade", "claude-haiku-4-5"
         )
         assert result["results"] == []

services/ollama-piha/README.md

@@ -0,0 +1,76 @@
+# ollama-piha
+
+Local CPU Ollama on **PIHA**, serving exactly one purpose: the **fallback embed
+backend** for `kb-query` while SOLARIA (the GPU node, ~16 h/day powered off)
+sleeps. Model: `bge-m3` — the **same** model as SOLARIA's Ollama, because query
+embeddings must live in the same vector space as the pgvector index
+(`document_chunk.embedding VECTOR(1024)`); a different/smaller model is not an
+option (module 5 phase 4 plan §2 decision 2).
+
+Expected latency: bge-m3 embeds in ~207 ms on SOLARIA's GPU vs ~790 ms on x86
+CPU; on the Pi 5 expect single seconds per query (plus model load, since the
+model is never resident — see below). Slower but alive beats fast but dead.
+
+## Design constraints
+
+- **`OLLAMA_KEEP_ALIVE=0`** (pinned in compose): PIHA is the RAM-bound 8 GB box
+  shared with Home Assistant. The model is unloaded immediately after every
+  call — a transient ~1.5–2 GB spike per embed, ~100 MB idle daemon, never a
+  resident cost.
+- **`mem_limit: 2560m`** (host override, `hosts/piha/runtime/ollama-piha/`):
+  hard cgroup ceiling, plan §2 D2 starting value. The cgroup OOM killer
+  restarts this container instead of the host OOM killer picking a victim
+  (which could be Home Assistant). Confirm/trim after live calibration.
+- **Bind**: `127.0.0.1` + `LAN_BIND_IP` (192.168.31.5) only — kb-query calls it
+  over the host LAN interface (same pattern as kb-query → kb-postgres:5433).
+  Never `0.0.0.0`, never a Tailscale bind, no public ingress.
+- **Storage**: Docker named volume `ollama_piha_models` (NVMe data-root), not a
+  bind mount — the ollama image runs as in-container root and would break
+  PIHA's uid pattern (host oskar=1004, containers uid 1000, setgid group pi)
+  if it wrote to a shared bind directory.
+
+## Deploy (PIHA, master, after merge)
+
+```bash
+cd ~/homelab-codex-ws && git pull
+cp services/ollama-piha/env.example services/ollama-piha/.env   # LAN_BIND_IP
+docker compose -f services/ollama-piha/docker-compose.yml \
+  -f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
+  --env-file services/ollama-piha/.env up -d
+```
+
+**Then pull the model — this does NOT happen automatically:**
+
+```bash
+docker exec ollama-piha ollama pull bge-m3
+```
+
+Verify:
+
+```bash
+services/ollama-piha/healthcheck.sh          # checks container + API + bge-m3 present
+time curl -s http://127.0.0.1:11434/api/embeddings \
+  -d '{"model":"bge-m3","prompt":"test kalibracyjny"}' | head -c 80
+```
+
+(`deploy-node.sh` on PIHA also picks this service up from
+`hosts/piha/services.yaml` once `.env` exists — the `ollama pull bge-m3` step
+stays manual either way.)
+
+## Calibration (plan §5 step 4 — gate, not formality)
+
+Before trusting the fallback under load, on live PIHA at a normal (not
+night-quiet) hour: run a few embeds as above while watching
+`docker stats ollama-piha`, note peak RAM and wall time. Verdict per plan §5
+step 5: keep as default fallback / tune `mem_limit` / fall back to explicit
+503 degradation.
+
+## Relation to kb-query
+
+kb-query's router (`services/kb-query/app/embed_router.py`) health-checks
+SOLARIA with a ~30 s cache and only sends embeds here while SOLARIA is down.
+kb-query verifies at first use that this backend actually serves `bge-m3`
+(`/api/tags`) and refuses to embed against a mismatched model. Configuration:
+`EMBED_FALLBACK_URL=http://192.168.31.5:11434` in `services/kb-query/.env`.
+See `services/kb-query/README.md` for the fallback verification plan (tests
+A/B/C).

services/ollama-piha/docker-compose.yml

@@ -0,0 +1,33 @@
+services:
+  ollama-piha:
+    # Multi-arch image with a published linux/arm64 manifest — runs natively on
+    # the Pi 5, no emulation (same mechanism as pgvector/pgvector:pg16 on PIHA).
+    image: ollama/ollama:latest
+    container_name: ollama-piha
+    restart: unless-stopped
+    environment:
+      # PIHA is the RAM-bound 8 GB box shared with Home Assistant. bge-m3 must be
+      # unloaded from RAM immediately after every call — a short spike per embed
+      # (idle daemon ~100 MB), never a resident ~2 GB cost. Fallback traffic is
+      # rare by design: kb-query only embeds here while SOLARIA sleeps.
+      - OLLAMA_KEEP_ALIVE=0
+    ports:
+      # Loopback: healthcheck.sh + manual curl on the node itself.
+      # LAN IP: kb-query runs in a separate compose project/network and reaches
+      # this backend over the host's LAN interface — same reasoning as
+      # kb-query -> kb-postgres (:5433). No 0.0.0.0, no Tailscale bind: nothing
+      # off-node ever embeds here (SOLARIA-side callers use SOLARIA's own Ollama).
+      # Requires .env (from env.example) next to this file at deploy.
+      - "127.0.0.1:11434:11434"
+      - "${LAN_BIND_IP}:11434:11434"
+    volumes:
+      # Named volume, NOT a bind mount: the ollama image runs as in-container
+      # root and would fill a bind dir with root-owned files, breaking PIHA's
+      # uid pattern (host oskar=1004, containers uid 1000 + setgid group pi).
+      # Docker-managed volumes live under the data-root on the NVMe — same
+      # convention as kb_postgres_data.
+      - ollama_piha_models:/root/.ollama
+
+volumes:
+  ollama_piha_models:
+    name: ollama_piha_models

services/ollama-piha/env.example

@@ -0,0 +1,7 @@
+# ollama-piha — copy to .env (gitignored) next to docker-compose.yml; docker
+# compose picks it up automatically. Same convention as services/kb-query.
+
+# LAN IP of PIHA. The published port (11434) binds ONLY to 127.0.0.1 and this
+# interface — never 0.0.0.0. kb-query's EMBED_FALLBACK_URL points at
+# http://<LAN_BIND_IP>:11434. Verify after host rebuilds: ip -4 addr.
+LAN_BIND_IP=192.168.31.5

services/ollama-piha/healthcheck.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+# Healthcheck for ollama-piha (local CPU embed fallback for kb-query)
+
+# Container must be running
+if ! docker ps --filter "name=ollama-piha" --filter "status=running" | grep -qw "ollama-piha"; then
+    echo "[FAIL] ollama-piha container is not running"
+    exit 1
+fi
+
+# API must answer on loopback
+TAGS=$(curl -sf http://127.0.0.1:11434/api/tags)
+if [ $? -ne 0 ]; then
+    echo "[FAIL] ollama-piha API is not responding on 127.0.0.1:11434"
+    exit 1
+fi
+
+# bge-m3 must be pulled — without it the API answers /api/tags but every kb-query
+# fallback embed fails the model check (deploy step 3 in README.md).
+if ! echo "$TAGS" | grep -q '"bge-m3'; then
+    echo "[FAIL] model bge-m3 missing (run: docker exec ollama-piha ollama pull bge-m3)"
+    exit 1
+fi
+
+echo "[OK] ollama-piha is healthy (bge-m3 present)"
+exit 0

services/ollama-piha/service.yaml

@@ -0,0 +1,26 @@
+service:
+  name: ollama-piha
+  owner_node: piha
+  role: embed-fallback            # module 5 phase 4 (plan §2 D2/§5): local CPU bge-m3 embed backend for kb-query while SOLARIA sleeps
+  exposure: private               # 127.0.0.1 + LAN bind only — never 0.0.0.0, never public ingress, no Tailscale bind
+  dependencies: []
+  ports:
+    - container: 11434
+      host: 11434                 # bound to 127.0.0.1 and LAN_BIND_IP only
+      protocol: tcp
+  healthcheck:
+    type: http
+    endpoint: http://127.0.0.1:11434/api/tags
+    interval: 1m
+    timeout: 10s
+    retries: 3
+  restart_policy: unless-stopped
+  persistence:
+    paths: []                     # models live in the Docker named volume ollama_piha_models (NVMe data-root)
+  runtime:
+    config_files:
+      - .env                      # LAN_BIND_IP (gitignored, from env.example)
+    env_vars:
+      - LAN_BIND_IP               # required — compose port-bind interpolation
+    # OLLAMA_KEEP_ALIVE is pinned to 0 in docker-compose.yml (not operator-tunable):
+    # the model must never stay resident on the RAM-bound, HA-sharing Pi 5.