feat(kb): aktywny fallback embeddingów SOLARIA→PIHA dla kb-query (faza 4 Krok 2)
Ostatni krok fazy 4 KB (plan §2 Decyzja 2, §5): kb-query przestaje być martwe
przez ~16 h/dobę, gdy SOLARIA (GPU) śpi — zapytania embeduje wtedy lokalna
Ollama CPU na PIHA (wolniej: ~790 ms+ vs ~207 ms na GPU, ale działa).
Nowy serwis services/ollama-piha (GitOps, owner_node: piha):
- ollama/ollama:latest (arm64 natywnie), OLLAMA_KEEP_ALIVE=0 — model zwalnia
RAM natychmiast po każdym wywołaniu (spike, nie rezydent; PIHA dzieli 8 GB z HA)
- bind wyłącznie 127.0.0.1 + LAN_BIND_IP (192.168.31.5), nigdy 0.0.0.0/Tailscale
- named volume ollama_piha_models (NVMe data-root) zamiast bind-mounta — obraz
biega jako root w kontenerze i bind łamałby wzorzec uid PIHA (oskar=1004,
kontenery uid 1000, setgid pi)
- override hosts/piha/runtime/ollama-piha: mem_limit 2560m (wartość startowa
z planu, do potwierdzenia kalibracją na żywo), świadomie bez mem_reservation
- pull bge-m3 to jawny, ręczny krok deployu (README) — obraz nie ma modeli
kb-query — maszyna stanów fallbacku (app/embed_router.py):
- health-check SOLARII (GET /api/tags, timeout 1.5 s) z cache 30 s — zero
sondowania per request; po powrocie SOLARII ruch wraca na GPU w ≤30 s
- primary up → embed na SOLARII z twardym timeoutem 3 s; błąd W TRAKCIE
zapytania = jednorazowe przełączenie (krok 3b planu): status down na 30 s
i TO SAMO zapytanie leci na fallback — user nie widzi błędu SOLARII
- primary down → embed prosto na ollama-piha (bez twardego timeoutu: CPU +
zimny load modelu to legalnie pojedyncze sekundy)
- 503 tylko gdy oba backendy padłe (lub fallback nieskonfigurowany)
- inwariant modelu, druga połowa: każdy backend weryfikowany raz, leniwie przy
pierwszym użyciu, że /api/tags zawiera EMBED_MODEL (bge-m3 — ta sama wartość
co startowy check przeciw document_chunk.model/document_summary.embedding_model);
niezgodność = ERROR log + 500, nigdy ciche liczenie dystansów między
różnymi przestrzeniami embeddingów; leniwie, bo śpiąca SOLARIA nie może
blokować startu serwisu
- odpowiedź /search: nowe pole embed_backend ("solaria"|"piha") + sol_status
wg realnego świata routera (UI już renderuje down jako "offline (fallback
embed)"); log INFO backend=... elapsed_ms=... per zapytanie
- /healthz: sol_status przez cache routera (spójny widok z routingiem) +
fallback_status (żywa, tania sonda /api/tags)
Konfiguracja spójnie przez env (compose + env.example + service.yaml + README):
EMBED_PRIMARY_URL (zastępuje OLLAMA_URL), EMBED_FALLBACK_URL (pusty = brak
fallbacku, zachowanie sprzed kroku 2), EMBED_{PRIMARY,FALLBACK}_NAME,
EMBED_HEALTH_TTL_S/EMBED_HEALTH_TIMEOUT_S/EMBED_PRIMARY_TIMEOUT_S.
Testy: 39 pass (14 nowych w test_embed_router.py: cache TTL, failover w trakcie
zapytania, powrót po TTL, oba padłe, mismatch modelu na primary i fallbacku,
tag "bge-m3:latest" vs "bge-m3"); docker build + smoke (importy + uvicorn do
guardu KB_DSN) OK; compose config OK dla obu stacków.
Deploy (Oskar, na PIHA z mastera po merge):
cd ~/homelab-codex-ws && git pull
# 1. ollama-piha
cp services/ollama-piha/env.example services/ollama-piha/.env
docker compose -f services/ollama-piha/docker-compose.yml \
-f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
--env-file services/ollama-piha/.env up -d
docker exec ollama-piha ollama pull bge-m3 # ręczny krok, obowiązkowy
services/ollama-piha/healthcheck.sh
# 2. kb-query (dopisać fallback do istniejącego .env)
echo 'EMBED_FALLBACK_URL=http://192.168.31.5:11434' >> services/kb-query/.env
docker compose -f services/kb-query/docker-compose.yml \
-f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build
services/kb-query/healthcheck.sh
# (deploy-node.sh też podniesie oba serwisy z hosts/piha/services.yaml,
# ale pull bge-m3 i .env pozostają ręczne)
Weryfikacja: testy A/B/C w services/kb-query/README.md (backend=solaria przy
SOLARII online; backend=piha przy symulacji offline; powrót na GPU w ≤30 s).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
oskar
parent
e02c925ffb
commit
e7625cd322
18
hosts/piha/runtime/ollama-piha/docker-compose.override.yml
Normal file
18
hosts/piha/runtime/ollama-piha/docker-compose.override.yml
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
# PIHA-specific overrides for ollama-piha (KB module 5, phase 4 — embed fallback).
|
||||
#
|
||||
# RESOURCE CONTEXT: PIHA is the RAM-bound 8 GB box shared with Home Assistant.
|
||||
# bge-m3 is ~1.2 GB on disk, estimated 1.5–2 GB resident during an embed burst
|
||||
# (plan §2 decision 2). OLLAMA_KEEP_ALIVE=0 (base compose) makes that a short
|
||||
# per-call spike, not a resident cost — but the spike lands exactly when other
|
||||
# SOLARIA-fallback consumers (e.g. paperless fallback OCR) may also be awake,
|
||||
# so it gets a hard ceiling.
|
||||
services:
|
||||
ollama-piha:
|
||||
# Plan §2 D2 starting value — the cgroup OOM killer restarts this container
|
||||
# on breach instead of the host OOM killer picking a victim (which could be
|
||||
# Home Assistant). Confirm or trim after live calibration (plan §5 step 4:
|
||||
# a few embeds + docker stats at a normal-load hour).
|
||||
mem_limit: 2560m
|
||||
# Deliberately no mem_reservation: the working set is a transient spike and
|
||||
# the idle daemon is ~100 MB — soft-reserving gigabytes would permanently
|
||||
# take them from HA for a backend that mostly sleeps.
|
||||
|
|
@ -105,14 +105,37 @@ services:
|
|||
# data is in Docker named volume kb_postgres_data — must land on the NVMe
|
||||
# (Docker data-root on /home); see hosts/piha/runtime/kb-postgres override.
|
||||
|
||||
ollama-piha:
|
||||
role: embed-fallback # module 5 phase 4 (plan §2 D2/§5): local CPU bge-m3 for kb-query while SOLARIA sleeps
|
||||
deployment_model: docker-compose
|
||||
exposure: private # 127.0.0.1 + LAN bind (LAN_BIND_IP) only; no public/Tailscale exposure
|
||||
offline_required: false
|
||||
depends_on:
|
||||
local: []
|
||||
external: []
|
||||
ports:
|
||||
- name: http
|
||||
container_port: 11434
|
||||
host_port: 11434
|
||||
protocol: tcp
|
||||
runtime:
|
||||
# .env (LAN_BIND_IP) lives alongside the compose file; models live in the
|
||||
# Docker named volume ollama_piha_models (NVMe data-root, like kb-postgres).
|
||||
# Deploy note: `docker exec ollama-piha ollama pull bge-m3` is a manual
|
||||
# post-deploy step — the image ships no models.
|
||||
config_path: services/ollama-piha
|
||||
|
||||
kb-query:
|
||||
role: kb-search-api # module 5 phase 4: FastAPI wrapper over kb_retrieval cascade/flat query
|
||||
deployment_model: docker-compose
|
||||
exposure: private # LAN bind (LAN_BIND_IP); npm@PIHA vhost + OIDC is a later step
|
||||
offline_required: false
|
||||
depends_on:
|
||||
local: [kb-postgres]
|
||||
external: [ollama] # SOLARIA may be offline -> /search 502/503, /healthz stays ok
|
||||
# ollama-piha is a soft local dependency: the embed fallback while SOLARIA
|
||||
# sleeps — kb-query starts and serves /healthz without it, /search degrades
|
||||
# to 503 only when BOTH embed backends are unreachable.
|
||||
local: [kb-postgres, ollama-piha]
|
||||
external: [ollama] # primary embed backend @ SOLARIA; may be offline -> fallback to ollama-piha
|
||||
ports:
|
||||
- name: http
|
||||
container_port: 8080
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ that's phase 5.
|
|||
|---|---|---|
|
||||
| `/` | GET | Search UI (Jinja2 shell + `/static/app.js`, no login yet — plan §8 OIDC is a later step) |
|
||||
| `/static/*` | GET | UI assets (`app.js`, `style.css`) |
|
||||
| `/healthz` | GET | `{"status": "ok", "sol_status": "up"\|"down"}` — `sol_status` is a live probe of Ollama@SOLARIA, no auth required (monitoring must reach it) |
|
||||
| `/healthz` | GET | `{"status": "ok", "sol_status": "up"\|"down", "fallback_status": "up"\|"down"\|"unconfigured"}` — `sol_status` comes from the embed router's ~30 s-cached SOLARIA probe (same world view `/search` routes by), `fallback_status` is a live cheap probe of ollama-piha; no auth required (monitoring must reach it) |
|
||||
| `/search?q=<text>&mode=cascade\|flat` | GET | `query_text -> embed -> cascade_query/flat_query -> results`, `mode` defaults to `cascade` |
|
||||
|
||||
`/search` response shape (module 5 phase 4 plan §4, `summary`/`summary_tags`
|
||||
|
|
@ -21,7 +21,7 @@ change any field the plan §4 shape already defined):
|
|||
|
||||
```json
|
||||
{
|
||||
"query": "...", "mode": "cascade", "sol_status": "up",
|
||||
"query": "...", "mode": "cascade", "sol_status": "up", "embed_backend": "solaria",
|
||||
"results": [
|
||||
{"envelope_id": "paperless:119", "source": "paperless", "dist": 0.34,
|
||||
"chunk_index": 2, "text": "...", "link": "https://paper.kapala.org/documents/119/details",
|
||||
|
|
@ -64,13 +64,39 @@ container, no node build step (plan §2 decision 4): `app/templates/index.html`
|
|||
- Stopka pokazuje `sol_status` dyskretnie (odświeżane z `/healthz` przy
|
||||
starcie strony i po każdym wyszukiwaniu).
|
||||
|
||||
## Embed path (current step)
|
||||
## Embed path — active SOLARIA→PIHA fallback (Krok 2, plan §2 D2/§5)
|
||||
|
||||
Calls Ollama on SOLARIA directly per request — no cache, no circuit breaker,
|
||||
no local-PIHA fallback yet (that state machine, plan §2 decision 2/§5, is a
|
||||
separate later step). If SOLARIA is unreachable, `/search` returns **503**;
|
||||
`/healthz` still answers (`sol_status: "down"`), same tolerance pattern as
|
||||
`llm-gateway`.
|
||||
Query embeddings go through `app/embed_router.py` (`EmbedRouter`, one instance
|
||||
per process):
|
||||
|
||||
1. SOLARIA's health verdict (`GET /api/tags`, 1.5 s timeout) is **cached for
|
||||
30 s** — no per-request probing.
|
||||
2. Verdict `up` → embed on `EMBED_PRIMARY_URL` (SOLARIA GPU, ~207 ms) under a
|
||||
hard 3 s timeout.
|
||||
3. A failure **during a real embed** flips the verdict to `down` for one TTL
|
||||
window and the **same request** is served from `EMBED_FALLBACK_URL`
|
||||
(`ollama-piha`, CPU, `OLLAMA_KEEP_ALIVE=0` — slower, single seconds, but
|
||||
alive) — the user never sees a SOLARIA error while a fallback exists.
|
||||
4. Verdict `down` → straight to the fallback until the TTL expires; when
|
||||
SOLARIA wakes up, traffic returns to the GPU within ≤30 s.
|
||||
|
||||
Every `/search` response and log line says which backend embedded the query
|
||||
(`embed_backend: "solaria"|"piha"`, log `backend=…` in `kb-query.embed`) —
|
||||
needed to debug result quality per backend. `sol_status` in the response is
|
||||
the router's world view; the UI footer renders `down` as
|
||||
„SOLARIA: offline (fallback embed)".
|
||||
|
||||
**Model invariant, both halves:** at startup kb-query pins `EMBED_MODEL`
|
||||
(bge-m3) to `document_chunk.model`/`document_summary.embedding_model` (below);
|
||||
additionally each backend is verified once, at its first use, that its
|
||||
`/api/tags` actually lists `EMBED_MODEL`. A backend serving the wrong model is
|
||||
a **loud ERROR + 500** — never a silent distance computation across two
|
||||
different embedding spaces. Verification is lazy because SOLARIA may be asleep
|
||||
at boot and must not block startup.
|
||||
|
||||
`/search` returns **503** only when BOTH backends are unreachable (or
|
||||
`EMBED_FALLBACK_URL` is unset — then the pre-fallback behaviour applies);
|
||||
`/healthz` still answers, same tolerance pattern as `llm-gateway`.
|
||||
|
||||
## Startup invariant (hard-fail)
|
||||
|
||||
|
|
@ -86,13 +112,23 @@ that *wrote* the summary, e.g. `claude-haiku-4-5`, not the embedder).
|
|||
## Configuration
|
||||
|
||||
`.env` — **gitignored**, copy from `env.example`. Required: `LAN_BIND_IP`,
|
||||
`KB_DSN`. Optional: `OLLAMA_URL`, `EMBED_MODEL`, `SUMMARY_MODEL`.
|
||||
`KB_DSN`. On PIHA also set `EMBED_FALLBACK_URL=http://192.168.31.5:11434`
|
||||
(ollama-piha). Optional (defaults in parentheses): `EMBED_PRIMARY_URL`
|
||||
(`http://solaria:11434`), `EMBED_PRIMARY_NAME`/`EMBED_FALLBACK_NAME`
|
||||
(`solaria`/`piha`), `EMBED_HEALTH_TTL_S` (30), `EMBED_HEALTH_TIMEOUT_S` (1.5),
|
||||
`EMBED_PRIMARY_TIMEOUT_S` (3), `EMBED_MODEL` (`bge-m3`), `SUMMARY_MODEL`
|
||||
(`claude-haiku-4-5`). `OLLAMA_URL` was **renamed** to `EMBED_PRIMARY_URL` in
|
||||
Krok 2 — if an old `.env` sets `OLLAMA_URL`, it is ignored.
|
||||
|
||||
## Deploy (PIHA)
|
||||
|
||||
0. Prerequisite: `ollama-piha` deployed and `bge-m3` pulled — see
|
||||
`services/ollama-piha/README.md` (the pull is a **manual** deploy step).
|
||||
1. `git pull` on PIHA (`~/homelab-codex-ws`).
|
||||
2. `cp services/kb-query/env.example services/kb-query/.env` and fill in the
|
||||
real `KB_DSN` password.
|
||||
real `KB_DSN` password (the template already sets `EMBED_FALLBACK_URL`).
|
||||
On an existing install: add `EMBED_FALLBACK_URL=http://192.168.31.5:11434`
|
||||
to the existing `.env`.
|
||||
3. ```
|
||||
docker compose -f services/kb-query/docker-compose.yml \
|
||||
-f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build
|
||||
|
|
@ -101,6 +137,22 @@ that *wrote* the summary, e.g. `claude-haiku-4-5`, not the embedder).
|
|||
`curl "http://192.168.31.5:8230/search?q=test"` and open
|
||||
`http://192.168.31.5:8230/` in a browser.
|
||||
|
||||
## Fallback verification (execution: operator, after deploy)
|
||||
|
||||
- **Test A — SOLARIA online**: query via UI/`curl`; response has
|
||||
`"embed_backend": "solaria"`, `docker logs kb-query` shows
|
||||
`backend=solaria`, latency ~sub-second.
|
||||
- **Test B — SOLARIA offline**: either wait for the nightly power-off, or
|
||||
simulate: set `EMBED_PRIMARY_URL=http://192.0.2.1:11434` (TEST-NET, always
|
||||
unreachable) in `.env` and `docker compose … up -d` again. Query still
|
||||
works; response has `"embed_backend": "piha"`, log shows `backend=piha`
|
||||
plus a `circuit open for 30s` warning on the first hit; latency visibly
|
||||
higher (CPU + cold model load each time, `OLLAMA_KEEP_ALIVE=0`). Revert
|
||||
`.env` afterwards if simulated.
|
||||
- **Test C — SOLARIA returns**: after it is back up, within ≤30 s (one
|
||||
health-cache TTL) responses show `"embed_backend": "solaria"` again, no
|
||||
restart needed.
|
||||
|
||||
## Tests
|
||||
|
||||
```
|
||||
|
|
@ -150,5 +202,8 @@ existed.
|
|||
|
||||
## Out of scope for this step
|
||||
|
||||
- Local-PIHA embed fallback / circuit breaker (plan §2 decision 2, §5).
|
||||
- OIDC login (see above) — separate session, needs `authlib` + Forgejo OAuth2 app.
|
||||
- Calibration verdict for the fallback (plan §5 steps 4–5: live PIHA
|
||||
measurements → keep/tune/degrade decision) — the mechanism is built and
|
||||
default-on when `EMBED_FALLBACK_URL` is set; the measurements are the
|
||||
operator's post-deploy step (see `services/ollama-piha/README.md`).
|
||||
|
|
|
|||
185
services/kb-query/app/embed_router.py
Normal file
185
services/kb-query/app/embed_router.py
Normal file
|
|
@ -0,0 +1,185 @@
|
|||
"""Embed-backend router -- module 5 phase 4, Krok 2 (docs/kb/modules/05-faza4-plan.md §2
|
||||
Decyzja 2, §5): the active SOLARIA->PIHA fallback state machine `/search` embeds through.
|
||||
One instance per process (`app.state.embed_router`), holding the single global `sol_status`
|
||||
cache the plan describes:
|
||||
|
||||
| state | behaviour |
|
||||
|-----------------|----------------------------------------------------------------------|
|
||||
| cache valid | use remembered up/down, no probe |
|
||||
| cache expired | GET {primary}/api/tags, short timeout -> up/down, cached for the TTL |
|
||||
| up -> embed | POST {primary}/api/embeddings under a hard timeout |
|
||||
| embed fails | one-shot flip (plan step 3b): mark down for one TTL window and serve |
|
||||
| | THE SAME query from the fallback -- the user never sees a SOLARIA |
|
||||
| | error unless there is no fallback configured |
|
||||
| down -> embed | straight to the fallback, no primary attempt until the TTL expires |
|
||||
|
||||
Both legs call `kb_retrieval.embed.embed_chunk(..., model=embed_model)` with the same
|
||||
configured model (the plan's "twardy inwariant"). On top of the DB-side startup check
|
||||
(app/startup.py pins EMBED_MODEL to `document_chunk.model`/`document_summary.embedding_model`),
|
||||
each backend is additionally verified ONCE per process lifetime, lazily at its first use:
|
||||
its `/api/tags` must list `embed_model`. A backend that answers but lacks bge-m3 raises
|
||||
`ModelMismatchError` (ERROR log, 500 upstream) -- never a silent embed in a vector space
|
||||
that doesn't match the index. Verification is lazy rather than at startup because either
|
||||
backend may legitimately be offline when kb-query boots (SOLARIA sleeps ~16 h/day, plan
|
||||
§1) and must not block the service.
|
||||
|
||||
Timeouts: probe 1.5 s (task spec "krótki, np. 1-2 s"; the plan's 500 ms proposal was for
|
||||
LAN-adjacent Tailscale -- 1.5 s absorbs LTE-grade jitter without hurting the 30 s cache
|
||||
economics), primary embed 3 s hard (plan step 3). The fallback embed gets NO hard timeout:
|
||||
Pi-5 CPU embed plus a cold model load (OLLAMA_KEEP_ALIVE=0 on ollama-piha) is legitimately
|
||||
slow (single seconds to low tens) and it is the last resort -- cutting it off would turn
|
||||
"slow but alive" back into "dead".
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import aiohttp
|
||||
|
||||
from kb_retrieval.embed import check_ollama_health, embed_chunk
|
||||
|
||||
logger = logging.getLogger("kb-query.embed")
|
||||
|
||||
|
||||
class EmbedBackendError(RuntimeError):
|
||||
"""No configured backend could embed the query (primary down and the fallback is
|
||||
unconfigured or unreachable). Maps to 503 in the HTTP layer."""
|
||||
|
||||
|
||||
class ModelMismatchError(RuntimeError):
|
||||
"""A backend is reachable but its /api/tags does not list the configured embed model.
|
||||
Config/ops error (wrong URL, model not pulled) -- deliberately loud, maps to 500."""
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Backend:
|
||||
name: str
|
||||
url: str
|
||||
model_verified: bool = field(default=False)
|
||||
|
||||
|
||||
class EmbedRouter:
|
||||
def __init__(
|
||||
self,
|
||||
primary_url: str,
|
||||
fallback_url: str | None = None,
|
||||
*,
|
||||
embed_model: str,
|
||||
primary_name: str = "solaria",
|
||||
fallback_name: str = "piha",
|
||||
health_ttl_s: float = 30.0,
|
||||
health_timeout_s: float = 1.5,
|
||||
primary_embed_timeout_s: float = 3.0,
|
||||
clock=time.monotonic,
|
||||
):
|
||||
self.primary = _Backend(primary_name, primary_url.rstrip("/"))
|
||||
self.fallback = _Backend(fallback_name, fallback_url.rstrip("/")) if fallback_url else None
|
||||
self.embed_model = embed_model
|
||||
self.health_ttl_s = health_ttl_s
|
||||
self.health_timeout_s = health_timeout_s
|
||||
self.primary_embed_timeout_s = primary_embed_timeout_s
|
||||
self._clock = clock
|
||||
self._sol_status: str | None = None # "up" | "down" | None (never probed yet)
|
||||
self._checked_at: float = 0.0
|
||||
|
||||
def _cache_valid(self) -> bool:
|
||||
return self._sol_status is not None and (self._clock() - self._checked_at) < self.health_ttl_s
|
||||
|
||||
def _set_status(self, status: str) -> None:
|
||||
self._sol_status = status
|
||||
self._checked_at = self._clock()
|
||||
|
||||
async def primary_status(self, session: aiohttp.ClientSession) -> str:
|
||||
"""Cached primary liveness ("up"/"down"), probing /api/tags only when the TTL expired.
|
||||
Shared by /search routing and /healthz so both report the same world view."""
|
||||
if not self._cache_valid():
|
||||
up = await check_ollama_health(session, self.primary.url, self.health_timeout_s)
|
||||
self._set_status("up" if up else "down")
|
||||
return self._sol_status
|
||||
|
||||
async def fallback_status(self, session: aiohttp.ClientSession) -> str:
|
||||
"""Live fallback liveness for /healthz: "up"/"down"/"unconfigured". Uncached -- the
|
||||
probe is a cheap /api/tags (does not load the model on ollama-piha) and monitoring
|
||||
calls this every 30 s anyway."""
|
||||
if self.fallback is None:
|
||||
return "unconfigured"
|
||||
up = await check_ollama_health(session, self.fallback.url, self.health_timeout_s)
|
||||
return "up" if up else "down"
|
||||
|
||||
async def _verify_model(self, session: aiohttp.ClientSession, backend: _Backend) -> None:
|
||||
"""First-use guard: `/api/tags` of `backend` must list `embed_model` (exact name or
|
||||
same base, so a pulled "bge-m3:latest" satisfies EMBED_MODEL=bge-m3). Raises
|
||||
ModelMismatchError when reachable-but-missing; connection errors propagate to the
|
||||
caller's failover handling. Passed once -> never re-checked for this process."""
|
||||
if backend.model_verified:
|
||||
return
|
||||
async with session.get(
|
||||
f"{backend.url}/api/tags",
|
||||
timeout=aiohttp.ClientTimeout(total=self.health_timeout_s),
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
data = await resp.json()
|
||||
names = {m.get("name", "") for m in data.get("models", [])}
|
||||
base = self.embed_model.split(":")[0]
|
||||
if not any(n == self.embed_model or n.split(":")[0] == base for n in names):
|
||||
raise ModelMismatchError(
|
||||
f"embed backend {backend.name} ({backend.url}) does not serve "
|
||||
f"model {self.embed_model!r} (available: {sorted(names) or 'none'}) -- "
|
||||
f"refusing to embed in a vector space that doesn't match document_chunk"
|
||||
)
|
||||
backend.model_verified = True
|
||||
|
||||
async def embed(self, session: aiohttp.ClientSession, text: str) -> tuple[list[float], str]:
|
||||
"""Embed one query text, returning (embedding, backend_name). Implements the state
|
||||
machine from the module docstring; raises EmbedBackendError (503) when no backend
|
||||
can serve, ModelMismatchError (500) when the fallback serves the wrong model."""
|
||||
if await self.primary_status(session) == "up":
|
||||
try:
|
||||
await self._verify_model(session, self.primary)
|
||||
embedding, elapsed = await asyncio.wait_for(
|
||||
embed_chunk(session, self.primary.url, self.embed_model, text),
|
||||
timeout=self.primary_embed_timeout_s,
|
||||
)
|
||||
logger.info(
|
||||
"query embedded backend=%s url=%s elapsed_ms=%.0f",
|
||||
self.primary.name, self.primary.url, elapsed * 1000,
|
||||
)
|
||||
return embedding, self.primary.name
|
||||
except ModelMismatchError as exc:
|
||||
# Reachable but serving the wrong model set -- loud, then try the fallback
|
||||
# (which is verified independently below, so this cannot cascade silently).
|
||||
logger.error("primary embed backend rejected: %s", exc)
|
||||
self._set_status("down")
|
||||
except (aiohttp.ClientError, TimeoutError, ValueError) as exc:
|
||||
# Plan step 3b: one-shot flip -- the SAME query continues to the fallback.
|
||||
logger.warning(
|
||||
"primary embed failed backend=%s url=%s (%s: %s); circuit open for %.0fs",
|
||||
self.primary.name, self.primary.url, type(exc).__name__, exc, self.health_ttl_s,
|
||||
)
|
||||
self._set_status("down")
|
||||
|
||||
if self.fallback is None:
|
||||
raise EmbedBackendError(
|
||||
f"primary embed backend {self.primary.name} ({self.primary.url}) is down "
|
||||
f"and no EMBED_FALLBACK_URL is configured"
|
||||
)
|
||||
|
||||
try:
|
||||
await self._verify_model(session, self.fallback)
|
||||
except ModelMismatchError:
|
||||
raise # loud config error -- never silently embed in a mismatched space
|
||||
except (aiohttp.ClientError, TimeoutError) as exc:
|
||||
raise EmbedBackendError(
|
||||
f"primary embed backend {self.primary.name} is down and fallback "
|
||||
f"{self.fallback.name} ({self.fallback.url}) is unreachable: {exc}"
|
||||
) from exc
|
||||
|
||||
embedding, elapsed = await embed_chunk(session, self.fallback.url, self.embed_model, text)
|
||||
logger.info(
|
||||
"query embedded backend=%s url=%s elapsed_ms=%.0f (fallback; primary=%s down)",
|
||||
self.fallback.name, self.fallback.url, elapsed * 1000, self.primary.name,
|
||||
)
|
||||
return embedding, self.fallback.name
|
||||
|
|
@ -1,12 +1,15 @@
|
|||
"""kb-query -- module 5 phase 4 (docs/kb/modules/05-faza4-plan.md §4): first user-facing HTTP
|
||||
entry point to the KB. Wraps `kb_retrieval.cascade_query`/`flat_query` (module 5 phase 3,
|
||||
already gated PASS -- docs/sessions/2026-07-21.md) in FastAPI. This is a search API, not chat:
|
||||
no answer synthesis, no LLM call over the results (that is phase 5, out of scope here).
|
||||
entry point to the KB. Wraps `kb_retrieval` retrieval (module 5 phase 3, gated PASS --
|
||||
docs/sessions/2026-07-21.md) in FastAPI. This is a search API, not chat: no answer synthesis,
|
||||
no LLM call over the results (that is phase 5, out of scope here).
|
||||
|
||||
Embed path is deliberately simple for this step: calls Ollama on SOLARIA directly, no
|
||||
cache/circuit-breaker/local-PIHA-fallback (plan §2 decision 2, §5) -- that state machine is a
|
||||
later, separate step. A failed embed call (SOLARIA unreachable) surfaces as 503 to the caller
|
||||
rather than a bare 500.
|
||||
Embed path (Krok 2, plan §2 decision 2 / §5 -- the active fallback): queries embed through
|
||||
`app.embed_router.EmbedRouter` -- SOLARIA's GPU Ollama (EMBED_PRIMARY_URL) when its cached
|
||||
~30 s health-check says up, the local CPU `ollama-piha` (EMBED_FALLBACK_URL) when SOLARIA
|
||||
sleeps. A mid-query primary failure fails over the SAME request instead of surfacing an
|
||||
error. Only when BOTH backends are unavailable does `/search` return 503; a backend serving
|
||||
the wrong model (vs the bge-m3 space `document_chunk` is indexed in) is a loud 500, never a
|
||||
silent cross-space distance computation.
|
||||
|
||||
`GET /` (Krok 4, plan §7) serves the search UI from this same FastAPI process -- one image, one
|
||||
container (plan §2 decision 4): a Jinja2 shell + a static vanilla-JS file, no node build step.
|
||||
|
|
@ -18,6 +21,7 @@ available explicitly starting here, but the default stays `cascade` until the qu
|
|||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import pathlib
|
||||
from contextlib import asynccontextmanager
|
||||
|
|
@ -28,18 +32,30 @@ from fastapi.responses import HTMLResponse
|
|||
from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.templating import Jinja2Templates
|
||||
|
||||
from kb_retrieval.embed import check_ollama_health
|
||||
|
||||
from app.db import create_pool
|
||||
from app.embed_router import EmbedBackendError, EmbedRouter, ModelMismatchError
|
||||
from app.search import run_search
|
||||
from app.startup import validate_embed_model
|
||||
|
||||
# uvicorn only configures its own loggers; without this the router's
|
||||
# backend=solaria/piha lines (kb-query.embed) would never reach docker logs.
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s")
|
||||
|
||||
BASE_DIR = pathlib.Path(__file__).resolve().parent
|
||||
KB_DSN = os.environ.get("KB_DSN")
|
||||
OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://solaria:11434")
|
||||
# EMBED_PRIMARY_URL replaced OLLAMA_URL in Krok 2 (aktywny fallback) -- one naming scheme
|
||||
# for both legs. Same default upstream as before: Ollama @ SOLARIA over Tailscale.
|
||||
EMBED_PRIMARY_URL = os.environ.get("EMBED_PRIMARY_URL", "http://solaria:11434")
|
||||
# Unset/empty -> no fallback leg: /search 503s when SOLARIA is down (pre-Krok-2 behaviour).
|
||||
EMBED_FALLBACK_URL = os.environ.get("EMBED_FALLBACK_URL") or None
|
||||
EMBED_PRIMARY_NAME = os.environ.get("EMBED_PRIMARY_NAME", "solaria")
|
||||
EMBED_FALLBACK_NAME = os.environ.get("EMBED_FALLBACK_NAME", "piha")
|
||||
EMBED_MODEL = os.environ.get("EMBED_MODEL", "bge-m3")
|
||||
SUMMARY_MODEL = os.environ.get("SUMMARY_MODEL", "claude-haiku-4-5")
|
||||
OLLAMA_HEALTH_TIMEOUT_S = 3.0
|
||||
# State-machine parameters (plan §2 D2 table; timeouts rationale in app/embed_router.py).
|
||||
EMBED_HEALTH_TTL_S = float(os.environ.get("EMBED_HEALTH_TTL_S", "30"))
|
||||
EMBED_HEALTH_TIMEOUT_S = float(os.environ.get("EMBED_HEALTH_TIMEOUT_S", "1.5"))
|
||||
EMBED_PRIMARY_TIMEOUT_S = float(os.environ.get("EMBED_PRIMARY_TIMEOUT_S", "3"))
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
|
|
@ -50,11 +66,23 @@ async def lifespan(app: FastAPI):
|
|||
pool = await create_pool(KB_DSN)
|
||||
async with pool.acquire() as conn:
|
||||
# Hard invariant (plan §2 decision 2): refuse to start rather than silently serve
|
||||
# queries against a mismatched embedding space.
|
||||
# queries against a mismatched embedding space. The per-backend half of the same
|
||||
# invariant (does the Ollama actually serve EMBED_MODEL?) lives in EmbedRouter and
|
||||
# runs lazily at each backend's first use -- a sleeping SOLARIA must not block boot.
|
||||
await validate_embed_model(conn, EMBED_MODEL)
|
||||
|
||||
app.state.pool = pool
|
||||
app.state.http = aiohttp.ClientSession()
|
||||
app.state.embed_router = EmbedRouter(
|
||||
EMBED_PRIMARY_URL,
|
||||
EMBED_FALLBACK_URL,
|
||||
embed_model=EMBED_MODEL,
|
||||
primary_name=EMBED_PRIMARY_NAME,
|
||||
fallback_name=EMBED_FALLBACK_NAME,
|
||||
health_ttl_s=EMBED_HEALTH_TTL_S,
|
||||
health_timeout_s=EMBED_HEALTH_TIMEOUT_S,
|
||||
primary_embed_timeout_s=EMBED_PRIMARY_TIMEOUT_S,
|
||||
)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
|
|
@ -74,8 +102,15 @@ async def index(request: Request):
|
|||
|
||||
@app.get("/healthz")
|
||||
async def healthz() -> dict:
|
||||
sol_up = await check_ollama_health(app.state.http, OLLAMA_URL, OLLAMA_HEALTH_TIMEOUT_S)
|
||||
return {"status": "ok", "sol_status": "up" if sol_up else "down"}
|
||||
# sol_status goes through the router's 30 s cache -- /healthz and /search routing
|
||||
# deliberately share one world view (and monitoring no longer re-probes a sleeping
|
||||
# SOLARIA on every scrape). fallback_status is a live cheap /api/tags probe.
|
||||
router = app.state.embed_router
|
||||
return {
|
||||
"status": "ok",
|
||||
"sol_status": await router.primary_status(app.state.http),
|
||||
"fallback_status": await router.fallback_status(app.state.http),
|
||||
}
|
||||
|
||||
|
||||
@app.get("/search")
|
||||
|
|
@ -86,7 +121,10 @@ async def search(
|
|||
try:
|
||||
async with app.state.pool.acquire() as conn:
|
||||
return await run_search(
|
||||
conn, app.state.http, OLLAMA_URL, q, mode, EMBED_MODEL, SUMMARY_MODEL
|
||||
conn, app.state.http, app.state.embed_router, q, mode, SUMMARY_MODEL
|
||||
)
|
||||
except aiohttp.ClientError as exc:
|
||||
except ModelMismatchError as exc:
|
||||
# Config/ops error (backend up but wrong model set) -- 500, deliberately loud.
|
||||
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||
except (EmbedBackendError, aiohttp.ClientError) as exc:
|
||||
raise HTTPException(status_code=503, detail=f"embed backend unavailable: {exc}") from exc
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
"""`/search` core -- module 5 phase 4 (docs/kb/modules/05-faza4-plan.md §4). Kept decoupled
|
||||
from FastAPI so it can be unit-tested with fake `conn`/`session` objects, the same style as
|
||||
`kb_retrieval`'s own tests, instead of needing a live DB/Ollama behind a TestClient.
|
||||
from FastAPI so it can be unit-tested with fake `conn`/`session`/`router` objects, the same
|
||||
style as `kb_retrieval`'s own tests, instead of needing a live DB/Ollama behind a TestClient.
|
||||
|
||||
Response shape (plan §4 exactly): `{"query", "mode", "sol_status", "results": [...]}`, each
|
||||
result carrying `dist` un-filtered -- the 0.45/0.55 colour thresholds (plan §7) are a frontend
|
||||
|
|
@ -8,43 +8,48 @@ concern (Krok 4, out of this step's scope), never applied server-side.
|
|||
|
||||
`summary`/`summary_tags` (document_summary, haiku track) are an additive field added in Krok 4
|
||||
for the frontend's per-envelope result header (plan §7) -- `None`/`[]` when the envelope has no
|
||||
summary for `summary_model` yet. Purely additive: does not change any field already covered by
|
||||
the phase-4 gate's HTTP-equivalence check (plan §9).
|
||||
summary for `summary_model` yet.
|
||||
|
||||
Krok 2 (aktywny fallback, plan §5): the query embedding no longer happens inside
|
||||
`kb_retrieval.cascade_query`/`flat_query`/`hybrid_query` -- it goes through
|
||||
`app.embed_router.EmbedRouter` (SOLARIA primary -> PIHA fallback state machine) and the
|
||||
resulting vector feeds the same `*_retrieve` functions those wrappers call. Two additive
|
||||
consequences for the response: `sol_status` now reports the router's real world view ("down"
|
||||
while serving from the fallback -- the frontend already renders that as "offline (fallback
|
||||
embed)"), and `embed_backend` names which backend actually embedded THIS query (task spec:
|
||||
needed to debug result quality per backend).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import aiohttp
|
||||
import asyncpg
|
||||
|
||||
from kb_retrieval.retrieval import cascade_query, flat_query, hybrid_query
|
||||
from kb_retrieval.embed import _vector_literal # same private-import convention as kb_retrieval.retrieval
|
||||
from kb_retrieval.retrieval import cascade_retrieve, flat_retrieve, hybrid_retrieve
|
||||
|
||||
from app.db import fetch_envelopes, fetch_summaries
|
||||
from app.embed_router import EmbedRouter
|
||||
from app.links import build_result
|
||||
|
||||
|
||||
async def run_search(
|
||||
conn: asyncpg.Connection,
|
||||
session: aiohttp.ClientSession,
|
||||
ollama_url: str,
|
||||
router: EmbedRouter,
|
||||
query_text: str,
|
||||
mode: str,
|
||||
embed_model: str,
|
||||
summary_model: str,
|
||||
) -> dict:
|
||||
if mode == "flat":
|
||||
retrieval = await flat_query(conn, session, ollama_url, query_text, embed_model=embed_model)
|
||||
elif mode == "hybrid":
|
||||
retrieval = await hybrid_query(
|
||||
conn, session, ollama_url, query_text,
|
||||
summary_model=summary_model, embed_model=embed_model,
|
||||
)
|
||||
else:
|
||||
retrieval = await cascade_query(
|
||||
conn, session, ollama_url, query_text,
|
||||
summary_model=summary_model, embed_model=embed_model,
|
||||
)
|
||||
embedding, backend = await router.embed(session, query_text)
|
||||
query_vector = _vector_literal(embedding)
|
||||
|
||||
if mode == "flat":
|
||||
chunks = await flat_retrieve(conn, query_vector)
|
||||
elif mode == "hybrid":
|
||||
chunks = (await hybrid_retrieve(conn, query_vector, summary_model))["chunks"]
|
||||
else:
|
||||
chunks = (await cascade_retrieve(conn, query_vector, summary_model))["chunks"]
|
||||
|
||||
chunks = retrieval["chunks"]
|
||||
envelope_ids = sorted({c["envelope_id"] for c in chunks})
|
||||
envelopes = await fetch_envelopes(conn, envelope_ids)
|
||||
summaries = await fetch_summaries(conn, envelope_ids, summary_model)
|
||||
|
|
@ -60,6 +65,9 @@ async def run_search(
|
|||
return {
|
||||
"query": query_text,
|
||||
"mode": mode,
|
||||
"sol_status": "up", # reaching this point means the embed call above succeeded
|
||||
# "up" iff the primary (SOLARIA) embedded this very query -- the fallback path
|
||||
# implies the router just observed the primary down (probe or mid-embed failure).
|
||||
"sol_status": "up" if backend == router.primary.name else "down",
|
||||
"embed_backend": backend,
|
||||
"results": results,
|
||||
}
|
||||
|
|
|
|||
|
|
@ -17,7 +17,17 @@ services:
|
|||
- "${LAN_BIND_IP}:8230:8080"
|
||||
environment:
|
||||
- KB_DSN=${KB_DSN}
|
||||
- OLLAMA_URL=${OLLAMA_URL:-http://solaria:11434}
|
||||
# Embed backends (Krok 2, aktywny fallback — app/embed_router.py): primary is
|
||||
# Ollama @ SOLARIA (GPU, duty-cycled), fallback is ollama-piha on this host
|
||||
# (CPU, OLLAMA_KEEP_ALIVE=0). Empty EMBED_FALLBACK_URL disables the fallback
|
||||
# leg (pre-Krok-2 behaviour: /search 503s while SOLARIA sleeps).
|
||||
- EMBED_PRIMARY_URL=${EMBED_PRIMARY_URL:-http://solaria:11434}
|
||||
- EMBED_FALLBACK_URL=${EMBED_FALLBACK_URL:-}
|
||||
- EMBED_PRIMARY_NAME=${EMBED_PRIMARY_NAME:-solaria}
|
||||
- EMBED_FALLBACK_NAME=${EMBED_FALLBACK_NAME:-piha}
|
||||
- EMBED_HEALTH_TTL_S=${EMBED_HEALTH_TTL_S:-30}
|
||||
- EMBED_HEALTH_TIMEOUT_S=${EMBED_HEALTH_TIMEOUT_S:-1.5}
|
||||
- EMBED_PRIMARY_TIMEOUT_S=${EMBED_PRIMARY_TIMEOUT_S:-3}
|
||||
- EMBED_MODEL=${EMBED_MODEL:-bge-m3}
|
||||
- SUMMARY_MODEL=${SUMMARY_MODEL:-claude-haiku-4-5}
|
||||
# python:3.12-slim has no curl/wget; same in-container check pattern as llm-gateway.
|
||||
|
|
|
|||
|
|
@ -11,9 +11,25 @@ LAN_BIND_IP=192.168.31.5
|
|||
# reasoning as paperless-worker@SOLARIA reaching paperless@PIHA over LAN.
|
||||
KB_DSN=postgresql://kb:CHANGE-ME@192.168.31.5:5433/kb
|
||||
|
||||
# Ollama upstream (SOLARIA, over Tailscale MagicDNS — same trick as
|
||||
# llm-gateway's OLLAMA_URL). Optional: defaults to this value if unset.
|
||||
# OLLAMA_URL=http://solaria:11434
|
||||
# Primary embed backend: Ollama @ SOLARIA (GPU, over Tailscale MagicDNS —
|
||||
# same trick as llm-gateway's OLLAMA_URL). Optional: defaults to this value.
|
||||
# EMBED_PRIMARY_URL=http://solaria:11434
|
||||
|
||||
# Fallback embed backend: ollama-piha on this host (services/ollama-piha/),
|
||||
# reached over PIHA's LAN interface (same pattern as KB_DSN above). SOLARIA is
|
||||
# powered off ~16 h/day — without this, /search is dead the whole time. Leave
|
||||
# unset/empty to disable the fallback leg entirely.
|
||||
EMBED_FALLBACK_URL=http://192.168.31.5:11434
|
||||
|
||||
# Backend labels used in logs and the /search "embed_backend" response field.
|
||||
# Optional: defaults solaria / piha.
|
||||
# EMBED_PRIMARY_NAME=solaria
|
||||
# EMBED_FALLBACK_NAME=piha
|
||||
|
||||
# Fallback state machine tuning (plan §2 D2). Optional; defaults shown.
|
||||
# EMBED_HEALTH_TTL_S=30 # how long an up/down verdict on SOLARIA is cached
|
||||
# EMBED_HEALTH_TIMEOUT_S=1.5 # /api/tags probe timeout
|
||||
# EMBED_PRIMARY_TIMEOUT_S=3 # hard timeout for a primary embed call
|
||||
|
||||
# Embedding model kb-query enforces as a startup invariant (plan §2 decision
|
||||
# 2) against document_chunk.model / document_summary.embedding_model.
|
||||
|
|
|
|||
|
|
@ -5,10 +5,12 @@ service:
|
|||
exposure: private # LAN/Tailscale only, npm@PIHA vhost (kb.kapala.org) is a later step
|
||||
dependencies:
|
||||
- kb-postgres
|
||||
- ollama-piha # soft: local CPU embed fallback (Krok 2) -- kb-query starts without it
|
||||
- forgejo # OIDC identity provider, wired in a later step (plan §8); not yet enforced
|
||||
# ollama@SOLARIA is an external, optional runtime dependency (embed calls), not a hard
|
||||
# dependency here -- SOLARIA may be offline; /search then fails per-request with 503,
|
||||
# /healthz still answers (same tolerance pattern as llm-gateway).
|
||||
# ollama@SOLARIA is an external, optional runtime dependency (primary embed backend), not
|
||||
# a hard dependency here -- SOLARIA sleeps ~16 h/day; the embed router (app/embed_router.py)
|
||||
# then falls back to ollama-piha on this host. /search 503s only when BOTH legs are down;
|
||||
# /healthz always answers (same tolerance pattern as llm-gateway).
|
||||
ports:
|
||||
- container: 8080
|
||||
host: 8230 # LAN_BIND_IP only, never 0.0.0.0
|
||||
|
|
@ -28,6 +30,12 @@ service:
|
|||
env_vars:
|
||||
- LAN_BIND_IP # required — compose port-bind interpolation
|
||||
- KB_DSN # required — asyncpg DSN for kb-postgres@PIHA
|
||||
- OLLAMA_URL # optional — defaults to http://solaria:11434
|
||||
- EMBED_MODEL # optional — defaults to bge-m3; startup invariant vs document_chunk/document_summary
|
||||
- SUMMARY_MODEL # optional — defaults to claude-haiku-4-5; cascade_query's stage-1 model
|
||||
- EMBED_PRIMARY_URL # optional — primary embed backend, defaults to http://solaria:11434
|
||||
- EMBED_FALLBACK_URL # optional — local fallback (ollama-piha); empty disables the fallback leg
|
||||
- EMBED_PRIMARY_NAME # optional — log/response label, defaults to solaria
|
||||
- EMBED_FALLBACK_NAME # optional — log/response label, defaults to piha
|
||||
- EMBED_HEALTH_TTL_S # optional — primary health verdict cache, defaults to 30
|
||||
- EMBED_HEALTH_TIMEOUT_S # optional — /api/tags probe timeout, defaults to 1.5
|
||||
- EMBED_PRIMARY_TIMEOUT_S # optional — hard timeout for a primary embed call, defaults to 3
|
||||
- EMBED_MODEL # optional — defaults to bge-m3; startup invariant vs document_chunk/document_summary + per-backend /api/tags check
|
||||
- SUMMARY_MODEL # optional — defaults to claude-haiku-4-5; cascade stage-1 model
|
||||
|
|
|
|||
224
services/kb-query/tests/test_embed_router.py
Normal file
224
services/kb-query/tests/test_embed_router.py
Normal file
|
|
@ -0,0 +1,224 @@
|
|||
"""Unit tests for the SOLARIA->PIHA embed fallback state machine (app.embed_router) -- no
|
||||
real Ollama on either side. Fake backends are keyed by URL so one fake session can serve
|
||||
both legs; the clock is injected so the 30 s TTL is tested without sleeping."""
|
||||
from __future__ import annotations
|
||||
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
import aiohttp
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
|
||||
|
||||
from app.embed_router import EmbedBackendError, EmbedRouter, ModelMismatchError # noqa: E402
|
||||
|
||||
PRIMARY = "http://solaria:11434"
|
||||
FALLBACK = "http://192.168.31.5:11434"
|
||||
|
||||
|
||||
class _FakeResponse:
|
||||
def __init__(self, payload=None, exc=None, status=200):
|
||||
self._payload = payload
|
||||
self._exc = exc
|
||||
self.status = status # check_ollama_health reads resp.status directly
|
||||
|
||||
async def __aenter__(self):
|
||||
if self._exc is not None:
|
||||
raise self._exc
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *exc):
|
||||
return False
|
||||
|
||||
def raise_for_status(self):
|
||||
pass
|
||||
|
||||
async def json(self):
|
||||
return self._payload
|
||||
|
||||
|
||||
class _FakeBackend:
|
||||
"""One fake Ollama: `models` drives /api/tags, `down=True` refuses every connection,
|
||||
`embed_exc` makes /api/embeddings fail while /api/tags still answers (the plan's step-3b
|
||||
scenario: probe says up, the real embed then dies mid-query)."""
|
||||
|
||||
def __init__(self, models=("bge-m3:latest",), down=False, embed_exc=None, vector_value=0.01):
|
||||
self.models = list(models)
|
||||
self.down = down
|
||||
self.embed_exc = embed_exc
|
||||
self.vector_value = vector_value
|
||||
self.tags_calls = 0
|
||||
self.embed_calls = 0
|
||||
|
||||
|
||||
class _FakeSession:
|
||||
def __init__(self, backends: dict):
|
||||
self._backends = backends
|
||||
|
||||
def _backend(self, url):
|
||||
for base, backend in self._backends.items():
|
||||
if url.startswith(base):
|
||||
return backend
|
||||
raise AssertionError(f"unexpected url: {url}")
|
||||
|
||||
def get(self, url, timeout=None):
|
||||
backend = self._backend(url)
|
||||
backend.tags_calls += 1
|
||||
if backend.down:
|
||||
return _FakeResponse(exc=aiohttp.ClientConnectionError("connection refused"))
|
||||
return _FakeResponse({"models": [{"name": n} for n in backend.models]})
|
||||
|
||||
def post(self, url, json):
|
||||
backend = self._backend(url)
|
||||
backend.embed_calls += 1
|
||||
if backend.down:
|
||||
return _FakeResponse(exc=aiohttp.ClientConnectionError("connection refused"))
|
||||
if backend.embed_exc is not None:
|
||||
return _FakeResponse(exc=backend.embed_exc)
|
||||
return _FakeResponse({"embedding": [backend.vector_value] * 1024})
|
||||
|
||||
|
||||
class _FakeClock:
|
||||
def __init__(self):
|
||||
self.now = 1000.0
|
||||
|
||||
def __call__(self):
|
||||
return self.now
|
||||
|
||||
|
||||
def _router(fallback_url=FALLBACK, clock=None, **kwargs):
|
||||
return EmbedRouter(
|
||||
PRIMARY,
|
||||
fallback_url,
|
||||
embed_model="bge-m3",
|
||||
clock=clock or _FakeClock(),
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
class TestPrimaryHealthy:
|
||||
async def test_embeds_on_primary_and_reports_its_name(self):
|
||||
solaria = _FakeBackend()
|
||||
piha = _FakeBackend()
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
|
||||
embedding, backend = await _router().embed(session, "q")
|
||||
assert backend == "solaria"
|
||||
assert len(embedding) == 1024
|
||||
assert solaria.embed_calls == 1
|
||||
assert piha.embed_calls == 0
|
||||
|
||||
async def test_health_verdict_is_cached_within_ttl(self):
|
||||
solaria = _FakeBackend()
|
||||
clock = _FakeClock()
|
||||
router = _router(clock=clock)
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: _FakeBackend()})
|
||||
await router.embed(session, "q1")
|
||||
probes_after_first = solaria.tags_calls # probe + first-use model verification
|
||||
clock.now += 10 # inside the 30 s TTL
|
||||
await router.embed(session, "q2")
|
||||
assert solaria.tags_calls == probes_after_first # no new probe, model check done once
|
||||
|
||||
async def test_model_tag_with_latest_suffix_satisfies_bare_model_name(self):
|
||||
solaria = _FakeBackend(models=["bge-m3:latest"])
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: _FakeBackend()})
|
||||
_, backend = await _router().embed(session, "q")
|
||||
assert backend == "solaria"
|
||||
|
||||
|
||||
class TestFailover:
|
||||
async def test_primary_down_at_probe_falls_back_to_piha(self):
|
||||
solaria = _FakeBackend(down=True)
|
||||
piha = _FakeBackend(vector_value=0.02)
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
|
||||
embedding, backend = await _router().embed(session, "q")
|
||||
assert backend == "piha"
|
||||
assert embedding[0] == 0.02
|
||||
assert solaria.embed_calls == 0
|
||||
|
||||
async def test_mid_embed_failure_serves_same_query_from_fallback(self):
|
||||
# Plan §2 D2 step 3b: probe said up, the real embed then dies -- the SAME request
|
||||
# must come back from the fallback, never as an error to the user.
|
||||
solaria = _FakeBackend(embed_exc=aiohttp.ClientConnectionError("died mid-embed"))
|
||||
piha = _FakeBackend(vector_value=0.02)
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
|
||||
router = _router()
|
||||
embedding, backend = await router.embed(session, "q")
|
||||
assert backend == "piha"
|
||||
assert embedding[0] == 0.02
|
||||
assert solaria.embed_calls == 1
|
||||
|
||||
async def test_down_verdict_is_cached_and_skips_primary_until_ttl_expires(self):
|
||||
solaria = _FakeBackend(down=True)
|
||||
piha = _FakeBackend()
|
||||
clock = _FakeClock()
|
||||
router = _router(clock=clock)
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
|
||||
await router.embed(session, "q1")
|
||||
probes = solaria.tags_calls
|
||||
clock.now += 10 # still inside TTL -> no re-probe, straight to fallback
|
||||
_, backend = await router.embed(session, "q2")
|
||||
assert backend == "piha"
|
||||
assert solaria.tags_calls == probes
|
||||
|
||||
async def test_traffic_returns_to_primary_after_ttl_expiry(self):
|
||||
# Test C from the task spec: SOLARIA comes back -> within one TTL window the
|
||||
# router re-probes and routes to the GPU again.
|
||||
solaria = _FakeBackend(down=True)
|
||||
piha = _FakeBackend()
|
||||
clock = _FakeClock()
|
||||
router = _router(clock=clock)
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
|
||||
_, backend = await router.embed(session, "q1")
|
||||
assert backend == "piha"
|
||||
solaria.down = False # SOLARIA wakes up
|
||||
clock.now += 31 # TTL (30 s) expired
|
||||
_, backend = await router.embed(session, "q2")
|
||||
assert backend == "solaria"
|
||||
|
||||
|
||||
class TestNoUsableBackend:
|
||||
async def test_primary_down_without_fallback_raises_backend_error(self):
|
||||
solaria = _FakeBackend(down=True)
|
||||
session = _FakeSession({PRIMARY: solaria})
|
||||
with pytest.raises(EmbedBackendError, match="no EMBED_FALLBACK_URL"):
|
||||
await _router(fallback_url=None).embed(session, "q")
|
||||
|
||||
async def test_both_backends_down_raises_backend_error(self):
|
||||
session = _FakeSession({PRIMARY: _FakeBackend(down=True), FALLBACK: _FakeBackend(down=True)})
|
||||
with pytest.raises(EmbedBackendError, match="unreachable"):
|
||||
await _router().embed(session, "q")
|
||||
|
||||
|
||||
class TestModelInvariant:
|
||||
async def test_fallback_without_bge_m3_is_a_loud_error_not_a_silent_embed(self):
|
||||
solaria = _FakeBackend(down=True)
|
||||
piha = _FakeBackend(models=["llama3:8b"])
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
|
||||
with pytest.raises(ModelMismatchError, match="bge-m3"):
|
||||
await _router().embed(session, "q")
|
||||
assert piha.embed_calls == 0 # never embedded in the wrong vector space
|
||||
|
||||
async def test_primary_without_bge_m3_fails_over_to_verified_fallback(self):
|
||||
solaria = _FakeBackend(models=["llama3:8b"])
|
||||
piha = _FakeBackend()
|
||||
session = _FakeSession({PRIMARY: solaria, FALLBACK: piha})
|
||||
_, backend = await _router().embed(session, "q")
|
||||
assert backend == "piha"
|
||||
assert solaria.embed_calls == 0
|
||||
|
||||
|
||||
class TestStatusReporting:
|
||||
async def test_primary_status_up_and_fallback_status_up(self):
|
||||
session = _FakeSession({PRIMARY: _FakeBackend(), FALLBACK: _FakeBackend()})
|
||||
router = _router()
|
||||
assert await router.primary_status(session) == "up"
|
||||
assert await router.fallback_status(session) == "up"
|
||||
|
||||
async def test_fallback_status_unconfigured_without_fallback_url(self):
|
||||
session = _FakeSession({PRIMARY: _FakeBackend()})
|
||||
assert await _router(fallback_url=None).fallback_status(session) == "unconfigured"
|
||||
|
||||
async def test_primary_status_down_when_probe_fails(self):
|
||||
session = _FakeSession({PRIMARY: _FakeBackend(down=True), FALLBACK: _FakeBackend()})
|
||||
assert await _router().primary_status(session) == "down"
|
||||
|
|
@ -75,30 +75,23 @@ class _FakeConn:
|
|||
raise AssertionError(f"unexpected query: {query}")
|
||||
|
||||
|
||||
class _FakeEmbedResponse:
|
||||
def __init__(self, payload):
|
||||
self._payload = payload
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *exc):
|
||||
return False
|
||||
|
||||
def raise_for_status(self):
|
||||
pass
|
||||
|
||||
async def json(self):
|
||||
return self._payload
|
||||
|
||||
|
||||
class _FakeSession:
|
||||
def __init__(self):
|
||||
self.post_calls: list[dict] = []
|
||||
"""run_search no longer talks HTTP itself -- embedding goes through the router
|
||||
(below), so the session is just passed through untouched."""
|
||||
|
||||
def post(self, url, json):
|
||||
self.post_calls.append({"url": url, "json": json})
|
||||
return _FakeEmbedResponse({"embedding": [0.01] * 1024})
|
||||
|
||||
class _FakeRouter:
|
||||
"""Stands in for app.embed_router.EmbedRouter: returns a fixed embedding and the name
|
||||
of the backend that 'served' it, mirroring EmbedRouter.embed's contract."""
|
||||
|
||||
def __init__(self, backend="solaria", primary_name="solaria"):
|
||||
self._backend = backend
|
||||
self.primary = type("B", (), {"name": primary_name})()
|
||||
self.embed_calls: list[str] = []
|
||||
|
||||
async def embed(self, session, text):
|
||||
self.embed_calls.append(text)
|
||||
return [0.01] * 1024, self._backend
|
||||
|
||||
|
||||
class TestRunSearchHappyPath:
|
||||
|
|
@ -110,11 +103,12 @@ class TestRunSearchHappyPath:
|
|||
)
|
||||
session = _FakeSession()
|
||||
result = await run_search(
|
||||
conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
|
||||
conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
|
||||
)
|
||||
assert result["query"] == "polisa PZU"
|
||||
assert result["mode"] == "cascade"
|
||||
assert result["sol_status"] == "up"
|
||||
assert result["embed_backend"] == "solaria"
|
||||
assert len(result["results"]) == 1
|
||||
hit = result["results"][0]
|
||||
assert hit["envelope_id"] == "paperless:119"
|
||||
|
|
@ -131,7 +125,7 @@ class TestRunSearchHappyPath:
|
|||
)
|
||||
session = _FakeSession()
|
||||
result = await run_search(
|
||||
conn, session, "http://fake-ollama", "q", "flat", "bge-m3", "claude-haiku-4-5"
|
||||
conn, session, _FakeRouter(), "q", "flat", "claude-haiku-4-5"
|
||||
)
|
||||
assert result["mode"] == "flat"
|
||||
assert len(result["results"]) == 1
|
||||
|
|
@ -145,7 +139,7 @@ class TestRunSearchHappyPath:
|
|||
)
|
||||
session = _FakeSession()
|
||||
result = await run_search(
|
||||
conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
|
||||
conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
|
||||
)
|
||||
hit = result["results"][0]
|
||||
assert hit["summary"] == "Polisa OC 2024"
|
||||
|
|
@ -159,7 +153,7 @@ class TestRunSearchHappyPath:
|
|||
)
|
||||
session = _FakeSession()
|
||||
result = await run_search(
|
||||
conn, session, "http://fake-ollama", "polisa PZU", "cascade", "bge-m3", "claude-haiku-4-5"
|
||||
conn, session, _FakeRouter(), "polisa PZU", "cascade", "claude-haiku-4-5"
|
||||
)
|
||||
hit = result["results"][0]
|
||||
assert hit["summary"] is None
|
||||
|
|
@ -180,7 +174,7 @@ class TestRunSearchHappyPath:
|
|||
)
|
||||
session = _FakeSession()
|
||||
result = await run_search(
|
||||
conn, session, "http://fake-ollama", "q", "hybrid", "bge-m3", "claude-haiku-4-5"
|
||||
conn, session, _FakeRouter(), "q", "hybrid", "claude-haiku-4-5"
|
||||
)
|
||||
assert result["mode"] == "hybrid"
|
||||
envelope_ids = [r["envelope_id"] for r in result["results"]]
|
||||
|
|
@ -201,7 +195,7 @@ class TestRunSearchHappyPath:
|
|||
)
|
||||
session = _FakeSession()
|
||||
result = await run_search(
|
||||
conn, session, "http://fake-ollama", "q", "cascade", "bge-m3", "claude-haiku-4-5"
|
||||
conn, session, _FakeRouter(), "q", "cascade", "claude-haiku-4-5"
|
||||
)
|
||||
hit = result["results"][0]
|
||||
assert hit["source"] == "gmail"
|
||||
|
|
@ -210,6 +204,24 @@ class TestRunSearchHappyPath:
|
|||
assert hit["mail_ui_url"] is None
|
||||
|
||||
|
||||
class TestRunSearchEmbedBackendMarking:
|
||||
async def test_fallback_embed_marks_backend_piha_and_sol_status_down(self):
|
||||
# Task spec: the response must say WHICH backend embedded the query (quality
|
||||
# debugging), and sol_status must reflect the router's world view -- the frontend
|
||||
# renders "down" as "offline (fallback embed)".
|
||||
conn = _FakeConn(
|
||||
summaries=[("paperless:119", 0.1)],
|
||||
chunks_by_envelope={"paperless:119": [(2, "hit text", 0.34)]},
|
||||
envelopes={"paperless:119": {"source": "paperless", "entities": []}},
|
||||
)
|
||||
result = await run_search(
|
||||
conn, _FakeSession(), _FakeRouter(backend="piha"), "q", "cascade", "claude-haiku-4-5"
|
||||
)
|
||||
assert result["embed_backend"] == "piha"
|
||||
assert result["sol_status"] == "down"
|
||||
assert len(result["results"]) == 1
|
||||
|
||||
|
||||
class TestRunSearchNoGoodResults:
|
||||
async def test_results_above_no_answer_threshold_are_still_returned_unfiltered(self):
|
||||
# Plan §7: the 0.55 "no answer" colour threshold is a frontend concern -- the API
|
||||
|
|
@ -222,7 +234,7 @@ class TestRunSearchNoGoodResults:
|
|||
)
|
||||
session = _FakeSession()
|
||||
result = await run_search(
|
||||
conn, session, "http://fake-ollama", "unrelated query", "cascade", "bge-m3", "claude-haiku-4-5"
|
||||
conn, session, _FakeRouter(), "unrelated query", "cascade", "claude-haiku-4-5"
|
||||
)
|
||||
assert len(result["results"]) == 1
|
||||
assert result["results"][0]["dist"] == 0.62
|
||||
|
|
@ -231,6 +243,6 @@ class TestRunSearchNoGoodResults:
|
|||
conn = _FakeConn(summaries=[], chunks_by_envelope={}, envelopes={})
|
||||
session = _FakeSession()
|
||||
result = await run_search(
|
||||
conn, session, "http://fake-ollama", "nothing matches", "cascade", "bge-m3", "claude-haiku-4-5"
|
||||
conn, session, _FakeRouter(), "nothing matches", "cascade", "claude-haiku-4-5"
|
||||
)
|
||||
assert result["results"] == []
|
||||
|
|
|
|||
76
services/ollama-piha/README.md
Normal file
76
services/ollama-piha/README.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# ollama-piha
|
||||
|
||||
Local CPU Ollama on **PIHA**, serving exactly one purpose: the **fallback embed
|
||||
backend** for `kb-query` while SOLARIA (the GPU node, ~16 h/day powered off)
|
||||
sleeps. Model: `bge-m3` — the **same** model as SOLARIA's Ollama, because query
|
||||
embeddings must live in the same vector space as the pgvector index
|
||||
(`document_chunk.embedding VECTOR(1024)`); a different/smaller model is not an
|
||||
option (module 5 phase 4 plan §2 decision 2).
|
||||
|
||||
Expected latency: bge-m3 embeds in ~207 ms on SOLARIA's GPU vs ~790 ms on x86
|
||||
CPU; on the Pi 5 expect single seconds per query (plus model load, since the
|
||||
model is never resident — see below). Slower but alive beats fast but dead.
|
||||
|
||||
## Design constraints
|
||||
|
||||
- **`OLLAMA_KEEP_ALIVE=0`** (pinned in compose): PIHA is the RAM-bound 8 GB box
|
||||
shared with Home Assistant. The model is unloaded immediately after every
|
||||
call — a transient ~1.5–2 GB spike per embed, ~100 MB idle daemon, never a
|
||||
resident cost.
|
||||
- **`mem_limit: 2560m`** (host override, `hosts/piha/runtime/ollama-piha/`):
|
||||
hard cgroup ceiling, plan §2 D2 starting value. The cgroup OOM killer
|
||||
restarts this container instead of the host OOM killer picking a victim
|
||||
(which could be Home Assistant). Confirm/trim after live calibration.
|
||||
- **Bind**: `127.0.0.1` + `LAN_BIND_IP` (192.168.31.5) only — kb-query calls it
|
||||
over the host LAN interface (same pattern as kb-query → kb-postgres:5433).
|
||||
Never `0.0.0.0`, never a Tailscale bind, no public ingress.
|
||||
- **Storage**: Docker named volume `ollama_piha_models` (NVMe data-root), not a
|
||||
bind mount — the ollama image runs as in-container root and would break
|
||||
PIHA's uid pattern (host oskar=1004, containers uid 1000, setgid group pi)
|
||||
if it wrote to a shared bind directory.
|
||||
|
||||
## Deploy (PIHA, master, after merge)
|
||||
|
||||
```bash
|
||||
cd ~/homelab-codex-ws && git pull
|
||||
cp services/ollama-piha/env.example services/ollama-piha/.env # LAN_BIND_IP
|
||||
docker compose -f services/ollama-piha/docker-compose.yml \
|
||||
-f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
|
||||
--env-file services/ollama-piha/.env up -d
|
||||
```
|
||||
|
||||
**Then pull the model — this does NOT happen automatically:**
|
||||
|
||||
```bash
|
||||
docker exec ollama-piha ollama pull bge-m3
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
services/ollama-piha/healthcheck.sh # checks container + API + bge-m3 present
|
||||
time curl -s http://127.0.0.1:11434/api/embeddings \
|
||||
-d '{"model":"bge-m3","prompt":"test kalibracyjny"}' | head -c 80
|
||||
```
|
||||
|
||||
(`deploy-node.sh` on PIHA also picks this service up from
|
||||
`hosts/piha/services.yaml` once `.env` exists — the `ollama pull bge-m3` step
|
||||
stays manual either way.)
|
||||
|
||||
## Calibration (plan §5 step 4 — gate, not formality)
|
||||
|
||||
Before trusting the fallback under load, on live PIHA at a normal (not
|
||||
night-quiet) hour: run a few embeds as above while watching
|
||||
`docker stats ollama-piha`, note peak RAM and wall time. Verdict per plan §5
|
||||
step 5: keep as default fallback / tune `mem_limit` / fall back to explicit
|
||||
503 degradation.
|
||||
|
||||
## Relation to kb-query
|
||||
|
||||
kb-query's router (`services/kb-query/app/embed_router.py`) health-checks
|
||||
SOLARIA with a ~30 s cache and only sends embeds here while SOLARIA is down.
|
||||
kb-query verifies at first use that this backend actually serves `bge-m3`
|
||||
(`/api/tags`) and refuses to embed against a mismatched model. Configuration:
|
||||
`EMBED_FALLBACK_URL=http://192.168.31.5:11434` in `services/kb-query/.env`.
|
||||
See `services/kb-query/README.md` for the fallback verification plan (tests
|
||||
A/B/C).
|
||||
33
services/ollama-piha/docker-compose.yml
Normal file
33
services/ollama-piha/docker-compose.yml
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
services:
|
||||
ollama-piha:
|
||||
# Multi-arch image with a published linux/arm64 manifest — runs natively on
|
||||
# the Pi 5, no emulation (same mechanism as pgvector/pgvector:pg16 on PIHA).
|
||||
image: ollama/ollama:latest
|
||||
container_name: ollama-piha
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
# PIHA is the RAM-bound 8 GB box shared with Home Assistant. bge-m3 must be
|
||||
# unloaded from RAM immediately after every call — a short spike per embed
|
||||
# (idle daemon ~100 MB), never a resident ~2 GB cost. Fallback traffic is
|
||||
# rare by design: kb-query only embeds here while SOLARIA sleeps.
|
||||
- OLLAMA_KEEP_ALIVE=0
|
||||
ports:
|
||||
# Loopback: healthcheck.sh + manual curl on the node itself.
|
||||
# LAN IP: kb-query runs in a separate compose project/network and reaches
|
||||
# this backend over the host's LAN interface — same reasoning as
|
||||
# kb-query -> kb-postgres (:5433). No 0.0.0.0, no Tailscale bind: nothing
|
||||
# off-node ever embeds here (SOLARIA-side callers use SOLARIA's own Ollama).
|
||||
# Requires .env (from env.example) next to this file at deploy.
|
||||
- "127.0.0.1:11434:11434"
|
||||
- "${LAN_BIND_IP}:11434:11434"
|
||||
volumes:
|
||||
# Named volume, NOT a bind mount: the ollama image runs as in-container
|
||||
# root and would fill a bind dir with root-owned files, breaking PIHA's
|
||||
# uid pattern (host oskar=1004, containers uid 1000 + setgid group pi).
|
||||
# Docker-managed volumes live under the data-root on the NVMe — same
|
||||
# convention as kb_postgres_data.
|
||||
- ollama_piha_models:/root/.ollama
|
||||
|
||||
volumes:
|
||||
ollama_piha_models:
|
||||
name: ollama_piha_models
|
||||
7
services/ollama-piha/env.example
Normal file
7
services/ollama-piha/env.example
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
# ollama-piha — copy to .env (gitignored) next to docker-compose.yml; docker
|
||||
# compose picks it up automatically. Same convention as services/kb-query.
|
||||
|
||||
# LAN IP of PIHA. The published port (11434) binds ONLY to 127.0.0.1 and this
|
||||
# interface — never 0.0.0.0. kb-query's EMBED_FALLBACK_URL points at
|
||||
# http://<LAN_BIND_IP>:11434. Verify after host rebuilds: ip -4 addr.
|
||||
LAN_BIND_IP=192.168.31.5
|
||||
25
services/ollama-piha/healthcheck.sh
Executable file
25
services/ollama-piha/healthcheck.sh
Executable file
|
|
@ -0,0 +1,25 @@
|
|||
#!/bin/bash
|
||||
# Healthcheck for ollama-piha (local CPU embed fallback for kb-query)
|
||||
|
||||
# Container must be running
|
||||
if ! docker ps --filter "name=ollama-piha" --filter "status=running" | grep -qw "ollama-piha"; then
|
||||
echo "[FAIL] ollama-piha container is not running"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# API must answer on loopback
|
||||
TAGS=$(curl -sf http://127.0.0.1:11434/api/tags)
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "[FAIL] ollama-piha API is not responding on 127.0.0.1:11434"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# bge-m3 must be pulled — without it the API answers /api/tags but every kb-query
|
||||
# fallback embed fails the model check (deploy step 3 in README.md).
|
||||
if ! echo "$TAGS" | grep -q '"bge-m3'; then
|
||||
echo "[FAIL] model bge-m3 missing (run: docker exec ollama-piha ollama pull bge-m3)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "[OK] ollama-piha is healthy (bge-m3 present)"
|
||||
exit 0
|
||||
26
services/ollama-piha/service.yaml
Normal file
26
services/ollama-piha/service.yaml
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
service:
|
||||
name: ollama-piha
|
||||
owner_node: piha
|
||||
role: embed-fallback # module 5 phase 4 (plan §2 D2/§5): local CPU bge-m3 embed backend for kb-query while SOLARIA sleeps
|
||||
exposure: private # 127.0.0.1 + LAN bind only — never 0.0.0.0, never public ingress, no Tailscale bind
|
||||
dependencies: []
|
||||
ports:
|
||||
- container: 11434
|
||||
host: 11434 # bound to 127.0.0.1 and LAN_BIND_IP only
|
||||
protocol: tcp
|
||||
healthcheck:
|
||||
type: http
|
||||
endpoint: http://127.0.0.1:11434/api/tags
|
||||
interval: 1m
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
restart_policy: unless-stopped
|
||||
persistence:
|
||||
paths: [] # models live in the Docker named volume ollama_piha_models (NVMe data-root)
|
||||
runtime:
|
||||
config_files:
|
||||
- .env # LAN_BIND_IP (gitignored, from env.example)
|
||||
env_vars:
|
||||
- LAN_BIND_IP # required — compose port-bind interpolation
|
||||
# OLLAMA_KEEP_ALIVE is pinned to 0 in docker-compose.yml (not operator-tunable):
|
||||
# the model must never stay resident on the RAM-bound, HA-sharing Pi 5.
|
||||
Loading…
Reference in a new issue