oskar cb8a19de83 test(kb-query): luki T1–T3 z test_fallback.py + status kalibracji ollama-piha (salvage S3+S4)

T1: one-shot switch przy timeoutcie mid-embed (ścieżka asyncio.wait_for,
dotąd nieprzetestowana — scenariusz 'SOLARIA wisi'). T2: breaker zostaje
'down' po mid-embed failure — kolejne requesty w oknie TTL idą prosto na
fallback bez probe'a. T3: noga fallbacku nie dziedziczy twardego timeoutu
primary. T4 pominięty (semantyka granicy TTL identyczna, wg raportu).
Pytest kb-query: 42/42 PASS.

S4: pomiar kalibracji 2026-07-27 (peak ~983 MiB, GO) dopisany do override'u
i sekcji Calibration w README — master mówił dotąd 'Confirm/trim after live
calibration'; konfiguracja kontenera identyczna z mierzoną, pomiar się
przenosi. Raport dedup: status zaktualizowany na 'salvage wykonany'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-30 16:40:24 +02:00

4.3 KiB

Raw Blame History

ollama-piha

Local CPU Ollama on PIHA, serving exactly one purpose: the fallback embed backend for kb-query while SOLARIA (the GPU node, ~16 h/day powered off) sleeps. Model: bge-m3 — the same model as SOLARIA's Ollama, because query embeddings must live in the same vector space as the pgvector index (document_chunk.embedding VECTOR(1024)); a different/smaller model is not an option (module 5 phase 4 plan §2 decision 2).

Expected latency: bge-m3 embeds in ~207 ms on SOLARIA's GPU vs ~790 ms on x86 CPU; on the Pi 5 expect single seconds per query (plus model load, since the model is never resident — see below). Slower but alive beats fast but dead.

Design constraints

OLLAMA_KEEP_ALIVE=0 (pinned in compose): PIHA is the RAM-bound 8 GB box shared with Home Assistant. The model is unloaded immediately after every call — a transient ~1.5–2 GB spike per embed, ~100 MB idle daemon, never a resident cost.
mem_limit: 2560m (host override, hosts/piha/runtime/ollama-piha/): hard cgroup ceiling, plan §2 D2 starting value. The cgroup OOM killer restarts this container instead of the host OOM killer picking a victim (which could be Home Assistant). Confirm/trim after live calibration.
Bind: 127.0.0.1 + LAN_BIND_IP (192.168.31.5) only — kb-query calls it over the host LAN interface (same pattern as kb-query → kb-postgres:5433). Never 0.0.0.0, never a Tailscale bind, no public ingress.
Storage: Docker named volume ollama_piha_models (NVMe data-root), not a bind mount — the ollama image runs as in-container root and would break PIHA's uid pattern (host oskar=1004, containers uid 1000, setgid group pi) if it wrote to a shared bind directory.

Deploy (PIHA, master, after merge)

cd ~/homelab-codex-ws && git pull
cp services/ollama-piha/env.example services/ollama-piha/.env   # LAN_BIND_IP
docker compose -f services/ollama-piha/docker-compose.yml \
  -f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
  --env-file services/ollama-piha/.env up -d

Then pull the model — this does NOT happen automatically:

docker exec ollama-piha ollama pull bge-m3

Verify:

services/ollama-piha/healthcheck.sh          # checks container + API + bge-m3 present
time curl -s http://127.0.0.1:11434/api/embeddings \
  -d '{"model":"bge-m3","prompt":"test kalibracyjny"}' | head -c 80

(deploy-node.sh on PIHA also picks this service up from hosts/piha/services.yaml once .env exists — the ollama pull bge-m3 step stays manual either way.)

Calibration (plan §5 step 4 — gate, not formality)

Before trusting the fallback under load, on live PIHA at a normal (not night-quiet) hour: run a few embeds as above while watching docker stats ollama-piha, note peak RAM and wall time. Verdict per plan §5 step 5: keep as default fallback / tune mem_limit / fall back to explicit 503 degradation.

Calibration status: measured 2026-07-27 — verdict GO (live PIHA under normal load, 3 consecutive /api/embeddings calls after ollama pull bge-m3; full protocol in docs/sessions/2026-07-27-kb-f4-fallback.md §4):

Latency: 5.25 s (cold start) / 4.41 s / 4.16 s — single seconds as expected, no warm-up between calls by design (OLLAMA_KEEP_ALIVE=0 releases the model after every request, ollama ps shows nothing resident in between).
RAM: idle ~66 MiB, burst peak ~983 MiB (docker stats sampled at 0.3 s) — well inside the 2560m ceiling; host available never dropped below ~1.3 GiB.
Kept as default fallback (no feature flag). The measurement was taken against the same container configuration this repo deploys (image, OLLAMA_KEEP_ALIVE=0, mem_limit: 2560m), so it carries over; only the model storage differed (bind mount then, named volume now), which does not affect RAM/latency.

Relation to kb-query

kb-query's router (services/kb-query/app/embed_router.py) health-checks SOLARIA with a ~30 s cache and only sends embeds here while SOLARIA is down. kb-query verifies at first use that this backend actually serves bge-m3 (/api/tags) and refuses to embed against a mismatched model. Configuration: EMBED_FALLBACK_URL=http://192.168.31.5:11434 in services/kb-query/.env. See services/kb-query/README.md for the fallback verification plan (tests A/B/C).

4.3 KiB Raw Blame History Unescape Escape