cb8a19de83
T1: one-shot switch przy timeoutcie mid-embed (ścieżka asyncio.wait_for, dotąd nieprzetestowana — scenariusz 'SOLARIA wisi'). T2: breaker zostaje 'down' po mid-embed failure — kolejne requesty w oknie TTL idą prosto na fallback bez probe'a. T3: noga fallbacku nie dziedziczy twardego timeoutu primary. T4 pominięty (semantyka granicy TTL identyczna, wg raportu). Pytest kb-query: 42/42 PASS. S4: pomiar kalibracji 2026-07-27 (peak ~983 MiB, GO) dopisany do override'u i sekcji Calibration w README — master mówił dotąd 'Confirm/trim after live calibration'; konfiguracja kontenera identyczna z mierzoną, pomiar się przenosi. Raport dedup: status zaktualizowany na 'salvage wykonany'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
92 lines
4.3 KiB
Markdown
92 lines
4.3 KiB
Markdown
# ollama-piha
|
||
|
||
Local CPU Ollama on **PIHA**, serving exactly one purpose: the **fallback embed
|
||
backend** for `kb-query` while SOLARIA (the GPU node, ~16 h/day powered off)
|
||
sleeps. Model: `bge-m3` — the **same** model as SOLARIA's Ollama, because query
|
||
embeddings must live in the same vector space as the pgvector index
|
||
(`document_chunk.embedding VECTOR(1024)`); a different/smaller model is not an
|
||
option (module 5 phase 4 plan §2 decision 2).
|
||
|
||
Expected latency: bge-m3 embeds in ~207 ms on SOLARIA's GPU vs ~790 ms on x86
|
||
CPU; on the Pi 5 expect single seconds per query (plus model load, since the
|
||
model is never resident — see below). Slower but alive beats fast but dead.
|
||
|
||
## Design constraints
|
||
|
||
- **`OLLAMA_KEEP_ALIVE=0`** (pinned in compose): PIHA is the RAM-bound 8 GB box
|
||
shared with Home Assistant. The model is unloaded immediately after every
|
||
call — a transient ~1.5–2 GB spike per embed, ~100 MB idle daemon, never a
|
||
resident cost.
|
||
- **`mem_limit: 2560m`** (host override, `hosts/piha/runtime/ollama-piha/`):
|
||
hard cgroup ceiling, plan §2 D2 starting value. The cgroup OOM killer
|
||
restarts this container instead of the host OOM killer picking a victim
|
||
(which could be Home Assistant). Confirm/trim after live calibration.
|
||
- **Bind**: `127.0.0.1` + `LAN_BIND_IP` (192.168.31.5) only — kb-query calls it
|
||
over the host LAN interface (same pattern as kb-query → kb-postgres:5433).
|
||
Never `0.0.0.0`, never a Tailscale bind, no public ingress.
|
||
- **Storage**: Docker named volume `ollama_piha_models` (NVMe data-root), not a
|
||
bind mount — the ollama image runs as in-container root and would break
|
||
PIHA's uid pattern (host oskar=1004, containers uid 1000, setgid group pi)
|
||
if it wrote to a shared bind directory.
|
||
|
||
## Deploy (PIHA, master, after merge)
|
||
|
||
```bash
|
||
cd ~/homelab-codex-ws && git pull
|
||
cp services/ollama-piha/env.example services/ollama-piha/.env # LAN_BIND_IP
|
||
docker compose -f services/ollama-piha/docker-compose.yml \
|
||
-f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
|
||
--env-file services/ollama-piha/.env up -d
|
||
```
|
||
|
||
**Then pull the model — this does NOT happen automatically:**
|
||
|
||
```bash
|
||
docker exec ollama-piha ollama pull bge-m3
|
||
```
|
||
|
||
Verify:
|
||
|
||
```bash
|
||
services/ollama-piha/healthcheck.sh # checks container + API + bge-m3 present
|
||
time curl -s http://127.0.0.1:11434/api/embeddings \
|
||
-d '{"model":"bge-m3","prompt":"test kalibracyjny"}' | head -c 80
|
||
```
|
||
|
||
(`deploy-node.sh` on PIHA also picks this service up from
|
||
`hosts/piha/services.yaml` once `.env` exists — the `ollama pull bge-m3` step
|
||
stays manual either way.)
|
||
|
||
## Calibration (plan §5 step 4 — gate, not formality)
|
||
|
||
Before trusting the fallback under load, on live PIHA at a normal (not
|
||
night-quiet) hour: run a few embeds as above while watching
|
||
`docker stats ollama-piha`, note peak RAM and wall time. Verdict per plan §5
|
||
step 5: keep as default fallback / tune `mem_limit` / fall back to explicit
|
||
503 degradation.
|
||
|
||
**Calibration status: measured 2026-07-27 — verdict GO** (live PIHA under
|
||
normal load, 3 consecutive `/api/embeddings` calls after `ollama pull bge-m3`;
|
||
full protocol in `docs/sessions/2026-07-27-kb-f4-fallback.md` §4):
|
||
|
||
- Latency: 5.25 s (cold start) / 4.41 s / 4.16 s — single seconds as expected,
|
||
no warm-up between calls by design (`OLLAMA_KEEP_ALIVE=0` releases the model
|
||
after every request, `ollama ps` shows nothing resident in between).
|
||
- RAM: idle ~66 MiB, burst peak ~983 MiB (`docker stats` sampled at 0.3 s) —
|
||
well inside the 2560m ceiling; host `available` never dropped below ~1.3 GiB.
|
||
- Kept as **default fallback** (no feature flag). The measurement was taken
|
||
against the same container configuration this repo deploys (image,
|
||
`OLLAMA_KEEP_ALIVE=0`, `mem_limit: 2560m`), so it carries over; only the
|
||
model storage differed (bind mount then, named volume now), which does not
|
||
affect RAM/latency.
|
||
|
||
## Relation to kb-query
|
||
|
||
kb-query's router (`services/kb-query/app/embed_router.py`) health-checks
|
||
SOLARIA with a ~30 s cache and only sends embeds here while SOLARIA is down.
|
||
kb-query verifies at first use that this backend actually serves `bge-m3`
|
||
(`/api/tags`) and refuses to embed against a mismatched model. Configuration:
|
||
`EMBED_FALLBACK_URL=http://192.168.31.5:11434` in `services/kb-query/.env`.
|
||
See `services/kb-query/README.md` for the fallback verification plan (tests
|
||
A/B/C).
|