oskar/homelab-codex-ws
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
homelab-codex-ws/services/ollama-piha
History
oskar 3d4ee3818d feat(kb-query): active embed fallback SOLARIA→PIHA (module 5 phase 4, plan §2/§5) 
Last missing core piece of KB phase 4: kb-query no longer hard-fails /search
when Ollama@SOLARIA is unreachable. app/fallback.py implements the plan's
circuit-breaker exactly (30s cached health probe, 3s hard embed timeout on
SOLARIA, one-shot same-request switch to a new local ollama-piha@PIHA
container on timeout/error). sol_status in /healthz and /search now reflects
the real breaker state instead of a hardcoded "up".

New services/ollama-piha (bge-m3, OLLAMA_KEEP_ALIVE=0, arm64/no-GPU) is the
local fallback leg. Live calibration on PIHA (2026-07-27, normal load):
embed latency 4.2-5.2s, RAM peak ~983MiB against a 2.5GiB ceiling -- both
inside the plan's go-bar, so the fallback is enabled by default rather than
gated behind a flag. Calibration also surfaced and disabled (not removed) a
previously-undocumented orphaned native ollama.service on PIHA that had been
conflicting with the container's port.

The embed-model invariant (query embedding == document_chunk.model) still
enforces once at startup, since both fallback legs share one EMBED_MODEL
constant by construction; a redundant per-request DB check was deliberately
skipped and the invariant is instead proven structurally by test.

retrieval_eval.py gains --transport http (plan §2 decision 6/§9), previously
unimplemented. Verified live: HTTP transport is bit-identical to direct
transport against the same live SOLARIA (0 mismatches), and a live sol-down
simulation (kb-query's own OLLAMA_URL pointed at a dead address, no other
Ollama consumer touched) shows the PIHA fallback answering with the same
hit@3 gate outcome and dist within ~3e-4 of the SOLARIA baseline.

Zero changes to DB schema or kb_retrieval's retrieval logic -- only the
embed + health layer, per task constraints.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-27 22:23:53 +02:00
..
docker-compose.yml feat(kb-query): active embed fallback SOLARIA→PIHA (module 5 phase 4, plan §2/§5) 2026-07-27 22:23:53 +02:00
env.example feat(kb-query): active embed fallback SOLARIA→PIHA (module 5 phase 4, plan §2/§5) 2026-07-27 22:23:53 +02:00
healthcheck.sh feat(kb-query): active embed fallback SOLARIA→PIHA (module 5 phase 4, plan §2/§5) 2026-07-27 22:23:53 +02:00
README.md feat(kb-query): active embed fallback SOLARIA→PIHA (module 5 phase 4, plan §2/§5) 2026-07-27 22:23:53 +02:00
service.yaml feat(kb-query): active embed fallback SOLARIA→PIHA (module 5 phase 4, plan §2/§5) 2026-07-27 22:23:53 +02:00

README.md

ollama-piha

Local, CPU-only Ollama on PIHA (arm64, 4 cores, no acceleration — hosts/piha/capabilities.yaml), serving exactly one purpose: kb-query's embed fallback (services/kb-query/app/fallback.py, module 5 phase 4 plan §2 decision 2 / §5) when Ollama@SOLARIA is unreachable or times out.

This is not a general-purpose Ollama instance — no other service should point at it. It runs bge-m3 only, the same embedding model kb-query's startup invariant enforces against document_chunk/document_summary (services/kb-query/app/startup.py). Do not pull additional models onto it.

Why OLLAMA_KEEP_ALIVE=0

The model is loaded into RAM only for the duration of a request and released immediately after, rather than staying resident. On a memory-constrained RPi5 already running kb-postgres, paperless, Home Assistant, Immich, Forgejo, and more (see docs/infra/piha-slim-audit-2026-07-02.md), a second permanently-resident ~1.5-2 GB model is a worse trade than a short RAM spike that only happens when this fallback is actually exercised (SOLARIA down — rare, plan §1.2 availability_target: medium).

Deploy

  1. git pull on PIHA.
  2. cp services/ollama-piha/env.example services/ollama-piha/.env (fill in LAN_BIND_IP if it differs from the default).
  3. docker compose -f services/ollama-piha/docker-compose.yml -f hosts/piha/runtime/ollama-piha/docker-compose.override.yml up -d
  4. docker exec ollama-piha ollama pull bge-m3 — not baked into the image; the model must be pulled once after first start (persists in the /opt/homelab/data/ollama-piha named volume across restarts).
  5. Verify: services/ollama-piha/healthcheck.sh, then curl http://192.168.31.5:11434/api/tags should list bge-m3.

Point kb-query's OLLAMA_PIHA_URL at http://192.168.31.5:11434 once this is live (see services/kb-query/env.example).

Calibration status (plan §5) — GO, measured live 2026-07-27

Measured on live PIHA under normal load (kb-postgres, paperless, Immich, Home Assistant, Forgejo, etc. all running, not a quiet-night window), 3 sequential embed calls through docker exec ollama-piha ollama pull bge-m3

  • /api/embeddings:
  • Latency: 5.25s (first call), 4.41s, 4.16s — consistently single-digit seconds, never tens of seconds. Latency doesn't drop on later calls because OLLAMA_KEEP_ALIVE=0 reloads the model every time by design (ollama ps shows zero resident models between calls) — this is the expected trade-off documented above, not a bug.
  • RAM: peak ~983 MiB during a burst (docker stats, baseline idle ~66 MiB), comfortably inside the mem_limit: 2560m ceiling (hosts/piha/runtime/ollama-piha/docker-compose.override.yml). System-wide available memory never dropped below ~1.3 GiB during the burst and settled back to ~4.2 GiB after — well above the "≥500 MB spare" bar from the plan.

Verdict: GO — enabled as the default fallback, no KB_QUERY_LOCAL_FALLBACK_ENABLED-style flag needed. kb-query's OLLAMA_PIHA_URL points at this container's real LAN address (http://192.168.31.5:11434) in the live PIHA deployment. Verified live: a sol-down simulation (OLLAMA_URL on kb-query pointed at an unreachable SOLARIA address) produced sol_status: "down" and correct /search results from this container, with dist within ~3e-4 of the SOLARIA-GPU baseline (same top-k order, same hit@3 gate outcome) — see docs/sessions/2026-07-27-kb-f4-fallback.md for the full numbers.

One-time finding from this calibration: PIHA also had a leftover, fully undocumented native (non-Docker) ollama.service (systemd, v0.6.1, running since 2026-06-22, zero models ever pulled) that conflicted with this container's port binding. Confirmed dead (only this session's own probe requests in its journal) and disabled (systemctl disable --now, not uninstalled — reversible) before deploying this container. See the session doc for the backlog note to fully remove it if nothing breaks.