# ollama-piha

Local, CPU-only Ollama on **PIHA** (arm64, 4 cores, no acceleration —
`hosts/piha/capabilities.yaml`), serving exactly one purpose: `kb-query`'s
embed fallback (`services/kb-query/app/fallback.py`, module 5 phase 4 plan
§2 decision 2 / §5) when Ollama@SOLARIA is unreachable or times out.

**This is not a general-purpose Ollama instance** — no other service should
point at it. It runs `bge-m3` only, the same embedding model kb-query's
startup invariant enforces against `document_chunk`/`document_summary`
(`services/kb-query/app/startup.py`). Do not pull additional models onto it.

## Why `OLLAMA_KEEP_ALIVE=0`

The model is loaded into RAM only for the duration of a request and released
immediately after, rather than staying resident. On a memory-constrained
RPi5 already running kb-postgres, paperless, Home Assistant, Immich,
Forgejo, and more (see `docs/infra/piha-slim-audit-2026-07-02.md`), a second
permanently-resident ~1.5-2 GB model is a worse trade than a short RAM spike
that only happens when this fallback is actually exercised (SOLARIA down —
rare, plan §1.2 `availability_target: medium`).

## Deploy

1. `git pull` on PIHA.
2. `cp services/ollama-piha/env.example services/ollama-piha/.env` (fill in
   `LAN_BIND_IP` if it differs from the default).
3. `docker compose -f services/ollama-piha/docker-compose.yml -f hosts/piha/runtime/ollama-piha/docker-compose.override.yml up -d`
4. `docker exec ollama-piha ollama pull bge-m3` — not baked into the image;
   the model must be pulled once after first start (persists in the
   `/opt/homelab/data/ollama-piha` named volume across restarts).
5. Verify: `services/ollama-piha/healthcheck.sh`, then
   `curl http://192.168.31.5:11434/api/tags` should list `bge-m3`.

Point `kb-query`'s `OLLAMA_PIHA_URL` at `http://192.168.31.5:11434` once this
is live (see `services/kb-query/env.example`).

## Calibration status (plan §5) — GO, measured live 2026-07-27

Measured on live PIHA under normal load (kb-postgres, paperless, Immich,
Home Assistant, Forgejo, etc. all running, not a quiet-night window),
3 sequential embed calls through `docker exec ollama-piha ollama pull bge-m3`
+ `/api/embeddings`:

- **Latency**: 5.25s (first call), 4.41s, 4.16s — consistently single-digit
  seconds, never tens of seconds. Latency doesn't drop on later calls because
  `OLLAMA_KEEP_ALIVE=0` reloads the model every time by design (`ollama ps`
  shows zero resident models between calls) — this is the expected trade-off
  documented above, not a bug.
- **RAM**: peak ~983 MiB during a burst (`docker stats`, baseline idle ~66
  MiB), comfortably inside the `mem_limit: 2560m` ceiling
  (`hosts/piha/runtime/ollama-piha/docker-compose.override.yml`). System-wide
  `available` memory never dropped below ~1.3 GiB during the burst and
  settled back to ~4.2 GiB after — well above the "≥500 MB spare" bar from
  the plan.

**Verdict: GO — enabled as the default fallback**, no
`KB_QUERY_LOCAL_FALLBACK_ENABLED`-style flag needed. `kb-query`'s
`OLLAMA_PIHA_URL` points at this container's real LAN address
(`http://192.168.31.5:11434`) in the live PIHA deployment. Verified live: a
sol-down simulation (`OLLAMA_URL` on kb-query pointed at an unreachable
SOLARIA address) produced `sol_status: "down"` and correct `/search` results
from this container, with `dist` within ~3e-4 of the SOLARIA-GPU baseline
(same top-k order, same hit@3 gate outcome) — see
`docs/sessions/2026-07-27-kb-f4-fallback.md` for the full numbers.

**One-time finding from this calibration**: PIHA also had a leftover, fully
undocumented *native* (non-Docker) `ollama.service` (systemd, v0.6.1, running
since 2026-06-22, zero models ever pulled) that conflicted with this
container's port binding. Confirmed dead (only this session's own probe
requests in its journal) and disabled (`systemctl disable --now`, not
uninstalled — reversible) before deploying this container. See the session
doc for the backlog note to fully remove it if nothing breaks.