64 lines
2.3 KiB
Markdown
64 lines
2.3 KiB
Markdown
|
---
|
||
|
|
okf: "0.1"
|
||
|
|
type: runbook
|
||
|
|
visibility: private
|
||
|
|
status: active
|
||
|
|
updated: 2026-07-30
|
||
|
|
links:
|
||
|
|
- ../services/ollama-piha.md
|
||
|
|
---
|
||
|
|
|
||
|
|
# ollama-piha — deploy i kalibracja
|
||
|
|
|
||
|
|
## Deploy (PIHA, master, after merge)
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cd ~/homelab-codex-ws && git pull
|
||
|
|
cp services/ollama-piha/env.example services/ollama-piha/.env # LAN_BIND_IP
|
||
|
|
docker compose -f services/ollama-piha/docker-compose.yml \
|
||
|
|
-f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
|
||
|
|
--env-file services/ollama-piha/.env up -d
|
||
|
|
```
|
||
|
|
|
||
|
|
**Then pull the model — this does NOT happen automatically:**
|
||
|
|
|
||
|
|
```bash
|
||
|
|
docker exec ollama-piha ollama pull bge-m3
|
||
|
|
```
|
||
|
|
|
||
|
|
Verify:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
services/ollama-piha/healthcheck.sh # checks container + API + bge-m3 present
|
||
|
|
time curl -s http://127.0.0.1:11434/api/embeddings \
|
||
|
|
-d '{"model":"bge-m3","prompt":"test kalibracyjny"}' | head -c 80
|
||
|
|
```
|
||
|
|
|
||
|
|
(`deploy-node.sh` on PIHA also picks this service up from
|
||
|
|
`hosts/piha/services.yaml` once `.env` exists — the `ollama pull bge-m3` step
|
||
|
|
stays manual either way.)
|
||
|
|
|
||
|
|
## Calibration (plan §5 step 4 — gate, not formality)
|
||
|
|
|
||
|
|
Before trusting the fallback under load, on live PIHA at a normal (not
|
||
|
|
night-quiet) hour: run a few embeds as above while watching
|
||
|
|
`docker stats ollama-piha`, note peak RAM and wall time. Verdict per plan §5
|
||
|
|
step 5: keep as default fallback / tune `mem_limit` / fall back to explicit
|
||
|
|
503 degradation.
|
||
|
|
|
||
|
|
**Calibration status: measured 2026-07-27 — verdict GO** (live PIHA under
|
||
|
|
normal load, 3 consecutive `/api/embeddings` calls after `ollama pull bge-m3`;
|
||
|
|
full protocol in `docs/sessions/2026-07-27-kb-f4-fallback.md` §4):
|
||
|
|
|
||
|
|
- Latency: 5.25 s (cold start) / 4.41 s / 4.16 s — single seconds as expected,
|
||
|
|
no warm-up between calls by design (`OLLAMA_KEEP_ALIVE=0` releases the model
|
||
|
|
after every request, `ollama ps` shows nothing resident in between).
|
||
|
|
- RAM: idle ~66 MiB, burst peak ~983 MiB (`docker stats` sampled at 0.3 s) —
|
||
|
|
well inside the 2560m ceiling; host `available` never dropped below ~1.3 GiB.
|
||
|
|
- Kept as **default fallback** (no feature flag). The measurement was taken
|
||
|
|
against the same container configuration this repo deploys (image,
|
||
|
|
`OLLAMA_KEEP_ALIVE=0`, `mem_limit: 2560m`), so it carries over; only the
|
||
|
|
model storage differed (bind mount then, named volume now), which does not
|
||
|
|
affect RAM/latency.
|
||
|
|
|