oskar/homelab-codex-ws
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

docs(ollama): GPU benchmark 207ms/embed vs 790ms CPU (~3.8x sequential) — RTX 4070 Ti SUPER, batching pozostaje dźwignią

Browse source
This commit is contained in:
oskar 2026-07-16 15:02:07 +02:00
parent 5e920dd7c1
commit fb77793c83
2 changed files with 8 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
docs/infra/ollama-solaria-cutover-2026-07-15.md
View file

@ -146,10 +146,10 @@ SOLARIA:
already in place (`nvidia` runtime registered in `daemon.json`), so no already in place (`nvidia` runtime registered in `daemon.json`), so no
further toolkit work was needed. The GPU reservation was restored in further toolkit work was needed. The GPU reservation was restored in
`docker-compose.yml` and the container recreated. `docker-compose.yml` and the container recreated.
- **GPU vs CPU embedding throughput**: GPU — TBD after measurement - **GPU vs CPU embedding throughput** (measured 2026-07-16): GPU 207ms/embed
2026-07-16 (CPU baseline: ≈0.79s/chunk, see vs CPU ≈790ms — ~3.8× faster sequential (50 calls, ~600-tok prompt, bge-m3
`jobs/documents-ingest/README.md` timing section, updated with the GPU at 100% GPU per `ollama ps`). Single-request overhead (HTTP/tokenization)
number once measured). dominates; batching remains the real lever (backlog).
## Nota operacyjna: kontener zniknął po reboocie (2026-07-15 wieczór) ## Nota operacyjna: kontener zniknął po reboocie (2026-07-15 wieczór)

11
jobs/documents-ingest/README.md
View file

@ -475,10 +475,7 @@ kb-postgres@PIHA:
average chunk count per item will need either a GPU driver fix, average chunk count per item will need either a GPU driver fix,
concurrent/batched Ollama calls, or both, before a full mail-corpus run concurrent/batched Ollama calls, or both, before a full mail-corpus run
is practical — flagged for whoever picks up the mail-indexer phase. is practical — flagged for whoever picks up the mail-indexer phase.
- **GPU (RTX 4070 Ti SUPER, driver 595-open, restored 2026-07-16)**: TBD po - **GPU (RTX 4070 Ti SUPER, driver 595-open, restored 2026-07-16)**: 207ms/embed
pomiarze 2026-07-16 — sterownik NVIDII naprawiony i GPU reservation (50 sekwencyjnych wywołań /api/embeddings, ~600-tok prompt) vs 790ms/chunk CPU
przywrócona w `services/ollama/docker-compose.yml` baseline — ~3.8× szybciej sekwencyjnie; przy pojedynczych requestach dominuje
(`docs/backlog.md`, "Ollama SOLARIA: brak sterownika NVIDII"), ale overhead HTTP/tokenizacji, realny skok da dopiero batching (backlog).
sekwencyjny embedding benchmark vs. ≈0.79s/chunk CPU baseline jeszcze nie
zmierzony w tym repo — liczba do uzupełnienia osobnym commitem po
pomiarze na żywym Ollamie.