docs(ollama): GPU benchmark 207ms/embed vs 790ms CPU (~3.8x sequential) — RTX 4070 Ti SUPER, batching pozostaje dźwignią

This commit is contained in:
oskar 2026-07-16 15:02:07 +02:00
parent 5e920dd7c1
commit fb77793c83
2 changed files with 8 additions and 11 deletions

View file

@ -146,10 +146,10 @@ SOLARIA:
already in place (`nvidia` runtime registered in `daemon.json`), so no
further toolkit work was needed. The GPU reservation was restored in
`docker-compose.yml` and the container recreated.
- **GPU vs CPU embedding throughput**: GPU — TBD after measurement
2026-07-16 (CPU baseline: ≈0.79s/chunk, see
`jobs/documents-ingest/README.md` timing section, updated with the GPU
number once measured).
- **GPU vs CPU embedding throughput** (measured 2026-07-16): GPU 207ms/embed
vs CPU ≈790ms — ~3.8× faster sequential (50 calls, ~600-tok prompt, bge-m3
at 100% GPU per `ollama ps`). Single-request overhead (HTTP/tokenization)
dominates; batching remains the real lever (backlog).
## Nota operacyjna: kontener zniknął po reboocie (2026-07-15 wieczór)

View file

@ -475,10 +475,7 @@ kb-postgres@PIHA:
average chunk count per item will need either a GPU driver fix,
concurrent/batched Ollama calls, or both, before a full mail-corpus run
is practical — flagged for whoever picks up the mail-indexer phase.
- **GPU (RTX 4070 Ti SUPER, driver 595-open, restored 2026-07-16)**: TBD po
pomiarze 2026-07-16 — sterownik NVIDII naprawiony i GPU reservation
przywrócona w `services/ollama/docker-compose.yml`
(`docs/backlog.md`, "Ollama SOLARIA: brak sterownika NVIDII"), ale
sekwencyjny embedding benchmark vs. ≈0.79s/chunk CPU baseline jeszcze nie
zmierzony w tym repo — liczba do uzupełnienia osobnym commitem po
pomiarze na żywym Ollamie.
- **GPU (RTX 4070 Ti SUPER, driver 595-open, restored 2026-07-16)**: 207ms/embed
(50 sekwencyjnych wywołań /api/embeddings, ~600-tok prompt) vs 790ms/chunk CPU
baseline — ~3.8× szybciej sekwencyjnie; przy pojedynczych requestach dominuje
overhead HTTP/tokenizacji, realny skok da dopiero batching (backlog).