docs(ollama): GPU benchmark 207ms/embed vs 790ms CPU (~3.8x sequential) — RTX 4070 Ti SUPER, batching pozostaje dźwignią
This commit is contained in:
parent
5e920dd7c1
commit
fb77793c83
|
|
@ -146,10 +146,10 @@ SOLARIA:
|
|||
already in place (`nvidia` runtime registered in `daemon.json`), so no
|
||||
further toolkit work was needed. The GPU reservation was restored in
|
||||
`docker-compose.yml` and the container recreated.
|
||||
- **GPU vs CPU embedding throughput**: GPU — TBD after measurement
|
||||
2026-07-16 (CPU baseline: ≈0.79s/chunk, see
|
||||
`jobs/documents-ingest/README.md` timing section, updated with the GPU
|
||||
number once measured).
|
||||
- **GPU vs CPU embedding throughput** (measured 2026-07-16): GPU 207ms/embed
|
||||
vs CPU ≈790ms — ~3.8× faster sequential (50 calls, ~600-tok prompt, bge-m3
|
||||
at 100% GPU per `ollama ps`). Single-request overhead (HTTP/tokenization)
|
||||
dominates; batching remains the real lever (backlog).
|
||||
|
||||
## Nota operacyjna: kontener zniknął po reboocie (2026-07-15 wieczór)
|
||||
|
||||
|
|
|
|||
|
|
@ -475,10 +475,7 @@ kb-postgres@PIHA:
|
|||
average chunk count per item will need either a GPU driver fix,
|
||||
concurrent/batched Ollama calls, or both, before a full mail-corpus run
|
||||
is practical — flagged for whoever picks up the mail-indexer phase.
|
||||
- **GPU (RTX 4070 Ti SUPER, driver 595-open, restored 2026-07-16)**: TBD po
|
||||
pomiarze 2026-07-16 — sterownik NVIDII naprawiony i GPU reservation
|
||||
przywrócona w `services/ollama/docker-compose.yml`
|
||||
(`docs/backlog.md`, "Ollama SOLARIA: brak sterownika NVIDII"), ale
|
||||
sekwencyjny embedding benchmark vs. ≈0.79s/chunk CPU baseline jeszcze nie
|
||||
zmierzony w tym repo — liczba do uzupełnienia osobnym commitem po
|
||||
pomiarze na żywym Ollamie.
|
||||
- **GPU (RTX 4070 Ti SUPER, driver 595-open, restored 2026-07-16)**: 207ms/embed
|
||||
(50 sekwencyjnych wywołań /api/embeddings, ~600-tok prompt) vs 790ms/chunk CPU
|
||||
baseline — ~3.8× szybciej sekwencyjnie; przy pojedynczych requestach dominuje
|
||||
overhead HTTP/tokenizacji, realny skok da dopiero batching (backlog).
|
||||
|
|
|
|||
Loading…
Reference in a new issue