a1a57b04bb
Serwisy o jednorodnej tresci (bez wydzielonej sekcji deploy/runbook): kb-postgres, stability-agent, llm-gateway, vikunja, zigbee2mqtt, npm, forgejo, ollama, ir-ac-ha-integration, chelsty-stability-agent. Deprecated: mosquitto (NOT DEPLOYED/LEGACY), home-assistant-ken-legacy (archiwum importu), joplin (stub z 2026-04-15, brak katalogu services/joplin/). git mv + frontmatter, tresc nietknieta. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
65 lines
2.1 KiB
Markdown
65 lines
2.1 KiB
Markdown
---
|
|
okf: "0.1"
|
|
type: service
|
|
visibility: private
|
|
status: active
|
|
updated: 2026-07-03
|
|
links: []
|
|
---
|
|
|
|
# llm-gateway
|
|
|
|
Small FastAPI router in front of Ollama on **SOLARIA**. Runs on the **PIHA**
|
|
node, bound to PIHA's Tailscale IP only (`http://piha:8080` on the mesh; no
|
|
public ingress). Stateless — no volumes, no secrets.
|
|
|
|
Origin: migrated 2026-07-03 from an unmanaged shadow deployment at
|
|
`/opt/llm-gateway` on PIHA into GitOps. The old copy is kept on disk as
|
|
rollback until explicitly retired.
|
|
|
|
## Endpoints
|
|
|
|
| Endpoint | Method | Model | Purpose |
|
|
|---|---|---|---|
|
|
| `/` | GET | — | Health: `{"status":"gateway ok"}` |
|
|
| `/api/chat` | POST | `deepcoder:14b` | General chat completion |
|
|
| `/api/code` | POST | `deepseek-coder:latest` | Code/command generation ("return only code" system prompt) |
|
|
|
|
Request body for both POST routes: `{"prompt": "...", "stream": false}`.
|
|
|
|
`/api/code` quirk (by design): a single-line prompt whose first word is not a
|
|
natural-language starter (write/create/explain/…) is treated as an
|
|
already-formed shell command and echoed back verbatim as `response`.
|
|
|
|
## Upstream dependency
|
|
|
|
Calls Ollama at `http://solaria:11434/api/generate` (override via `OLLAMA_URL`).
|
|
SOLARIA is a GPU node that may be powered down — the gateway stays healthy
|
|
(`/` still answers) but proxied routes return **502** until Ollama is back.
|
|
`solaria` resolves via the host's Tailscale MagicDNS, which Docker's embedded
|
|
DNS forwards to.
|
|
|
|
## Configuration
|
|
|
|
- `.env` — **gitignored**, copy from `env.example`. Holds `TAILSCALE_BIND_IP`
|
|
(required for the mesh-only port bind) and optional `OLLAMA_URL`.
|
|
- No secrets anywhere in the stack.
|
|
|
|
## Deploy (PIHA)
|
|
|
|
1. `git pull` on PIHA (`~/homelab-codex-ws`).
|
|
2. `cp services/llm-gateway/env.example services/llm-gateway/.env` (defaults
|
|
are correct for PIHA).
|
|
3. ```
|
|
docker compose -f services/llm-gateway/docker-compose.yml \
|
|
-f hosts/piha/runtime/llm-gateway/docker-compose.override.yml up -d --build
|
|
```
|
|
4. Verify: `services/llm-gateway/healthcheck.sh`, then from any mesh node:
|
|
`curl http://piha:8080/` → `{"status":"gateway ok"}`.
|
|
|
|
## Tests
|
|
|
|
```
|
|
cd services/llm-gateway && pytest
|
|
```
|