cbb910d856
Kod zyl tylko na dysku PIHA w /opt/llm-gateway (bez gita). Przeniesiony do
services/llm-gateway/ pod pelny wzorzec homelaba:
- app/main.py: kod 1:1 z PIHA + OLLAMA_URL/CHAT_MODEL/CODE_MODEL
nadpisywalne przez env (defaulty bez zmian)
- docker-compose.yml: bind TYLKO do Tailscale IP (${TAILSCALE_BIND_IP},
wzorzec fleet-prometheus), nie 0.0.0.0 jak w starym compose
- service.yaml, env.example (bez sekretow), healthcheck.sh, README, testy
- hosts/piha/runtime/llm-gateway: mem_limit 256m (PIHA jest RAM-bound)
- rejestracja w hosts/piha/services.yaml i inventory/topology.yaml
Zepsuty /opt/llm-gateway/docker-compose.yml (zduplikowany klucz ports)
celowo NIE przeniesiony.
DoD: pytest 4 passed; docker build + smoke run OK (GET / -> gateway ok).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| app | ||
| tests | ||
| .dockerignore | ||
| docker-compose.yml | ||
| Dockerfile | ||
| env.example | ||
| healthcheck.sh | ||
| README.md | ||
| requirements.txt | ||
| service.yaml | ||
llm-gateway
Small FastAPI router in front of Ollama on SOLARIA. Runs on the PIHA
node, bound to PIHA's Tailscale IP only (http://piha:8080 on the mesh; no
public ingress). Stateless — no volumes, no secrets.
Origin: migrated 2026-07-03 from an unmanaged shadow deployment at
/opt/llm-gateway on PIHA into GitOps. The old copy is kept on disk as
rollback until explicitly retired.
Endpoints
| Endpoint | Method | Model | Purpose |
|---|---|---|---|
/ |
GET | — | Health: {"status":"gateway ok"} |
/api/chat |
POST | deepcoder:14b |
General chat completion |
/api/code |
POST | deepseek-coder:latest |
Code/command generation ("return only code" system prompt) |
Request body for both POST routes: {"prompt": "...", "stream": false}.
/api/code quirk (by design): a single-line prompt whose first word is not a
natural-language starter (write/create/explain/…) is treated as an
already-formed shell command and echoed back verbatim as response.
Upstream dependency
Calls Ollama at http://solaria:11434/api/generate (override via OLLAMA_URL).
SOLARIA is a GPU node that may be powered down — the gateway stays healthy
(/ still answers) but proxied routes return 502 until Ollama is back.
solaria resolves via the host's Tailscale MagicDNS, which Docker's embedded
DNS forwards to.
Configuration
.env— gitignored, copy fromenv.example. HoldsTAILSCALE_BIND_IP(required for the mesh-only port bind) and optionalOLLAMA_URL.- No secrets anywhere in the stack.
Deploy (PIHA)
git pullon PIHA (~/homelab-codex-ws).cp services/llm-gateway/env.example services/llm-gateway/.env(defaults are correct for PIHA).-
docker compose -f services/llm-gateway/docker-compose.yml \ -f hosts/piha/runtime/llm-gateway/docker-compose.override.yml up -d --build - Verify:
services/llm-gateway/healthcheck.sh, then from any mesh node:curl http://piha:8080/→{"status":"gateway ok"}.
Tests
cd services/llm-gateway && pytest