homelab-codex-ws/kb/services/llm-gateway.md
oskar a1a57b04bb feat(kb): przenosiny type=service do kb/services/ (13 plikow, bez SPLIT)
Serwisy o jednorodnej tresci (bez wydzielonej sekcji deploy/runbook):
kb-postgres, stability-agent, llm-gateway, vikunja, zigbee2mqtt, npm,
forgejo, ollama, ir-ac-ha-integration, chelsty-stability-agent.

Deprecated: mosquitto (NOT DEPLOYED/LEGACY), home-assistant-ken-legacy
(archiwum importu), joplin (stub z 2026-04-15, brak katalogu services/joplin/).

git mv + frontmatter, tresc nietknieta.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 16:58:04 +02:00

2.1 KiB

okf type visibility status updated links
0.1 service private active 2026-07-03

llm-gateway

Small FastAPI router in front of Ollama on SOLARIA. Runs on the PIHA node, bound to PIHA's Tailscale IP only (http://piha:8080 on the mesh; no public ingress). Stateless — no volumes, no secrets.

Origin: migrated 2026-07-03 from an unmanaged shadow deployment at /opt/llm-gateway on PIHA into GitOps. The old copy is kept on disk as rollback until explicitly retired.

Endpoints

Endpoint Method Model Purpose
/ GET Health: {"status":"gateway ok"}
/api/chat POST deepcoder:14b General chat completion
/api/code POST deepseek-coder:latest Code/command generation ("return only code" system prompt)

Request body for both POST routes: {"prompt": "...", "stream": false}.

/api/code quirk (by design): a single-line prompt whose first word is not a natural-language starter (write/create/explain/…) is treated as an already-formed shell command and echoed back verbatim as response.

Upstream dependency

Calls Ollama at http://solaria:11434/api/generate (override via OLLAMA_URL). SOLARIA is a GPU node that may be powered down — the gateway stays healthy (/ still answers) but proxied routes return 502 until Ollama is back. solaria resolves via the host's Tailscale MagicDNS, which Docker's embedded DNS forwards to.

Configuration

  • .envgitignored, copy from env.example. Holds TAILSCALE_BIND_IP (required for the mesh-only port bind) and optional OLLAMA_URL.
  • No secrets anywhere in the stack.

Deploy (PIHA)

  1. git pull on PIHA (~/homelab-codex-ws).
  2. cp services/llm-gateway/env.example services/llm-gateway/.env (defaults are correct for PIHA).
  3. docker compose -f services/llm-gateway/docker-compose.yml \
      -f hosts/piha/runtime/llm-gateway/docker-compose.override.yml up -d --build
    
  4. Verify: services/llm-gateway/healthcheck.sh, then from any mesh node: curl http://piha:8080/{"status":"gateway ok"}.

Tests

cd services/llm-gateway && pytest