---
okf: "0.1"
type: service
visibility: private
status: active
updated: 2026-07-03
links: []
---

# llm-gateway

Small FastAPI router in front of Ollama on **SOLARIA**. Runs on the **PIHA**
node, bound to PIHA's Tailscale IP only (`http://piha:8080` on the mesh; no
public ingress). Stateless — no volumes, no secrets.

Origin: migrated 2026-07-03 from an unmanaged shadow deployment at
`/opt/llm-gateway` on PIHA into GitOps. The old copy is kept on disk as
rollback until explicitly retired.

## Endpoints

| Endpoint | Method | Model | Purpose |
|---|---|---|---|
| `/` | GET | — | Health: `{"status":"gateway ok"}` |
| `/api/chat` | POST | `deepcoder:14b` | General chat completion |
| `/api/code` | POST | `deepseek-coder:latest` | Code/command generation ("return only code" system prompt) |

Request body for both POST routes: `{"prompt": "...", "stream": false}`.

`/api/code` quirk (by design): a single-line prompt whose first word is not a
natural-language starter (write/create/explain/…) is treated as an
already-formed shell command and echoed back verbatim as `response`.

## Upstream dependency

Calls Ollama at `http://solaria:11434/api/generate` (override via `OLLAMA_URL`).
SOLARIA is a GPU node that may be powered down — the gateway stays healthy
(`/` still answers) but proxied routes return **502** until Ollama is back.
`solaria` resolves via the host's Tailscale MagicDNS, which Docker's embedded
DNS forwards to.

## Configuration

- `.env` — **gitignored**, copy from `env.example`. Holds `TAILSCALE_BIND_IP`
  (required for the mesh-only port bind) and optional `OLLAMA_URL`.
- No secrets anywhere in the stack.

## Deploy (PIHA)

1. `git pull` on PIHA (`~/homelab-codex-ws`).
2. `cp services/llm-gateway/env.example services/llm-gateway/.env` (defaults
   are correct for PIHA).
3. ```
   docker compose -f services/llm-gateway/docker-compose.yml \
     -f hosts/piha/runtime/llm-gateway/docker-compose.override.yml up -d --build
   ```
4. Verify: `services/llm-gateway/healthcheck.sh`, then from any mesh node:
   `curl http://piha:8080/` → `{"status":"gateway ok"}`.

## Tests

```
cd services/llm-gateway && pytest
```