Stack ai-cluster (openclaw, codex/planner/service-ops workery, redis, mosquitto)
biega na VPS spoza GitOps, z /home/dockeruser/docker/ai-cluster/. Branch
feat/vps-service-migration (862c04a) probowal wciagnac go do GitOps *w miejscu*,
na VPS. Ten etap pomijamy: workloady sa compute'owe, VPS ma 4 GiB bez swapu i rolę
ingressu, SOLARIA ma 64 GiB i GPU. Migrujemy od razu na SOLARIA, VPS zostaje samym
NPM-em proxujacym po Tailscale.
Zero deployu. Kontenery na VPS nietkniete — recon byl read-only (katalog nalezy do
dockeruser, odczyt przez efemeryczny kontener z mountem :ro, bez dotykania
dzialajacych kontenerow). Plan przelaczenia: services/ai-cluster/CUTOVER.md.
Zrodla wciagniete do services/ai-cluster/src/ (openclaw, worker, mosquitto).
Pominiete: .env i mosquitto/passwd (sekrety), start-codex.sh* (smiec spoza stacku),
__pycache__.
Ustalenia z reconu, ktore zmienily manifest z 862c04a:
- codex-worker/ na VPS to NIE osobny build context ani duplikat worker/ — to
martwy 609-bajtowy prototyp na `requests` (biblioteka nie wystepuje w zadnym
requirements.txt), bez Dockerfile'a i bez referencji w compose. Realny
codex-worker buduje sie z ./worker (Dockerfile default). Nie przenoszony.
- image: ai-cluster-* -> build: z src/openclaw i src/worker (3 Dockerfile'e nad
jednym kontekstem: default/planner/service-ops).
- openclaw: usuniete publish 0.0.0.0:8000 -> "100.100.231.104:8000:8000";
usunieta siec npm_default (nie istnieje na SOLARIA).
- mosquitto: bind 100.95.58.48 -> 100.100.231.104.
- mem_limity urealnione (256m workery/openclaw, 64m redis/mosquitto) i przeniesione
do hosts/solaria/runtime/. Zmierzone RSS na VPS: workery 7-10 MiB, openclaw
24.6 MiB, redis 4.8, mosquitto 4.2 — limity to zapas ~10x, nie reakcja na presje.
- healthcheck zostaje na python/urllib: obraz openclaw (python:3.12-slim) nie ma
ani wget, ani curl — sprawdzone na zywym obrazie.
- GATEWAY_BASE_URL http://piha:8080 -> http://100.108.208.3:8080. Tailscale DNS
jest WYLACZONY na SOLARIA (`tailscale dns status` -> disabled), wiec zadna nazwa
MagicDNS sie nie rozwiazuje. Na VPS bare `piha` tez nie rozwiazuje sie (rc=2),
czyli ten default byl martwy juz tam.
- telegram-bot nie przenoszony (na VPS nie biega zaden kontener). telegram_bot.py
zostaje w src/, bo openclaw/Dockerfile go COPY-uje — bez niego build sie wywala.
- sekrety wylacznie przez env_file /opt/homelab/config/ai-cluster/.env, zero
interpolacji ${VAR:?} — stack nie zalezy od cwd deployu. service-ops-worker
montuje ten katalog pod ta sama sciezka absolutna, bo compose waliduje env_file
na etapie parsowania i inaczej `docker compose ps` w tym kontenerze padnie.
- AGENT_ID zostaja `vps-*`: to cele routingu MQTT, nie hostnamey. Zmiana nazw
lamie producentow adresujacych konkretnego agenta — osobne, skoordynowane zadanie.
Konsumenci mosquitto do aktualizacji: w repo BRAK. chelsty (z2m, frigate,
stability-agent) uzywaja wlasnego, host-networkowego brokera i musza zostac
offline-first. Lista i sposob potwierdzenia klientow spoza repo — CUTOVER.md krok 5.
DoD (docker build + smoke + pytest) swiadomie odroczone: zadanie zabrania budowania
obrazow. Pokrywaja to kroki 1-3 CUTOVER.md, do wykonania w sesji cutoveru.
Zweryfikowane tutaj: `docker compose config` z overrideem (rc=0, bindy host_ip,
mem_limity, merge volumes), skladnia YAML, `bash -n healthcheck.sh`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Driver fixed 2026-07-16 (nvidia-driver-595-open from distro repo, old
graphics-drivers PPA for jammy disabled) — RTX 4070 Ti SUPER 16GB, CUDA
13.2, nvidia-container-toolkit already present from the 07-15 cutover
prerequisite install. Uncomments deploy.resources.reservations (nvidia
gpu) in services/ollama/docker-compose.yml, restoring clean formatting.
Docs close out the loose ends from the 07-15 declarative cutover:
- ollama-solaria-cutover runbook gets a "Wykonanie" section documenting
what actually happened (mv instead of rsync for the model store, the
missing nvidia-container-toolkit prerequisite, the driver-missing
discovery, CPU-only cutover, then the 07-16 GPU fix) plus a note on
the container disappearing after the 07-15 evening reboot (one-off,
boots fine now, root cause not established).
- hosts/solaria/services.yaml: ollama role comment now reflects actual
GPU-backed state instead of the previously-aspirational wording.
- hosts/solaria/README.md: drop stale Open WebUI mention (not in repo).
- docs/backlog.md: close the NVIDIA driver item; leave two follow-ups
(Ollama call batching before the mail phase, UNIQUE(envelope_id,
chunk_index) schema change for multi-model embeddings at phase 3).
- jobs/documents-ingest/README.md: timing section gets a GPU placeholder
line next to the existing 0.79s/chunk CPU baseline, to be filled in
after the live GPU benchmark.
Live recreate + GPU-vs-CPU embedding benchmark deliberately left for the
operator to run from the main checkout after merge, per worktree-aware
discipline — this worktree only owns the declarative fix and the docs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ollama's service.yaml/docker-compose.yml declared owner_node: solaria but
was never added to hosts/solaria/services.yaml, the manifest deploy-node.sh
actually reads — so it stayed running manually/natively instead of via the
declarative pipeline. Add the missing entry.
Also switch the port publish from 0.0.0.0 to loopback + TAILSCALE_BIND_IP
(same convention as llm-gateway@PIHA), since this is a private service and
llm-gateway@PIHA is the only consumer beyond the host itself.
Live cutover (relocating the existing native model store, disabling the
systemd unit, bringing up the container, verifying bge-m3 embeddings +
GPU use) is documented in docs/infra/ollama-solaria-cutover-2026-07-15.md
but not executed here — no SSH access to SOLARIA from this worktree.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Oskar Kapala