homelab-codex-ws/hosts/piha/services.yaml

201 lines
7.3 KiB
YAML
Raw Normal View History

host: piha
services:
ha-diag-agent:
role: ha-diagnostic-agent
deployment_model: docker-compose
exposure: local-only
offline_required: false
depends_on:
local: []
external: [homeassistant]
config:
target_url: http://localhost:8123
location_tag: "ken"
events_dir: /opt/homelab/events/piha
runtime:
config_path: /opt/homelab/config/ha-diag-agent
data_path: /var/lib/ha-diag-agent
feat(node-agent): implement health monitor and safe cleanup policy scripts/monitor/health-monitor.sh (new): - Standalone bash health monitor: disk/RAM/CPU checks + docker container health - Per-node-type cleanup policy enforced: lte_node (chelsty-infra, chelsty-ha): NO cleanup, no docker ops sd_card (piha, saturn): dangling images + containers, rate-limited once/24h ai_node (solaria): dangling + containers + build cache, NEVER -a standard (vps): dangling + containers + build cache + CP filesystem rotation - VPS filesystem rotation: completed/failed actions >7d, deploy logs >30d, events >3d AND past observer checkpoint - Emits structured JSON events (node_health, disk_pressure, high_memory, high_cpu, containers_not_running, healthcheck_failed) services/node-agent/ (new): - Python daemon (node_agent.py): same policy as bash script, Docker SDK for container checks and cleanup, /proc for system metrics - Optional event shipping to VPS via rsync+SSH (VPS_EVENTS_HOST env var) - Dockerfile: python:3.11-slim + openssh-client + rsync + docker>=6.0 - docker-compose.yml: mounts docker socket, /opt/homelab, repo read-only observer.py: - Handle node_health: update node status + disk/mem/cpu metrics, clear disk_pressure - Handle disk_pressure: record severity on node, clear when healthy - Handle high_memory / high_cpu: record pressure level for correlation supervisor.py: - Add NO_DISK_CLEANUP_NODES = {chelsty-infra, chelsty-ha} - reconcile() step 3: generate disk_cleanup actions for nodes with high disk pressure - _generate_disk_cleanup_recommendation(): stable ID disk-cleanup-{node}, checks all active states, risk=guarded (operator approval required) executor.py: - Handle disk_cleanup action type via _execute_disk_cleanup() - Commands come from action payload; safety gate rejects any command touching /opt/homelab/data/, /opt/homelab/config/, /opt/homelab/state/, or rm -rf / hosts/*/services.yaml: - Rename stability-agent -> node-agent on piha, vps, solaria, chelsty-infra - Add node-agent to chelsty-ha (previously missing) - Add cleanup policy notes to LTE node comments Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 13:15:06 +02:00
node-agent:
role: node-stability-monitor
deployment_model: docker-compose
exposure: local-only
offline_required: true
depends_on:
local: []
external: []
runtime:
feat(node-agent): implement health monitor and safe cleanup policy scripts/monitor/health-monitor.sh (new): - Standalone bash health monitor: disk/RAM/CPU checks + docker container health - Per-node-type cleanup policy enforced: lte_node (chelsty-infra, chelsty-ha): NO cleanup, no docker ops sd_card (piha, saturn): dangling images + containers, rate-limited once/24h ai_node (solaria): dangling + containers + build cache, NEVER -a standard (vps): dangling + containers + build cache + CP filesystem rotation - VPS filesystem rotation: completed/failed actions >7d, deploy logs >30d, events >3d AND past observer checkpoint - Emits structured JSON events (node_health, disk_pressure, high_memory, high_cpu, containers_not_running, healthcheck_failed) services/node-agent/ (new): - Python daemon (node_agent.py): same policy as bash script, Docker SDK for container checks and cleanup, /proc for system metrics - Optional event shipping to VPS via rsync+SSH (VPS_EVENTS_HOST env var) - Dockerfile: python:3.11-slim + openssh-client + rsync + docker>=6.0 - docker-compose.yml: mounts docker socket, /opt/homelab, repo read-only observer.py: - Handle node_health: update node status + disk/mem/cpu metrics, clear disk_pressure - Handle disk_pressure: record severity on node, clear when healthy - Handle high_memory / high_cpu: record pressure level for correlation supervisor.py: - Add NO_DISK_CLEANUP_NODES = {chelsty-infra, chelsty-ha} - reconcile() step 3: generate disk_cleanup actions for nodes with high disk pressure - _generate_disk_cleanup_recommendation(): stable ID disk-cleanup-{node}, checks all active states, risk=guarded (operator approval required) executor.py: - Handle disk_cleanup action type via _execute_disk_cleanup() - Commands come from action payload; safety gate rejects any command touching /opt/homelab/data/, /opt/homelab/config/, /opt/homelab/state/, or rm -rf / hosts/*/services.yaml: - Rename stability-agent -> node-agent on piha, vps, solaria, chelsty-infra - Add node-agent to chelsty-ha (previously missing) - Add cleanup policy notes to LTE node comments Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 13:15:06 +02:00
config_path: /opt/homelab/config/node-agent
data_path: /opt/homelab/state
logs_path: /opt/homelab/events
brain-watchdog:
role: control-plane-watchdog
deployment_model: docker-compose
exposure: private
offline_required: false
depends_on:
local: []
external: [control-plane]
runtime:
config_path: /opt/homelab/config/brain-watchdog
vikunja:
role: task-management
deployment_model: docker-compose
exposure: private # public via npm reverse proxy at vikunja.okit.pl
offline_required: false
depends_on:
local: []
external: [forgejo] # OIDC identity provider
ports:
- name: http
container_port: 3456
protocol: tcp
runtime:
# secrets (.env) + OIDC config (config.yml) live alongside the compose file
config_path: services/vikunja
# data is in Docker named volumes: vikunja_vikunja_db, vikunja_vikunja_files
llm-gateway:
role: llm-router # FastAPI proxy -> Ollama @ SOLARIA :11434
deployment_model: docker-compose
exposure: private # Tailscale-only bind (TAILSCALE_BIND_IP); no public ingress
offline_required: false
depends_on:
local: []
external: [ollama] # SOLARIA may be powered down -> routes 502, health stays ok
ports:
- name: http
container_port: 8080
protocol: tcp
runtime:
# .env (TAILSCALE_BIND_IP) lives alongside the compose file; stateless, no data path
config_path: services/llm-gateway
feat(kb): faza 3 krok 5 — cykliczny ingest (systemd timer) + alerting documents-ingest-cyclic (jobs/documents-ingest/src/documents_ingest/cyclic_ingest.py): orkiestruje paperless_adapter -> chunk_embed -> summarize(--backend anthropic, claude-haiku-4-5) -> summarize(--embed-summaries) bez zmian w samych jobach. Ollama@SOLARIA (availability_target: medium) jest tolerowana offline: probe GET /api/tags przed obu etapami embed, brak -> pominięcie, nie fail (oba embed passy idempotentne, nadrobią się na kolejnym ticku). Czwarty etap (embed-summaries) dopisany ponad plan §7.1 (który wymieniał tylko 3 kroki) — bez niego nowe streszczenia miałyby embedding=NULL i byłyby niewidoczne dla cascade_query (bramka kroku 4, WHERE embedding IS NOT NULL); potwierdzone z Oskarem. Predykaty pass/fail każdego etapu 1:1 z exit-checkiem danego joba (chunks_errors, llm_errors, stats-balance itd.) — etapy izolowane, nie fail-fast (wcześniejszy fail nie blokuje kolejnych, tak jak joby izolują błędy per wiersz). Metryki .prom (atomowy zapis, last_success_timestamp trzymany z poprzedniego pliku przy failu) do /opt/homelab/state/node-exporter/kb-ingest.prom. 36 nowych testów (202/202 pakietu). systemd (jobs/documents-ingest/systemd/): pierwszy systemd-timer w repo — kb-ingest.timer (OnCalendar=*-*-* 03:30, Persistent=true, plan §7.1) + kb-ingest.service (host-level, User oskar, EnvironmentFile /opt/homelab/kb/.env) + kb-ingest-run.sh (log per-run do /opt/homelab/logs/kb-ingest/, konwencja repo). Instalacja i sekrety udokumentowane w README (Faza 3 krok 5) — instalacja na PIHA dopiero po merge. fleet-prometheus (rules/kb-ingest.yml): KbIngestStale (>172800s od last_success, critical) + KbEmbedBacklogGrowing (backlog>0 przez 72h, warning) — dostawa istniejącym torem brain-watchdog->Telegram, bez Alertmanagera (konwencja liveness.yml). node_exporter: owner_node vps -> per-host (service.yaml) + wpis + override (--collector.textfile.directory, bez nowego mountu — czyta przez istniejący /:/host:ro) + topology.yaml dla PIHA. Domyka pozycję z docs/backlog.md "stability-agent / node_exporter owner_node single, biegaja wielomiejscowo -> per-host" (połowę — node_exporter; stability-agent zostaje osobnym follow-upem) w ramach paczki B inwentaryzacji monitoringu dla PIHA. Test end-to-end na żywo na PIHA (2× --apply, po potwierdzeniu z Oskarem): pierwszy run złapał 5 dokumentów faktycznie nowych w Paperless (nieoczekiwane, niezwiązane z tym taskiem) -> 82 nowe chunki (2 ocr_junk), 5 nowych streszczeń, 5 embeddingów streszczeń, 0 błędów, metryki zapisane. Drugi run: pełna idempotencja, wszystko 0. ANTHROPIC_API_KEY dodany przez Oskara ręcznie do /opt/homelab/kb/.env (nigdy nie logowany/generowany). Co dalej: prawdziwa instalacja systemd (services.yaml już przygotowany, po merge), zdecydowanie czy stability-agent też idzie na per-host przy okazji. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 15:05:18 +02:00
node_exporter:
role: metrics-exporter
deployment_model: docker-compose
exposure: local-only
offline_required: true
depends_on:
local: []
external: []
runtime:
# textfile collector reads /opt/homelab/state/node-exporter (module 5 phase 3 step 5,
# kb/phases/kb-m5-faza3.md §7.2 — kb-ingest.prom) via the existing /:/host:ro
feat(kb): faza 3 krok 5 — cykliczny ingest (systemd timer) + alerting documents-ingest-cyclic (jobs/documents-ingest/src/documents_ingest/cyclic_ingest.py): orkiestruje paperless_adapter -> chunk_embed -> summarize(--backend anthropic, claude-haiku-4-5) -> summarize(--embed-summaries) bez zmian w samych jobach. Ollama@SOLARIA (availability_target: medium) jest tolerowana offline: probe GET /api/tags przed obu etapami embed, brak -> pominięcie, nie fail (oba embed passy idempotentne, nadrobią się na kolejnym ticku). Czwarty etap (embed-summaries) dopisany ponad plan §7.1 (który wymieniał tylko 3 kroki) — bez niego nowe streszczenia miałyby embedding=NULL i byłyby niewidoczne dla cascade_query (bramka kroku 4, WHERE embedding IS NOT NULL); potwierdzone z Oskarem. Predykaty pass/fail każdego etapu 1:1 z exit-checkiem danego joba (chunks_errors, llm_errors, stats-balance itd.) — etapy izolowane, nie fail-fast (wcześniejszy fail nie blokuje kolejnych, tak jak joby izolują błędy per wiersz). Metryki .prom (atomowy zapis, last_success_timestamp trzymany z poprzedniego pliku przy failu) do /opt/homelab/state/node-exporter/kb-ingest.prom. 36 nowych testów (202/202 pakietu). systemd (jobs/documents-ingest/systemd/): pierwszy systemd-timer w repo — kb-ingest.timer (OnCalendar=*-*-* 03:30, Persistent=true, plan §7.1) + kb-ingest.service (host-level, User oskar, EnvironmentFile /opt/homelab/kb/.env) + kb-ingest-run.sh (log per-run do /opt/homelab/logs/kb-ingest/, konwencja repo). Instalacja i sekrety udokumentowane w README (Faza 3 krok 5) — instalacja na PIHA dopiero po merge. fleet-prometheus (rules/kb-ingest.yml): KbIngestStale (>172800s od last_success, critical) + KbEmbedBacklogGrowing (backlog>0 przez 72h, warning) — dostawa istniejącym torem brain-watchdog->Telegram, bez Alertmanagera (konwencja liveness.yml). node_exporter: owner_node vps -> per-host (service.yaml) + wpis + override (--collector.textfile.directory, bez nowego mountu — czyta przez istniejący /:/host:ro) + topology.yaml dla PIHA. Domyka pozycję z docs/backlog.md "stability-agent / node_exporter owner_node single, biegaja wielomiejscowo -> per-host" (połowę — node_exporter; stability-agent zostaje osobnym follow-upem) w ramach paczki B inwentaryzacji monitoringu dla PIHA. Test end-to-end na żywo na PIHA (2× --apply, po potwierdzeniu z Oskarem): pierwszy run złapał 5 dokumentów faktycznie nowych w Paperless (nieoczekiwane, niezwiązane z tym taskiem) -> 82 nowe chunki (2 ocr_junk), 5 nowych streszczeń, 5 embeddingów streszczeń, 0 błędów, metryki zapisane. Drugi run: pełna idempotencja, wszystko 0. ANTHROPIC_API_KEY dodany przez Oskara ręcznie do /opt/homelab/kb/.env (nigdy nie logowany/generowany). Co dalej: prawdziwa instalacja systemd (services.yaml już przygotowany, po merge), zdecydowanie czy stability-agent też idzie na per-host przy okazji. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 15:05:18 +02:00
# mount, see hosts/piha/runtime/node_exporter/docker-compose.override.yml.
data_path: /opt/homelab/state/node-exporter
kb-postgres:
role: kb-database # KB spine: Postgres 16 + pgvector (always-on)
deployment_model: docker-compose
exposure: local-only # Tailscale-accessible; no public ingress
offline_required: false
depends_on:
local: []
external: []
ports:
- name: postgres
host_port: 5433
protocol: tcp
runtime:
config_path: /opt/homelab/config/kb-postgres
# data is in Docker named volume kb_postgres_data — must land on the NVMe
# (Docker data-root on /home); see hosts/piha/runtime/kb-postgres override.
feat(kb): add kb-query service skeleton (search API, no ingress yet) Module 5 phase 4 step 1 (docs/kb/modules/05-faza4-plan.md, §4): first user-facing HTTP entry point to the KB. FastAPI wrapping kb_retrieval.cascade_query/flat_query — GET /search (query_text -> embed via Ollama@SOLARIA -> cascade/flat -> envelope join -> JSON with per-source links) and GET /healthz. Search API only, no answer synthesis (phase 5) and no server-side dist filtering — the 0.45/0.55 colour thresholds are a frontend concern (plan §7, a later step). Hard startup invariant (plan §2 decision 2): refuses to start unless the configured EMBED_MODEL is present in both document_chunk.model and document_summary.embedding_model. Note the latter: document_summary.model is the LLM that *wrote* the summary (claude-haiku-4-5/gemma3:12b), not the embedder — checked live against kb-postgres@PIHA before writing this, see app/startup.py's docstring. Verified end-to-end with a live docker run: the invariant crash-loops on a mismatched EMBED_MODEL and passes through to a real /search hit against the live corpus with a correct model. Repo-only: no deploy, no npm/OIDC/DNS wiring (plan §8, later step), no local embed fallback (plan §5, later step) — Ollama@SOLARIA is called directly and a failure surfaces as 503, not a crash. Also: scripts/deploy/deploy.sh's gate now builds each service via `docker compose build` instead of a raw `docker build <svc_dir>`, so a service whose docker-compose.yml declares a repo-root build context (needed here to COPY packages/kb-retrieval/, the packages/ Dockerfile convention already documented in CLAUDE.md) resolves the same way in the gate as it does at real deploy time (deploy-node.sh's `docker compose ... up --build`). No behavior change for existing single-context services — verified against llm-gateway's compose file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-22 16:06:18 +02:00
feat(kb): aktywny fallback embeddingów SOLARIA→PIHA dla kb-query (faza 4 Krok 2) Ostatni krok fazy 4 KB (plan §2 Decyzja 2, §5): kb-query przestaje być martwe przez ~16 h/dobę, gdy SOLARIA (GPU) śpi — zapytania embeduje wtedy lokalna Ollama CPU na PIHA (wolniej: ~790 ms+ vs ~207 ms na GPU, ale działa). Nowy serwis services/ollama-piha (GitOps, owner_node: piha): - ollama/ollama:latest (arm64 natywnie), OLLAMA_KEEP_ALIVE=0 — model zwalnia RAM natychmiast po każdym wywołaniu (spike, nie rezydent; PIHA dzieli 8 GB z HA) - bind wyłącznie 127.0.0.1 + LAN_BIND_IP (192.168.31.5), nigdy 0.0.0.0/Tailscale - named volume ollama_piha_models (NVMe data-root) zamiast bind-mounta — obraz biega jako root w kontenerze i bind łamałby wzorzec uid PIHA (oskar=1004, kontenery uid 1000, setgid pi) - override hosts/piha/runtime/ollama-piha: mem_limit 2560m (wartość startowa z planu, do potwierdzenia kalibracją na żywo), świadomie bez mem_reservation - pull bge-m3 to jawny, ręczny krok deployu (README) — obraz nie ma modeli kb-query — maszyna stanów fallbacku (app/embed_router.py): - health-check SOLARII (GET /api/tags, timeout 1.5 s) z cache 30 s — zero sondowania per request; po powrocie SOLARII ruch wraca na GPU w ≤30 s - primary up → embed na SOLARII z twardym timeoutem 3 s; błąd W TRAKCIE zapytania = jednorazowe przełączenie (krok 3b planu): status down na 30 s i TO SAMO zapytanie leci na fallback — user nie widzi błędu SOLARII - primary down → embed prosto na ollama-piha (bez twardego timeoutu: CPU + zimny load modelu to legalnie pojedyncze sekundy) - 503 tylko gdy oba backendy padłe (lub fallback nieskonfigurowany) - inwariant modelu, druga połowa: każdy backend weryfikowany raz, leniwie przy pierwszym użyciu, że /api/tags zawiera EMBED_MODEL (bge-m3 — ta sama wartość co startowy check przeciw document_chunk.model/document_summary.embedding_model); niezgodność = ERROR log + 500, nigdy ciche liczenie dystansów między różnymi przestrzeniami embeddingów; leniwie, bo śpiąca SOLARIA nie może blokować startu serwisu - odpowiedź /search: nowe pole embed_backend ("solaria"|"piha") + sol_status wg realnego świata routera (UI już renderuje down jako "offline (fallback embed)"); log INFO backend=... elapsed_ms=... per zapytanie - /healthz: sol_status przez cache routera (spójny widok z routingiem) + fallback_status (żywa, tania sonda /api/tags) Konfiguracja spójnie przez env (compose + env.example + service.yaml + README): EMBED_PRIMARY_URL (zastępuje OLLAMA_URL), EMBED_FALLBACK_URL (pusty = brak fallbacku, zachowanie sprzed kroku 2), EMBED_{PRIMARY,FALLBACK}_NAME, EMBED_HEALTH_TTL_S/EMBED_HEALTH_TIMEOUT_S/EMBED_PRIMARY_TIMEOUT_S. Testy: 39 pass (14 nowych w test_embed_router.py: cache TTL, failover w trakcie zapytania, powrót po TTL, oba padłe, mismatch modelu na primary i fallbacku, tag "bge-m3:latest" vs "bge-m3"); docker build + smoke (importy + uvicorn do guardu KB_DSN) OK; compose config OK dla obu stacków. Deploy (Oskar, na PIHA z mastera po merge): cd ~/homelab-codex-ws && git pull # 1. ollama-piha cp services/ollama-piha/env.example services/ollama-piha/.env docker compose -f services/ollama-piha/docker-compose.yml \ -f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \ --env-file services/ollama-piha/.env up -d docker exec ollama-piha ollama pull bge-m3 # ręczny krok, obowiązkowy services/ollama-piha/healthcheck.sh # 2. kb-query (dopisać fallback do istniejącego .env) echo 'EMBED_FALLBACK_URL=http://192.168.31.5:11434' >> services/kb-query/.env docker compose -f services/kb-query/docker-compose.yml \ -f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build services/kb-query/healthcheck.sh # (deploy-node.sh też podniesie oba serwisy z hosts/piha/services.yaml, # ale pull bge-m3 i .env pozostają ręczne) Weryfikacja: testy A/B/C w services/kb-query/README.md (backend=solaria przy SOLARII online; backend=piha przy symulacji offline; powrót na GPU w ≤30 s). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 19:01:29 +02:00
ollama-piha:
role: embed-fallback # module 5 phase 4 (plan §2 D2/§5): local CPU bge-m3 for kb-query while SOLARIA sleeps
deployment_model: docker-compose
exposure: private # 127.0.0.1 + LAN bind (LAN_BIND_IP) only; no public/Tailscale exposure
offline_required: false
depends_on:
local: []
external: []
ports:
- name: http
container_port: 11434
host_port: 11434
protocol: tcp
runtime:
# .env (LAN_BIND_IP) lives alongside the compose file; models live in the
# Docker named volume ollama_piha_models (NVMe data-root, like kb-postgres).
# Deploy note: `docker exec ollama-piha ollama pull bge-m3` is a manual
# post-deploy step — the image ships no models.
config_path: services/ollama-piha
feat(kb): add kb-query service skeleton (search API, no ingress yet) Module 5 phase 4 step 1 (docs/kb/modules/05-faza4-plan.md, §4): first user-facing HTTP entry point to the KB. FastAPI wrapping kb_retrieval.cascade_query/flat_query — GET /search (query_text -> embed via Ollama@SOLARIA -> cascade/flat -> envelope join -> JSON with per-source links) and GET /healthz. Search API only, no answer synthesis (phase 5) and no server-side dist filtering — the 0.45/0.55 colour thresholds are a frontend concern (plan §7, a later step). Hard startup invariant (plan §2 decision 2): refuses to start unless the configured EMBED_MODEL is present in both document_chunk.model and document_summary.embedding_model. Note the latter: document_summary.model is the LLM that *wrote* the summary (claude-haiku-4-5/gemma3:12b), not the embedder — checked live against kb-postgres@PIHA before writing this, see app/startup.py's docstring. Verified end-to-end with a live docker run: the invariant crash-loops on a mismatched EMBED_MODEL and passes through to a real /search hit against the live corpus with a correct model. Repo-only: no deploy, no npm/OIDC/DNS wiring (plan §8, later step), no local embed fallback (plan §5, later step) — Ollama@SOLARIA is called directly and a failure surfaces as 503, not a crash. Also: scripts/deploy/deploy.sh's gate now builds each service via `docker compose build` instead of a raw `docker build <svc_dir>`, so a service whose docker-compose.yml declares a repo-root build context (needed here to COPY packages/kb-retrieval/, the packages/ Dockerfile convention already documented in CLAUDE.md) resolves the same way in the gate as it does at real deploy time (deploy-node.sh's `docker compose ... up --build`). No behavior change for existing single-context services — verified against llm-gateway's compose file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-22 16:06:18 +02:00
kb-query:
role: kb-search-api # module 5 phase 4: FastAPI wrapper over kb_retrieval cascade/flat query
deployment_model: docker-compose
exposure: private # LAN bind (LAN_BIND_IP); npm@PIHA vhost + OIDC is a later step
offline_required: false
depends_on:
feat(kb): aktywny fallback embeddingów SOLARIA→PIHA dla kb-query (faza 4 Krok 2) Ostatni krok fazy 4 KB (plan §2 Decyzja 2, §5): kb-query przestaje być martwe przez ~16 h/dobę, gdy SOLARIA (GPU) śpi — zapytania embeduje wtedy lokalna Ollama CPU na PIHA (wolniej: ~790 ms+ vs ~207 ms na GPU, ale działa). Nowy serwis services/ollama-piha (GitOps, owner_node: piha): - ollama/ollama:latest (arm64 natywnie), OLLAMA_KEEP_ALIVE=0 — model zwalnia RAM natychmiast po każdym wywołaniu (spike, nie rezydent; PIHA dzieli 8 GB z HA) - bind wyłącznie 127.0.0.1 + LAN_BIND_IP (192.168.31.5), nigdy 0.0.0.0/Tailscale - named volume ollama_piha_models (NVMe data-root) zamiast bind-mounta — obraz biega jako root w kontenerze i bind łamałby wzorzec uid PIHA (oskar=1004, kontenery uid 1000, setgid pi) - override hosts/piha/runtime/ollama-piha: mem_limit 2560m (wartość startowa z planu, do potwierdzenia kalibracją na żywo), świadomie bez mem_reservation - pull bge-m3 to jawny, ręczny krok deployu (README) — obraz nie ma modeli kb-query — maszyna stanów fallbacku (app/embed_router.py): - health-check SOLARII (GET /api/tags, timeout 1.5 s) z cache 30 s — zero sondowania per request; po powrocie SOLARII ruch wraca na GPU w ≤30 s - primary up → embed na SOLARII z twardym timeoutem 3 s; błąd W TRAKCIE zapytania = jednorazowe przełączenie (krok 3b planu): status down na 30 s i TO SAMO zapytanie leci na fallback — user nie widzi błędu SOLARII - primary down → embed prosto na ollama-piha (bez twardego timeoutu: CPU + zimny load modelu to legalnie pojedyncze sekundy) - 503 tylko gdy oba backendy padłe (lub fallback nieskonfigurowany) - inwariant modelu, druga połowa: każdy backend weryfikowany raz, leniwie przy pierwszym użyciu, że /api/tags zawiera EMBED_MODEL (bge-m3 — ta sama wartość co startowy check przeciw document_chunk.model/document_summary.embedding_model); niezgodność = ERROR log + 500, nigdy ciche liczenie dystansów między różnymi przestrzeniami embeddingów; leniwie, bo śpiąca SOLARIA nie może blokować startu serwisu - odpowiedź /search: nowe pole embed_backend ("solaria"|"piha") + sol_status wg realnego świata routera (UI już renderuje down jako "offline (fallback embed)"); log INFO backend=... elapsed_ms=... per zapytanie - /healthz: sol_status przez cache routera (spójny widok z routingiem) + fallback_status (żywa, tania sonda /api/tags) Konfiguracja spójnie przez env (compose + env.example + service.yaml + README): EMBED_PRIMARY_URL (zastępuje OLLAMA_URL), EMBED_FALLBACK_URL (pusty = brak fallbacku, zachowanie sprzed kroku 2), EMBED_{PRIMARY,FALLBACK}_NAME, EMBED_HEALTH_TTL_S/EMBED_HEALTH_TIMEOUT_S/EMBED_PRIMARY_TIMEOUT_S. Testy: 39 pass (14 nowych w test_embed_router.py: cache TTL, failover w trakcie zapytania, powrót po TTL, oba padłe, mismatch modelu na primary i fallbacku, tag "bge-m3:latest" vs "bge-m3"); docker build + smoke (importy + uvicorn do guardu KB_DSN) OK; compose config OK dla obu stacków. Deploy (Oskar, na PIHA z mastera po merge): cd ~/homelab-codex-ws && git pull # 1. ollama-piha cp services/ollama-piha/env.example services/ollama-piha/.env docker compose -f services/ollama-piha/docker-compose.yml \ -f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \ --env-file services/ollama-piha/.env up -d docker exec ollama-piha ollama pull bge-m3 # ręczny krok, obowiązkowy services/ollama-piha/healthcheck.sh # 2. kb-query (dopisać fallback do istniejącego .env) echo 'EMBED_FALLBACK_URL=http://192.168.31.5:11434' >> services/kb-query/.env docker compose -f services/kb-query/docker-compose.yml \ -f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build services/kb-query/healthcheck.sh # (deploy-node.sh też podniesie oba serwisy z hosts/piha/services.yaml, # ale pull bge-m3 i .env pozostają ręczne) Weryfikacja: testy A/B/C w services/kb-query/README.md (backend=solaria przy SOLARII online; backend=piha przy symulacji offline; powrót na GPU w ≤30 s). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 19:01:29 +02:00
# ollama-piha is a soft local dependency: the embed fallback while SOLARIA
# sleeps — kb-query starts and serves /healthz without it, /search degrades
# to 503 only when BOTH embed backends are unreachable.
local: [kb-postgres, ollama-piha]
external: [ollama] # primary embed backend @ SOLARIA; may be offline -> fallback to ollama-piha
feat(kb): add kb-query service skeleton (search API, no ingress yet) Module 5 phase 4 step 1 (docs/kb/modules/05-faza4-plan.md, §4): first user-facing HTTP entry point to the KB. FastAPI wrapping kb_retrieval.cascade_query/flat_query — GET /search (query_text -> embed via Ollama@SOLARIA -> cascade/flat -> envelope join -> JSON with per-source links) and GET /healthz. Search API only, no answer synthesis (phase 5) and no server-side dist filtering — the 0.45/0.55 colour thresholds are a frontend concern (plan §7, a later step). Hard startup invariant (plan §2 decision 2): refuses to start unless the configured EMBED_MODEL is present in both document_chunk.model and document_summary.embedding_model. Note the latter: document_summary.model is the LLM that *wrote* the summary (claude-haiku-4-5/gemma3:12b), not the embedder — checked live against kb-postgres@PIHA before writing this, see app/startup.py's docstring. Verified end-to-end with a live docker run: the invariant crash-loops on a mismatched EMBED_MODEL and passes through to a real /search hit against the live corpus with a correct model. Repo-only: no deploy, no npm/OIDC/DNS wiring (plan §8, later step), no local embed fallback (plan §5, later step) — Ollama@SOLARIA is called directly and a failure surfaces as 503, not a crash. Also: scripts/deploy/deploy.sh's gate now builds each service via `docker compose build` instead of a raw `docker build <svc_dir>`, so a service whose docker-compose.yml declares a repo-root build context (needed here to COPY packages/kb-retrieval/, the packages/ Dockerfile convention already documented in CLAUDE.md) resolves the same way in the gate as it does at real deploy time (deploy-node.sh's `docker compose ... up --build`). No behavior change for existing single-context services — verified against llm-gateway's compose file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-22 16:06:18 +02:00
ports:
- name: http
container_port: 8080
host_port: 8230
protocol: tcp
runtime:
# .env (KB_DSN, LAN_BIND_IP) lives alongside the compose file; stateless, no data path
config_path: services/kb-query
narty27:
role: static-html-host # single self-contained viz.html (narty 2027 / Saalbach KB export)
deployment_model: docker-compose
exposure: private # LAN only; no npm vhost, no public ingress
offline_required: false
depends_on:
local: []
external: []
ports:
- name: http
container_port: 80
host_port: 8240
protocol: tcp
runtime:
# No config and no secrets. Content is PERSONAL and deliberately outside
# the repo — it lives only in the Docker named volume
# narty27_narty27_content, refreshed from SOLARIA (kb/runbooks/narty27-deploy.md).
# No backup job, no /opt/homelab/data bind.
config_path: services/narty27
kb-site:
role: static-html-host # public KB slice generated by scripts/kb/gen_pages.py
deployment_model: docker-compose
exposure: public # via npm@PIHA vhost kb.okit.pl; container binds LAN :8250
offline_required: false
depends_on:
local: []
external: []
ports:
- name: http
container_port: 80
host_port: 8250
protocol: tcp
runtime:
# No config and no secrets. Content is a pure artifact of the repo: it
# lives only in the Docker named volume kb-site_kb-site_content and is
# refreshed by regenerating from kb/ (kb/runbooks/kb-site-deploy.md).
# No backup job, no /opt/homelab/data bind.
config_path: services/kb-site
# --- Known unmanaged containers on piha (recon B5/B6, 2026-07-27) ----------
# ~28 running containers have no entry above and are deliberately NOT being
# added piecemeal — bringing them under desired state is a later stage
# (recon open question 5). The set: agent-system ×4 (webui,
# runtime-materializer, telegram-bot, redis — these STAY, decision
# 2026-07-28), stability-agent, paperless ×3, immich ×4, forgejo ×3,
# zigbee2mqtt, audiobookshelf, grafana, wikijs ×2, code-server, portainer,
# prometheus, homepage, actual-budget, mqtt-exporter, owntracks ×3,
# vaultwarden, pihole-exporter (unlabelled), fail2ban-exporter,
# nginxproxymanager — plus the HOST systemd mosquitto (OS package, no repo
# definition at all, recon C8). When adding any of these later, match the
# entry key to the world-state service key (supervisor does exact
# <host>/<name> matching) or it will drift as missing_service.