feat(kb): SPLIT service+runbook — 10 serwisow -> 20 dokumentow
Wzorzec mechaniczny: sekcje deploy/verify/install/testy wycinane do kb/runbooks/<serwis>-*.md, reszta zostaje dokumentem type: service. Wzajemne `links` w obie strony. Tresc sekcji nietknieta — przenoszone doslownie, dodany wylacznie naglowek H1 nowego runbooka. kb-query, paperless-worker, planner-agent, ha-diag-agent, ollama-piha, narty27, home-assistant, ha-mcp, job-gmail-header-backfill, job-mail-body-ingest. Weryfikacja: dla kazdego pliku multizbior niepustych linii (main + runbook) == oryginal z HEAD. Zero zgubionych, zero dodanych. Recon szacowal 13 splitow service+runbook; faktycznie 2-typowych jest 10, pozostale 5 (paperless, nextcloud, gokapi, fleet-prometheus, deploy-runner) sa 3-typowe i ida osobno jako splity wielotypowe. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
cb3fa95801
commit
3292ab54e2
49
kb/runbooks/gmail-header-backfill-run.md
Normal file
49
kb/runbooks/gmail-header-backfill-run.md
Normal file
|
|
@ -0,0 +1,49 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-14
|
||||||
|
links:
|
||||||
|
- ../services/job-gmail-header-backfill.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# gmail-header-backfill — uruchomienie i testy
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Dry run (default) — parse and count only, no DB writes:
|
||||||
|
gmail-header-backfill --dsn postgresql://kb:<pw>@localhost:5433/kb --limit 100
|
||||||
|
|
||||||
|
# Real run — apply the UPDATE for this slice:
|
||||||
|
gmail-header-backfill --dsn ... --limit 1000 --offset 0 --apply
|
||||||
|
|
||||||
|
# Next slice — offset is stable/deterministic (ORDER BY id), independent of
|
||||||
|
# how many rows in earlier slices were already backfilled:
|
||||||
|
gmail-header-backfill --dsn ... --limit 1000 --offset 1000 --apply
|
||||||
|
```
|
||||||
|
|
||||||
|
DSN can also come from the `KB_DSN` env var instead of `--dsn`.
|
||||||
|
|
||||||
|
`--limit`/`--offset` exist so the full 225 030-row backfill can be run in
|
||||||
|
verifiable partitions instead of one long unattended run (plan §5.3) — start
|
||||||
|
small (`--limit 100`), check the result in the DB, then widen.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -e jobs/gmail-header-backfill/
|
||||||
|
cd jobs/gmail-header-backfill && pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
Pure unit tests, no DB or filesystem outside `tmp_path`/synthetic `.eml`
|
||||||
|
bytes — `run()` is tested by monkeypatching `asyncpg.connect` with an
|
||||||
|
in-memory fake connection. Covers: header parsing (multi-address `To`/`Cc`,
|
||||||
|
quoted display names with commas, multiple `Delivered-To` occurrences, RFC
|
||||||
|
2047 encoded-words including Polish diacritics, malformed encoded-words that
|
||||||
|
must not raise, missing/multiple `From`, `date_raw` preserving literal text
|
||||||
|
vs. `Date` header reformatting), idempotency (rows already carrying a
|
||||||
|
`headers` entity are skipped and never re-appended), batch flushing, and
|
||||||
|
`--limit`/`--offset` query shape.
|
||||||
|
|
||||||
64
kb/runbooks/ha-diag-agent-runbook.md
Normal file
64
kb/runbooks/ha-diag-agent-runbook.md
Normal file
|
|
@ -0,0 +1,64 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-06-11
|
||||||
|
links:
|
||||||
|
- ../services/ha-diag-agent.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# ha-diag-agent — deployment i testy
|
||||||
|
|
||||||
|
## First-time deployment
|
||||||
|
|
||||||
|
See **[DEPLOY.md](DEPLOY.md)** for the full procedure: HA token creation,
|
||||||
|
per-host `.env` config, deploy commands, verification steps, 48h shadow-mode
|
||||||
|
observation, and rollback.
|
||||||
|
|
||||||
|
**Shadow mode** (`HA_DIAG_SHADOW_MODE`, default `true` on the control-plane):
|
||||||
|
`ha_websocket_dead` events are downgraded to `alert_only` with a `[SHADOW MODE]`
|
||||||
|
note instead of queuing an automatic `container_restart`. Set to `false` in
|
||||||
|
`/opt/homelab/config/control-plane/.env` on the VPS when ready for live actions.
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Create config on target node
|
||||||
|
ssh oskar@<node-ip>
|
||||||
|
mkdir -p /opt/homelab/config/ha-diag-agent /var/lib/ha-diag-agent
|
||||||
|
cat > /opt/homelab/config/ha-diag-agent/.env << 'EOF'
|
||||||
|
HA_URL=http://homeassistant.local:8123 # or http://100.70.180.90:8123 for chelsty-infra
|
||||||
|
HA_TOKEN=<long-lived-token>
|
||||||
|
NODE_NAME=piha # or chelsty-infra
|
||||||
|
LOCATION_TAG=ken # or chelsty
|
||||||
|
CHECK_INTERVAL=60
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# 2. Deploy
|
||||||
|
scripts/deploy/deploy.sh --service ha-diag-agent
|
||||||
|
|
||||||
|
# 3. Verify
|
||||||
|
docker ps --filter name=ha-diag-agent
|
||||||
|
docker exec ha-diag-agent python -c "import urllib.request; print(urllib.request.urlopen('http://localhost:8087/health', timeout=5).read().decode())"
|
||||||
|
```
|
||||||
|
|
||||||
|
### chelsty-infra note
|
||||||
|
|
||||||
|
`chelsty-infra` runs docker-compose v1 (1.29.2). Use `docker-compose` (hyphenated):
|
||||||
|
```bash
|
||||||
|
docker-compose -f docker-compose.yml up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
### HA long-lived token
|
||||||
|
|
||||||
|
In HA UI: Profile → Long-Lived Access Tokens → Create token.
|
||||||
|
|
||||||
|
## Running Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd services/ha-diag-agent
|
||||||
|
pip install -e ".[dev]"
|
||||||
|
pytest tests/ -v
|
||||||
|
```
|
||||||
|
|
||||||
97
kb/runbooks/ha-mcp-install.md
Normal file
97
kb/runbooks/ha-mcp-install.md
Normal file
|
|
@ -0,0 +1,97 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-30
|
||||||
|
links:
|
||||||
|
- ../services/ha-mcp.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# ha-mcp — instalacja i rejestracja
|
||||||
|
|
||||||
|
## Install
|
||||||
|
|
||||||
|
The `mcp` SDK (and its pydantic/anyio/httpx dependency tree) is not packaged
|
||||||
|
for Debian and is not needed by anything else in this repo, so it goes into a
|
||||||
|
venv rather than into the system Python:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -m venv --system-site-packages services/ha-mcp/.venv
|
||||||
|
services/ha-mcp/.venv/bin/pip install mcp pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
`--system-site-packages` is deliberate: `requests`, `PyYAML` and
|
||||||
|
`websocket-client` are already installed system-wide and used by
|
||||||
|
`scripts/ha/lib/*`; the venv reuses those exact versions instead of shadowing
|
||||||
|
them with a second copy. `.venv/` is already covered by the repo `.gitignore`.
|
||||||
|
|
||||||
|
`pip install --break-system-packages mcp` also works and is one line shorter,
|
||||||
|
but it writes into the system interpreter that runs every deploy script on
|
||||||
|
this workstation — a venv keeps a 40-package dependency tree out of that blast
|
||||||
|
radius for a tool only Claude Code uses. Use the venv.
|
||||||
|
|
||||||
|
`run.sh` prefers `services/ha-mcp/.venv/bin/python` and falls back to the
|
||||||
|
system `python3`; if neither can import `mcp` it exits with one actionable
|
||||||
|
line on stderr rather than a traceback.
|
||||||
|
|
||||||
|
## Registration in Claude Code
|
||||||
|
|
||||||
|
`.mcp.json` in the repo root (project scope — shared with anyone who checks
|
||||||
|
out this repo):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"mcpServers": {
|
||||||
|
"ha": {
|
||||||
|
"command": "./services/ha-mcp/run.sh",
|
||||||
|
"args": [],
|
||||||
|
"env": {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The command path is relative, so it resolves in any checkout (main or task
|
||||||
|
worktree) as long as Claude Code is started from the repo root. Start CC
|
||||||
|
there; on first run it asks whether to trust the project's MCP servers. Check
|
||||||
|
with `/mcp` — the server appears as `ha`, its tools as `mcp__ha__<tool>`.
|
||||||
|
Which repo the server reads is derived from its own location; override with
|
||||||
|
the `HA_MCP_REPO` environment variable if you ever need to point one checkout
|
||||||
|
at another's config.
|
||||||
|
|
||||||
|
## Standalone
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./services/ha-mcp/run.sh # stdio server — speaks JSON-RPC on stdout
|
||||||
|
services/ha-mcp/tests/run.sh # offline test suite (no HA, no token)
|
||||||
|
services/ha-mcp/.venv/bin/python services/ha-mcp/tests/smoke_live.py [instance] [entity_id]
|
||||||
|
```
|
||||||
|
|
||||||
|
`smoke_live.py` is the only thing here that touches the network (GET only). A
|
||||||
|
one-off tool call without a client is easiest through the package:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PYTHONPATH=services/ha-mcp/src services/ha-mcp/.venv/bin/python -c "
|
||||||
|
from ha_mcp import tools; from ha_mcp.backend import LiveBackend
|
||||||
|
from ha_mcp.config import get_instance, repo_root
|
||||||
|
n, c = get_instance('ken', repo_root()); print(tools.instance_status(LiveBackend(n, c, repo_root())))"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
services/ha-mcp/tests/run.sh # 42 tests, offline
|
||||||
|
```
|
||||||
|
|
||||||
|
Offline in the same sense as `scripts/ha/tests/*`: no network, no HA
|
||||||
|
instance, no token. Small hand-made fixtures under `tests/fixtures/` (shaped
|
||||||
|
like real `/api/states` and WebSocket registry payloads) cover the tool logic;
|
||||||
|
the repo's own `config/ken/automations/` and newest `fixtures/ken-states-*.yaml`
|
||||||
|
cover the repo-backed and at-scale paths (ranking over ~1650 real entities
|
||||||
|
behaves differently from ranking over ten).
|
||||||
|
|
||||||
|
Live smoke (read-only, run against `ken` on 2026-07-30):
|
||||||
|
`instance_status` → HA 2026.7.2, 1647 entities, 377 unavailable, 115
|
||||||
|
automations, 13 areas; `get_state("sensor.thsalon_temperature")` → `24.5 °C`.
|
||||||
|
|
||||||
|
|
@ -1,29 +1,14 @@
|
||||||
# home-assistant (configs-as-code)
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-22
|
||||||
|
links:
|
||||||
|
- ../services/home-assistant.md
|
||||||
|
---
|
||||||
|
|
||||||
**Status: phase 1 (partial).** Read-only import tooling, plus a deploy
|
# home-assistant — import, deploy, testy
|
||||||
(repo -> instance) write path for the `api` adapter's automations/scripts/
|
|
||||||
scenes scope only (`scripts/ha/deploy.sh`) — see "Deploy" below. Dashboards,
|
|
||||||
helpers, and the `docker-exec` adapter have no write path yet. See
|
|
||||||
`DESIGN.md` for the full phasing, adapter, sync, and validation model, and
|
|
||||||
for the open questions still blocking phase 2/3.
|
|
||||||
|
|
||||||
## Layout
|
|
||||||
|
|
||||||
```
|
|
||||||
services/home-assistant/
|
|
||||||
├── DESIGN.md # decision registry — read this first
|
|
||||||
├── instances.yaml # per-instance adapter/host/token config
|
|
||||||
├── .gitignore # excludes secrets/db/log/token paths from every import
|
|
||||||
├── config/<instance>/ # canonical, normalized /config mirror per instance
|
|
||||||
├── storage-export/<instance>/ # curated .storage/* export (registries, dashboards)
|
|
||||||
└── fixtures/ # dated /api/states snapshots
|
|
||||||
```
|
|
||||||
|
|
||||||
Instances: `ken` (RPi4/HAOS, LAN 192.168.31.7, api adapter — canonical home
|
|
||||||
instance since the 2026-07-22 cutover), `ken-legacy` (PIHA, container
|
|
||||||
`homeassistant5`, docker-exec adapter, archived — pre-migration instance,
|
|
||||||
import only, never deploy), `chelsty-ha` (Tailscale, api adapter — see
|
|
||||||
`instances.yaml` and `DESIGN.md` "Incident log").
|
|
||||||
|
|
||||||
## Import
|
## Import
|
||||||
|
|
||||||
61
kb/runbooks/kb-query-deploy.md
Normal file
61
kb/runbooks/kb-query-deploy.md
Normal file
|
|
@ -0,0 +1,61 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-29
|
||||||
|
links:
|
||||||
|
- ../services/kb-query.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# kb-query — deploy i weryfikacja (PIHA)
|
||||||
|
|
||||||
|
## Deploy (PIHA)
|
||||||
|
|
||||||
|
0. Prerequisite: `ollama-piha` deployed and `bge-m3` pulled — see
|
||||||
|
`services/ollama-piha/README.md` (the pull is a **manual** deploy step).
|
||||||
|
1. `git pull` on PIHA (`~/homelab-codex-ws`).
|
||||||
|
2. `cp services/kb-query/env.example services/kb-query/.env` and fill in the
|
||||||
|
real `KB_DSN` password (the template already sets `EMBED_FALLBACK_URL`).
|
||||||
|
On an existing install: add `EMBED_FALLBACK_URL=http://192.168.31.5:11434`
|
||||||
|
to the existing `.env`.
|
||||||
|
3. ```
|
||||||
|
docker compose -f services/kb-query/docker-compose.yml \
|
||||||
|
-f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build
|
||||||
|
```
|
||||||
|
4. Verify: `services/kb-query/healthcheck.sh`, then from PIHA:
|
||||||
|
`curl "http://192.168.31.5:8230/search?q=test"` and open
|
||||||
|
`http://192.168.31.5:8230/` in a browser.
|
||||||
|
|
||||||
|
## Fallback verification (execution: operator, after deploy)
|
||||||
|
|
||||||
|
- **Test A — SOLARIA online**: query via UI/`curl`; response has
|
||||||
|
`"embed_backend": "solaria"`, `docker logs kb-query` shows
|
||||||
|
`backend=solaria`, latency ~sub-second.
|
||||||
|
- **Test B — SOLARIA offline**: either wait for the nightly power-off, or
|
||||||
|
simulate: set `EMBED_PRIMARY_URL=http://192.0.2.1:11434` (TEST-NET, always
|
||||||
|
unreachable) in `.env` and `docker compose … up -d` again. Query still
|
||||||
|
works; response has `"embed_backend": "piha"`, log shows `backend=piha`
|
||||||
|
plus a `circuit open for 30s` warning on the first hit; latency visibly
|
||||||
|
higher (CPU + cold model load each time, `OLLAMA_KEEP_ALIVE=0`). Revert
|
||||||
|
`.env` afterwards if simulated.
|
||||||
|
- **Test C — SOLARIA returns**: after it is back up, within ≤30 s (one
|
||||||
|
health-cache TTL) responses show `"embed_backend": "solaria"` again, no
|
||||||
|
restart needed.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```
|
||||||
|
pip install -e packages/kb-retrieval/
|
||||||
|
cd services/kb-query && pip install -r requirements.txt pytest pytest-asyncio && pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
Unit tests mock the DB connection and Ollama HTTP session (no live DB/Ollama
|
||||||
|
required) — same style as `packages/kb-retrieval/tests/`. `tests/test_frontend.py`
|
||||||
|
drives `GET /`/`/static/*` through FastAPI's `TestClient` without entering it
|
||||||
|
as a context manager, so the DB-requiring `lifespan` never runs.
|
||||||
|
|
||||||
|
Frontend JS has its own pure-function tests (query-URL encoding, threshold
|
||||||
|
colouring, envelope grouping), run without a browser via Node's built-in
|
||||||
|
test runner: `node --test services/kb-query/tests/frontend/`.
|
||||||
|
|
||||||
44
kb/runbooks/mail-body-ingest-run.md
Normal file
44
kb/runbooks/mail-body-ingest-run.md
Normal file
|
|
@ -0,0 +1,44 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-22
|
||||||
|
links:
|
||||||
|
- ../services/job-mail-body-ingest.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# mail-body-ingest — uruchomienie i testy
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Dry run (default) — parse, quote-strip, classify, chunk, count. Zero Ollama calls, zero
|
||||||
|
# DB writes (including the threading UPDATE):
|
||||||
|
mail-body-ingest --dsn postgresql://kb:<pw>@piha:5433/kb --archive-root /home/oskar/kb/mail/archive
|
||||||
|
|
||||||
|
# Etap A pilot — last 12 months only (plan Decyzja 9):
|
||||||
|
mail-body-ingest --dsn ... --since 2025-07-01 --apply > mail-ingest-etapA.log 2>&1
|
||||||
|
|
||||||
|
# Smoke-test slice:
|
||||||
|
mail-body-ingest --dsn ... --apply --limit 10
|
||||||
|
```
|
||||||
|
|
||||||
|
DSN can also come from `KB_DSN`, Ollama URL from `OLLAMA_URL` (default
|
||||||
|
`http://localhost:11434` — this job is meant to run where Ollama lives).
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -e "jobs/mail-body-ingest[dev]"
|
||||||
|
cd jobs/mail-body-ingest && pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
Pure unit tests (48), no DB/Ollama — `run()` is tested by monkeypatching `asyncpg.connect`
|
||||||
|
and `aiohttp.ClientSession` with in-memory fakes, `.eml` bytes written to `tmp_path`. Covers:
|
||||||
|
quote-strip (EN/PL/Outlook markers, bare `>` lines), HTML->text (style/script/blockquote/
|
||||||
|
gmail_quote skipping), newsletter classification, threading extraction, prefix building,
|
||||||
|
body extraction (plain-preferred, HTML fallback, attachment-only), the typed/compat32 parse
|
||||||
|
fallback, stats balance, idempotency (second run inserts nothing new), newsletter chunks
|
||||||
|
never reaching Ollama, Ollama-offline batch isolation, and dimension-mismatch abort.
|
||||||
|
|
||||||
|
|
@ -1,27 +1,14 @@
|
||||||
# narty27
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-31
|
||||||
|
links:
|
||||||
|
- ../services/narty27.md
|
||||||
|
---
|
||||||
|
|
||||||
Static hosting for a single self-contained `viz.html` (narty 2027 / Saalbach KB
|
# narty27 — uruchomienie i operacje
|
||||||
export) on PIHA. Plain `nginx:alpine` serving one Docker named volume — no build,
|
|
||||||
no database, no dependencies.
|
|
||||||
|
|
||||||
- URL: `http://192.168.31.5:8240/viz.html` (and `/` — same file, see below)
|
|
||||||
- Exposure: private (LAN/Tailscale only; no npm vhost, no public ingress)
|
|
||||||
- Volume: `narty27_narty27_content` → `/usr/share/nginx/html:ro`
|
|
||||||
|
|
||||||
## Content is personal and lives outside the repo
|
|
||||||
|
|
||||||
The visualisation is personal content. It is **never committed** — not to this
|
|
||||||
repo, not to any other. It exists in exactly two places:
|
|
||||||
|
|
||||||
1. the source on SOLARIA (`~/narty-2027/saalbach-kb/viz.html`), and
|
|
||||||
2. the `narty27_narty27_content` Docker volume on PIHA.
|
|
||||||
|
|
||||||
There is no backup job and no bind mount under `/opt/homelab/data/`. If the
|
|
||||||
volume is lost, re-run the update procedure below from SOLARIA.
|
|
||||||
|
|
||||||
Two copies of the same file are stored in the volume: `viz.html` (canonical name)
|
|
||||||
and `index.html` (so the bare root `http://192.168.31.5:8240/` works without a
|
|
||||||
path). Both must be refreshed together on every update.
|
|
||||||
|
|
||||||
## Updating the content (from SOLARIA)
|
## Updating the content (from SOLARIA)
|
||||||
|
|
||||||
63
kb/runbooks/ollama-piha-deploy.md
Normal file
63
kb/runbooks/ollama-piha-deploy.md
Normal file
|
|
@ -0,0 +1,63 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-30
|
||||||
|
links:
|
||||||
|
- ../services/ollama-piha.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# ollama-piha — deploy i kalibracja
|
||||||
|
|
||||||
|
## Deploy (PIHA, master, after merge)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/homelab-codex-ws && git pull
|
||||||
|
cp services/ollama-piha/env.example services/ollama-piha/.env # LAN_BIND_IP
|
||||||
|
docker compose -f services/ollama-piha/docker-compose.yml \
|
||||||
|
-f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
|
||||||
|
--env-file services/ollama-piha/.env up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
**Then pull the model — this does NOT happen automatically:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec ollama-piha ollama pull bge-m3
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
services/ollama-piha/healthcheck.sh # checks container + API + bge-m3 present
|
||||||
|
time curl -s http://127.0.0.1:11434/api/embeddings \
|
||||||
|
-d '{"model":"bge-m3","prompt":"test kalibracyjny"}' | head -c 80
|
||||||
|
```
|
||||||
|
|
||||||
|
(`deploy-node.sh` on PIHA also picks this service up from
|
||||||
|
`hosts/piha/services.yaml` once `.env` exists — the `ollama pull bge-m3` step
|
||||||
|
stays manual either way.)
|
||||||
|
|
||||||
|
## Calibration (plan §5 step 4 — gate, not formality)
|
||||||
|
|
||||||
|
Before trusting the fallback under load, on live PIHA at a normal (not
|
||||||
|
night-quiet) hour: run a few embeds as above while watching
|
||||||
|
`docker stats ollama-piha`, note peak RAM and wall time. Verdict per plan §5
|
||||||
|
step 5: keep as default fallback / tune `mem_limit` / fall back to explicit
|
||||||
|
503 degradation.
|
||||||
|
|
||||||
|
**Calibration status: measured 2026-07-27 — verdict GO** (live PIHA under
|
||||||
|
normal load, 3 consecutive `/api/embeddings` calls after `ollama pull bge-m3`;
|
||||||
|
full protocol in `docs/sessions/2026-07-27-kb-f4-fallback.md` §4):
|
||||||
|
|
||||||
|
- Latency: 5.25 s (cold start) / 4.41 s / 4.16 s — single seconds as expected,
|
||||||
|
no warm-up between calls by design (`OLLAMA_KEEP_ALIVE=0` releases the model
|
||||||
|
after every request, `ollama ps` shows nothing resident in between).
|
||||||
|
- RAM: idle ~66 MiB, burst peak ~983 MiB (`docker stats` sampled at 0.3 s) —
|
||||||
|
well inside the 2560m ceiling; host `available` never dropped below ~1.3 GiB.
|
||||||
|
- Kept as **default fallback** (no feature flag). The measurement was taken
|
||||||
|
against the same container configuration this repo deploys (image,
|
||||||
|
`OLLAMA_KEEP_ALIVE=0`, `mem_limit: 2560m`), so it carries over; only the
|
||||||
|
model storage differed (bind mount then, named volume now), which does not
|
||||||
|
affect RAM/latency.
|
||||||
|
|
||||||
67
kb/runbooks/paperless-worker-deploy.md
Normal file
67
kb/runbooks/paperless-worker-deploy.md
Normal file
|
|
@ -0,0 +1,67 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-12
|
||||||
|
links:
|
||||||
|
- ../services/paperless-worker.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# Paperless OCR worker — NFS i cutover
|
||||||
|
|
||||||
|
## NFS: export na PIHA, mount na SOLARIA
|
||||||
|
|
||||||
|
Transfer idzie po **LAN** (PIHA `192.168.31.5` ↔ SOLARIA `192.168.31.70`,
|
||||||
|
1 Gb/s, ten sam switch) — NIE po Tailscale. Przepustowość nie jest wąskim
|
||||||
|
gardłem OCR.
|
||||||
|
|
||||||
|
### Host-side na PIHA (NIE w compose — krok przy deployu modułu 3)
|
||||||
|
|
||||||
|
```
|
||||||
|
# /etc/exports na PIHA — export TYLKO dla SOLARII:
|
||||||
|
/opt/homelab/data/paperless 192.168.31.70(rw,sync,no_subtree_check,no_root_squash)
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install nfs-kernel-server # jesli brak
|
||||||
|
sudo exportfs -ra
|
||||||
|
```
|
||||||
|
|
||||||
|
`no_root_squash` jest potrzebne, bo entrypoint obrazu (root) robi `chown` na
|
||||||
|
katalogach przy starcie kontenera na SOLARII; export jest ograniczony do
|
||||||
|
jednego IP w zaufanym LAN.
|
||||||
|
|
||||||
|
### Strona SOLARII
|
||||||
|
|
||||||
|
Mounty definiuje compose jako named volumes z driverem NFS — **zero wpisów
|
||||||
|
w /etc/fstab**; jedyny host-side wymóg to pakiet klienta:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install nfs-common
|
||||||
|
```
|
||||||
|
|
||||||
|
### UID mapping (krytyczne)
|
||||||
|
|
||||||
|
Pliki na exporcie mają numerycznego właściciela — NFS nie tłumaczy nazw.
|
||||||
|
Dlatego `USERMAP_UID/GID=1000` jest ustawione **w obu** compose (PIHA
|
||||||
|
i SOLARIA); zmiana po jednej stronie = worker traci dostęp do plików.
|
||||||
|
Weryfikacja po deployu: `./healthcheck.sh` robi test zapisu na mount.
|
||||||
|
|
||||||
|
## Cutover checklist (przy deployu modułu 3 — po działającym module 2)
|
||||||
|
|
||||||
|
1. ✅ `services/paperless/` działa na PIHA (healthcheck zielony).
|
||||||
|
2. ✅ Export NFS na PIHA (wyżej) + `showmount -e 192.168.31.5` z SOLARII.
|
||||||
|
3. ✅ `nfs-common` na SOLARII.
|
||||||
|
4. ✅ `.env` z `env.example` — sekrety SKOPIOWANE z PIHA, nie nowe.
|
||||||
|
5. ✅ `docker compose up -d` + `./healthcheck.sh`.
|
||||||
|
6. ✅ Test (2026-07-12): PDF-y wrzucone do consume na PIHA, część odebrana i
|
||||||
|
dokończona przez worker@SOLARIA (dowód w logach, zero File not found).
|
||||||
|
7. ⬜ Test fallbacku: stop workera → zadanie czeka/mieli PIHA → start → drenaż
|
||||||
|
(jeszcze niewykonany formalnie, ale mechanizm nie zmienił się tym fixem —
|
||||||
|
fallback na PIHA działał już wcześniej, patrz sekcja "Fallback" wyżej).
|
||||||
|
8. ⬜ **OTWARTE**: wpis `paperless-worker` w `hosts/solaria/services.yaml` +
|
||||||
|
`inventory/topology.yaml` (obecnie SOLARIA ma tam tylko `node-agent`) —
|
||||||
|
bez tego supervisor/observer nie widzą tego serwisu w desired-state, więc
|
||||||
|
drift między `hosts/solaria/services.yaml` a rzeczywistością nie jest
|
||||||
|
wykrywany. Patrz `docs/backlog.md`.
|
||||||
105
kb/runbooks/planner-agent-deploy.md
Normal file
105
kb/runbooks/planner-agent-deploy.md
Normal file
|
|
@ -0,0 +1,105 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: runbook
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-05-27
|
||||||
|
links:
|
||||||
|
- ../services/planner-agent.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# planner-agent — deploy, uruchamianie, testy
|
||||||
|
|
||||||
|
## Deployment na SOLARIA
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Przygotuj .env na solaria
|
||||||
|
ssh oskar@100.100.231.104
|
||||||
|
mkdir -p /opt/homelab/config/planner-agent
|
||||||
|
cat > /opt/homelab/config/planner-agent/.env << 'EOF'
|
||||||
|
OLLAMA_HOST=http://host-gateway:11434
|
||||||
|
OLLAMA_MODEL=qwen2.5-coder:14b
|
||||||
|
REDIS_URL=redis://100.108.208.3:6379
|
||||||
|
NODE_NAME=solaria
|
||||||
|
COOLDOWN_SECONDS=300
|
||||||
|
RUNTIME_PATH=/opt/homelab
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# 2. Deploy przez standardowy skrypt (z SATURN)
|
||||||
|
scripts/deploy/deploy.sh --service planner-agent
|
||||||
|
|
||||||
|
# 3. Weryfikacja
|
||||||
|
docker ps --filter name=planner-agent
|
||||||
|
docker logs planner-agent --tail 30
|
||||||
|
ls -la /opt/homelab/state/planner-agent.heartbeat
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Uwaga:** stage `verify` w deploy.sh może failować bezpośrednio po starcie
|
||||||
|
> (race condition — heartbeat jest pisany co ~5 s). Kontener jest zdrowy gdy
|
||||||
|
> `docker ps` pokazuje `(healthy)`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Uruchamianie lokalne (bez Dockera)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd services/planner-agent
|
||||||
|
|
||||||
|
# Zainstaluj zależności
|
||||||
|
pip install -r requirements.txt
|
||||||
|
|
||||||
|
# Wyeksportuj zmienne (lub stwórz .env i użyj `export $(cat .env | xargs)`)
|
||||||
|
export REDIS_URL=redis://localhost:6379
|
||||||
|
export OLLAMA_HOST=http://localhost:11434
|
||||||
|
export OLLAMA_MODEL=qwen2.5:7b
|
||||||
|
export NODE_NAME=dev-local
|
||||||
|
export RUNTIME_PATH=/tmp/homelab-dev
|
||||||
|
export COOLDOWN_SECONDS=10
|
||||||
|
|
||||||
|
# Uruchom
|
||||||
|
python src/planner.py
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testowanie
|
||||||
|
|
||||||
|
### Testy jednostkowe
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd services/planner-agent
|
||||||
|
pip install -r requirements.txt pytest pytest-asyncio
|
||||||
|
pytest tests/ -v
|
||||||
|
# 49 testów planner + 34 testy llm_router
|
||||||
|
```
|
||||||
|
|
||||||
|
### Ręczny end-to-end test przez Redis
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Opublikuj sztuczne zdarzenie unhealthy na kanale health_events
|
||||||
|
redis-cli -h 100.108.208.3 PUBLISH health_events '{
|
||||||
|
"type": "service_unhealthy",
|
||||||
|
"node": "piha",
|
||||||
|
"service": "mosquitto",
|
||||||
|
"severity": "error",
|
||||||
|
"payload": {"exit_code": 1, "reason": "OOMKilled"},
|
||||||
|
"timestamp": "2026-05-27T20:00:00Z"
|
||||||
|
}'
|
||||||
|
|
||||||
|
# Obserwuj logi planera
|
||||||
|
docker logs planner-agent -f
|
||||||
|
|
||||||
|
# Sprawdź czy propozycja trafiła do pending
|
||||||
|
ls /opt/homelab/actions/pending/
|
||||||
|
cat /opt/homelab/actions/pending/plan-piha-mosquitto-*.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Śledzenie metryk LLM
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Subskrybuj metryki routera w czasie rzeczywistym
|
||||||
|
redis-cli -h 100.108.208.3 SUBSCRIBE llm_router_metrics
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
@ -1,3 +1,13 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-06-11
|
||||||
|
links:
|
||||||
|
- ../runbooks/ha-diag-agent-runbook.md
|
||||||
|
---
|
||||||
|
|
||||||
# ha-diag-agent
|
# ha-diag-agent
|
||||||
|
|
||||||
Per-host Home Assistant diagnostic agent. Polls HA REST API on a schedule,
|
Per-host Home Assistant diagnostic agent. Polls HA REST API on a schedule,
|
||||||
|
|
@ -52,17 +62,6 @@ checks are APScheduler intervals (stateless REST polls).
|
||||||
Event routing in supervisor (Phase 5) maps these to `notify` actions.
|
Event routing in supervisor (Phase 5) maps these to `notify` actions.
|
||||||
`ha_websocket_recovered` should be routed to clear any active `ha_websocket_dead` incident.
|
`ha_websocket_recovered` should be routed to clear any active `ha_websocket_dead` incident.
|
||||||
|
|
||||||
## First-time deployment
|
|
||||||
|
|
||||||
See **[DEPLOY.md](DEPLOY.md)** for the full procedure: HA token creation,
|
|
||||||
per-host `.env` config, deploy commands, verification steps, 48h shadow-mode
|
|
||||||
observation, and rollback.
|
|
||||||
|
|
||||||
**Shadow mode** (`HA_DIAG_SHADOW_MODE`, default `true` on the control-plane):
|
|
||||||
`ha_websocket_dead` events are downgraded to `alert_only` with a `[SHADOW MODE]`
|
|
||||||
note instead of queuing an automatic `container_restart`. Set to `false` in
|
|
||||||
`/opt/homelab/config/control-plane/.env` on the VPS when ready for live actions.
|
|
||||||
|
|
||||||
## Deployment model
|
## Deployment model
|
||||||
|
|
||||||
The agent is deployed **per-host** but targets a potentially remote HA instance:
|
The agent is deployed **per-host** but targets a potentially remote HA instance:
|
||||||
|
|
@ -78,47 +77,6 @@ OS VM. `chelsty-infra` is the hypervisor but does not run HA itself. The agent o
|
||||||
gets a new Tailscale IP, update `HA_URL` in `/opt/homelab/config/ha-diag-agent/.env` on
|
gets a new Tailscale IP, update `HA_URL` in `/opt/homelab/config/ha-diag-agent/.env` on
|
||||||
`chelsty-infra`.
|
`chelsty-infra`.
|
||||||
|
|
||||||
## Deployment
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Create config on target node
|
|
||||||
ssh oskar@<node-ip>
|
|
||||||
mkdir -p /opt/homelab/config/ha-diag-agent /var/lib/ha-diag-agent
|
|
||||||
cat > /opt/homelab/config/ha-diag-agent/.env << 'EOF'
|
|
||||||
HA_URL=http://homeassistant.local:8123 # or http://100.70.180.90:8123 for chelsty-infra
|
|
||||||
HA_TOKEN=<long-lived-token>
|
|
||||||
NODE_NAME=piha # or chelsty-infra
|
|
||||||
LOCATION_TAG=ken # or chelsty
|
|
||||||
CHECK_INTERVAL=60
|
|
||||||
EOF
|
|
||||||
|
|
||||||
# 2. Deploy
|
|
||||||
scripts/deploy/deploy.sh --service ha-diag-agent
|
|
||||||
|
|
||||||
# 3. Verify
|
|
||||||
docker ps --filter name=ha-diag-agent
|
|
||||||
docker exec ha-diag-agent python -c "import urllib.request; print(urllib.request.urlopen('http://localhost:8087/health', timeout=5).read().decode())"
|
|
||||||
```
|
|
||||||
|
|
||||||
### chelsty-infra note
|
|
||||||
|
|
||||||
`chelsty-infra` runs docker-compose v1 (1.29.2). Use `docker-compose` (hyphenated):
|
|
||||||
```bash
|
|
||||||
docker-compose -f docker-compose.yml up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
### HA long-lived token
|
|
||||||
|
|
||||||
In HA UI: Profile → Long-Lived Access Tokens → Create token.
|
|
||||||
|
|
||||||
## Running Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd services/ha-diag-agent
|
|
||||||
pip install -e ".[dev]"
|
|
||||||
pytest tests/ -v
|
|
||||||
```
|
|
||||||
|
|
||||||
## Optional YAML config
|
## Optional YAML config
|
||||||
|
|
||||||
Place `/opt/homelab/config/ha-diag-agent/ha-diag-agent.yaml` on the node.
|
Place `/opt/homelab/config/ha-diag-agent/ha-diag-agent.yaml` on the node.
|
||||||
|
|
@ -1,3 +1,13 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-30
|
||||||
|
links:
|
||||||
|
- ../runbooks/ha-mcp-install.md
|
||||||
|
---
|
||||||
|
|
||||||
# ha-mcp — read-only MCP server for Home Assistant
|
# ha-mcp — read-only MCP server for Home Assistant
|
||||||
|
|
||||||
**Status: phase 2a** of `services/home-assistant/DESIGN.md` — own minimal MCP
|
**Status: phase 2a** of `services/home-assistant/DESIGN.md` — own minimal MCP
|
||||||
|
|
@ -32,74 +42,6 @@ verify). See `services/home-assistant/DESIGN.md`, "Sync model" and
|
||||||
"Validation gate". Nothing in this server bypasses that, and nothing in this
|
"Validation gate". Nothing in this server bypasses that, and nothing in this
|
||||||
server should ever learn to.
|
server should ever learn to.
|
||||||
|
|
||||||
## Install
|
|
||||||
|
|
||||||
The `mcp` SDK (and its pydantic/anyio/httpx dependency tree) is not packaged
|
|
||||||
for Debian and is not needed by anything else in this repo, so it goes into a
|
|
||||||
venv rather than into the system Python:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 -m venv --system-site-packages services/ha-mcp/.venv
|
|
||||||
services/ha-mcp/.venv/bin/pip install mcp pytest
|
|
||||||
```
|
|
||||||
|
|
||||||
`--system-site-packages` is deliberate: `requests`, `PyYAML` and
|
|
||||||
`websocket-client` are already installed system-wide and used by
|
|
||||||
`scripts/ha/lib/*`; the venv reuses those exact versions instead of shadowing
|
|
||||||
them with a second copy. `.venv/` is already covered by the repo `.gitignore`.
|
|
||||||
|
|
||||||
`pip install --break-system-packages mcp` also works and is one line shorter,
|
|
||||||
but it writes into the system interpreter that runs every deploy script on
|
|
||||||
this workstation — a venv keeps a 40-package dependency tree out of that blast
|
|
||||||
radius for a tool only Claude Code uses. Use the venv.
|
|
||||||
|
|
||||||
`run.sh` prefers `services/ha-mcp/.venv/bin/python` and falls back to the
|
|
||||||
system `python3`; if neither can import `mcp` it exits with one actionable
|
|
||||||
line on stderr rather than a traceback.
|
|
||||||
|
|
||||||
## Registration in Claude Code
|
|
||||||
|
|
||||||
`.mcp.json` in the repo root (project scope — shared with anyone who checks
|
|
||||||
out this repo):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"mcpServers": {
|
|
||||||
"ha": {
|
|
||||||
"command": "./services/ha-mcp/run.sh",
|
|
||||||
"args": [],
|
|
||||||
"env": {}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The command path is relative, so it resolves in any checkout (main or task
|
|
||||||
worktree) as long as Claude Code is started from the repo root. Start CC
|
|
||||||
there; on first run it asks whether to trust the project's MCP servers. Check
|
|
||||||
with `/mcp` — the server appears as `ha`, its tools as `mcp__ha__<tool>`.
|
|
||||||
Which repo the server reads is derived from its own location; override with
|
|
||||||
the `HA_MCP_REPO` environment variable if you ever need to point one checkout
|
|
||||||
at another's config.
|
|
||||||
|
|
||||||
## Standalone
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./services/ha-mcp/run.sh # stdio server — speaks JSON-RPC on stdout
|
|
||||||
services/ha-mcp/tests/run.sh # offline test suite (no HA, no token)
|
|
||||||
services/ha-mcp/.venv/bin/python services/ha-mcp/tests/smoke_live.py [instance] [entity_id]
|
|
||||||
```
|
|
||||||
|
|
||||||
`smoke_live.py` is the only thing here that touches the network (GET only). A
|
|
||||||
one-off tool call without a client is easiest through the package:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
PYTHONPATH=services/ha-mcp/src services/ha-mcp/.venv/bin/python -c "
|
|
||||||
from ha_mcp import tools; from ha_mcp.backend import LiveBackend
|
|
||||||
from ha_mcp.config import get_instance, repo_root
|
|
||||||
n, c = get_instance('ken', repo_root()); print(tools.instance_status(LiveBackend(n, c, repo_root())))"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Tools
|
## Tools
|
||||||
|
|
||||||
All seven are read-only, take an optional `instance` (default `ken` — the
|
All seven are read-only, take an optional `instance` (default `ken` — the
|
||||||
|
|
@ -231,23 +173,6 @@ a log line, in a tool result, or in this repo. A missing token is a reported
|
||||||
tool error naming the path it looked at — not a crash, and not a silent empty
|
tool error naming the path it looked at — not a crash, and not a silent empty
|
||||||
result.
|
result.
|
||||||
|
|
||||||
## Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
services/ha-mcp/tests/run.sh # 42 tests, offline
|
|
||||||
```
|
|
||||||
|
|
||||||
Offline in the same sense as `scripts/ha/tests/*`: no network, no HA
|
|
||||||
instance, no token. Small hand-made fixtures under `tests/fixtures/` (shaped
|
|
||||||
like real `/api/states` and WebSocket registry payloads) cover the tool logic;
|
|
||||||
the repo's own `config/ken/automations/` and newest `fixtures/ken-states-*.yaml`
|
|
||||||
cover the repo-backed and at-scale paths (ranking over ~1650 real entities
|
|
||||||
behaves differently from ranking over ten).
|
|
||||||
|
|
||||||
Live smoke (read-only, run against `ken` on 2026-07-30):
|
|
||||||
`instance_status` → HA 2026.7.2, 1647 entities, 377 unavailable, 115
|
|
||||||
automations, 13 areas; `get_state("sensor.thsalon_temperature")` → `24.5 °C`.
|
|
||||||
|
|
||||||
## Not a deployed service
|
## Not a deployed service
|
||||||
|
|
||||||
No `docker-compose.yml`, no `service.yaml`, no `healthcheck.sh`: this is
|
No `docker-compose.yml`, no `service.yaml`, no `healthcheck.sh`: this is
|
||||||
37
kb/services/home-assistant.md
Normal file
37
kb/services/home-assistant.md
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-22
|
||||||
|
links:
|
||||||
|
- ../runbooks/home-assistant-deploy.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# home-assistant (configs-as-code)
|
||||||
|
|
||||||
|
**Status: phase 1 (partial).** Read-only import tooling, plus a deploy
|
||||||
|
(repo -> instance) write path for the `api` adapter's automations/scripts/
|
||||||
|
scenes scope only (`scripts/ha/deploy.sh`) — see "Deploy" below. Dashboards,
|
||||||
|
helpers, and the `docker-exec` adapter have no write path yet. See
|
||||||
|
`DESIGN.md` for the full phasing, adapter, sync, and validation model, and
|
||||||
|
for the open questions still blocking phase 2/3.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
services/home-assistant/
|
||||||
|
├── DESIGN.md # decision registry — read this first
|
||||||
|
├── instances.yaml # per-instance adapter/host/token config
|
||||||
|
├── .gitignore # excludes secrets/db/log/token paths from every import
|
||||||
|
├── config/<instance>/ # canonical, normalized /config mirror per instance
|
||||||
|
├── storage-export/<instance>/ # curated .storage/* export (registries, dashboards)
|
||||||
|
└── fixtures/ # dated /api/states snapshots
|
||||||
|
```
|
||||||
|
|
||||||
|
Instances: `ken` (RPi4/HAOS, LAN 192.168.31.7, api adapter — canonical home
|
||||||
|
instance since the 2026-07-22 cutover), `ken-legacy` (PIHA, container
|
||||||
|
`homeassistant5`, docker-exec adapter, archived — pre-migration instance,
|
||||||
|
import only, never deploy), `chelsty-ha` (Tailscale, api adapter — see
|
||||||
|
`instances.yaml` and `DESIGN.md` "Incident log").
|
||||||
|
|
||||||
|
|
@ -1,3 +1,13 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-14
|
||||||
|
links:
|
||||||
|
- ../runbooks/gmail-header-backfill-run.md
|
||||||
|
---
|
||||||
|
|
||||||
# gmail-header-backfill
|
# gmail-header-backfill
|
||||||
|
|
||||||
One-shot job, module 5 phase 2 backfill (`docs/kb/modules/05-faza2-plan.md`, §5).
|
One-shot job, module 5 phase 2 backfill (`docs/kb/modules/05-faza2-plan.md`, §5).
|
||||||
|
|
@ -27,26 +37,6 @@ Install (from repo root, on PIHA):
|
||||||
pip install -e jobs/gmail-header-backfill/
|
pip install -e jobs/gmail-header-backfill/
|
||||||
```
|
```
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Dry run (default) — parse and count only, no DB writes:
|
|
||||||
gmail-header-backfill --dsn postgresql://kb:<pw>@localhost:5433/kb --limit 100
|
|
||||||
|
|
||||||
# Real run — apply the UPDATE for this slice:
|
|
||||||
gmail-header-backfill --dsn ... --limit 1000 --offset 0 --apply
|
|
||||||
|
|
||||||
# Next slice — offset is stable/deterministic (ORDER BY id), independent of
|
|
||||||
# how many rows in earlier slices were already backfilled:
|
|
||||||
gmail-header-backfill --dsn ... --limit 1000 --offset 1000 --apply
|
|
||||||
```
|
|
||||||
|
|
||||||
DSN can also come from the `KB_DSN` env var instead of `--dsn`.
|
|
||||||
|
|
||||||
`--limit`/`--offset` exist so the full 225 030-row backfill can be run in
|
|
||||||
verifiable partitions instead of one long unattended run (plan §5.3) — start
|
|
||||||
small (`--limit 100`), check the result in the DB, then widen.
|
|
||||||
|
|
||||||
## Source of headers: the archived `.eml`, not the original mbox
|
## Source of headers: the archived `.eml`, not the original mbox
|
||||||
|
|
||||||
Same access pattern as `documents-ingest`: `archive_root / row["raw_ref"]` ->
|
Same access pattern as `documents-ingest`: `archive_root / row["raw_ref"]` ->
|
||||||
|
|
@ -171,23 +161,6 @@ of streaming one mbox — the plan's estimate is "same order of magnitude,
|
||||||
likely tens of minutes." Not measured directly in this change — see the
|
likely tens of minutes." Not measured directly in this change — see the
|
||||||
DoD note below.
|
DoD note below.
|
||||||
|
|
||||||
## Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -e jobs/gmail-header-backfill/
|
|
||||||
cd jobs/gmail-header-backfill && pytest
|
|
||||||
```
|
|
||||||
|
|
||||||
Pure unit tests, no DB or filesystem outside `tmp_path`/synthetic `.eml`
|
|
||||||
bytes — `run()` is tested by monkeypatching `asyncpg.connect` with an
|
|
||||||
in-memory fake connection. Covers: header parsing (multi-address `To`/`Cc`,
|
|
||||||
quoted display names with commas, multiple `Delivered-To` occurrences, RFC
|
|
||||||
2047 encoded-words including Polish diacritics, malformed encoded-words that
|
|
||||||
must not raise, missing/multiple `From`, `date_raw` preserving literal text
|
|
||||||
vs. `Date` header reformatting), idempotency (rows already carrying a
|
|
||||||
`headers` entity are skipped and never re-appended), batch flushing, and
|
|
||||||
`--limit`/`--offset` query shape.
|
|
||||||
|
|
||||||
## Definition of Done
|
## Definition of Done
|
||||||
|
|
||||||
Per `CLAUDE.md`: this job's smoke run is `gmail-header-backfill --dsn ...
|
Per `CLAUDE.md`: this job's smoke run is `gmail-header-backfill --dsn ...
|
||||||
|
|
@ -1,3 +1,13 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-22
|
||||||
|
links:
|
||||||
|
- ../runbooks/mail-body-ingest-run.md
|
||||||
|
---
|
||||||
|
|
||||||
# mail-body-ingest
|
# mail-body-ingest
|
||||||
|
|
||||||
Module 5, faza mailowa (`docs/kb/modules/05-faza-mailowa-plan.md`, §5, Krok 2). Second full
|
Module 5, faza mailowa (`docs/kb/modules/05-faza-mailowa-plan.md`, §5, Krok 2). Second full
|
||||||
|
|
@ -28,23 +38,6 @@ pip install -e packages/kb-retrieval/
|
||||||
pip install -e jobs/mail-body-ingest/
|
pip install -e jobs/mail-body-ingest/
|
||||||
```
|
```
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Dry run (default) — parse, quote-strip, classify, chunk, count. Zero Ollama calls, zero
|
|
||||||
# DB writes (including the threading UPDATE):
|
|
||||||
mail-body-ingest --dsn postgresql://kb:<pw>@piha:5433/kb --archive-root /home/oskar/kb/mail/archive
|
|
||||||
|
|
||||||
# Etap A pilot — last 12 months only (plan Decyzja 9):
|
|
||||||
mail-body-ingest --dsn ... --since 2025-07-01 --apply > mail-ingest-etapA.log 2>&1
|
|
||||||
|
|
||||||
# Smoke-test slice:
|
|
||||||
mail-body-ingest --dsn ... --apply --limit 10
|
|
||||||
```
|
|
||||||
|
|
||||||
DSN can also come from `KB_DSN`, Ollama URL from `OLLAMA_URL` (default
|
|
||||||
`http://localhost:11434` — this job is meant to run where Ollama lives).
|
|
||||||
|
|
||||||
## Pipeline (per envelope)
|
## Pipeline (per envelope)
|
||||||
|
|
||||||
1. **Read** `archive_root / raw_ref` — `missing_file`/`read_error` counted like
|
1. **Read** `archive_root / raw_ref` — `missing_file`/`read_error` counted like
|
||||||
|
|
@ -107,21 +100,6 @@ fetch is already scoped to it. Built this way from the start per the plan's note
|
||||||
models (not an active bug there today, since one run always uses one model, but worth not
|
models (not an active bug there today, since one run always uses one model, but worth not
|
||||||
repeating the ambiguity here).
|
repeating the ambiguity here).
|
||||||
|
|
||||||
## Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -e "jobs/mail-body-ingest[dev]"
|
|
||||||
cd jobs/mail-body-ingest && pytest
|
|
||||||
```
|
|
||||||
|
|
||||||
Pure unit tests (48), no DB/Ollama — `run()` is tested by monkeypatching `asyncpg.connect`
|
|
||||||
and `aiohttp.ClientSession` with in-memory fakes, `.eml` bytes written to `tmp_path`. Covers:
|
|
||||||
quote-strip (EN/PL/Outlook markers, bare `>` lines), HTML->text (style/script/blockquote/
|
|
||||||
gmail_quote skipping), newsletter classification, threading extraction, prefix building,
|
|
||||||
body extraction (plain-preferred, HTML fallback, attachment-only), the typed/compat32 parse
|
|
||||||
fallback, stats balance, idempotency (second run inserts nothing new), newsletter chunks
|
|
||||||
never reaching Ollama, Ollama-offline batch isolation, and dimension-mismatch abort.
|
|
||||||
|
|
||||||
## Definition of Done
|
## Definition of Done
|
||||||
|
|
||||||
Per `CLAUDE.md`: `pytest` passes (48/48) + a `--limit 5` dry-run smoke against live
|
Per `CLAUDE.md`: `pytest` passes (48/48) + a `--limit 5` dry-run smoke against live
|
||||||
|
|
@ -1,3 +1,13 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-29
|
||||||
|
links:
|
||||||
|
- ../runbooks/kb-query-deploy.md
|
||||||
|
---
|
||||||
|
|
||||||
# kb-query
|
# kb-query
|
||||||
|
|
||||||
FastAPI search API in front of the module-5 KB retrieval engine
|
FastAPI search API in front of the module-5 KB retrieval engine
|
||||||
|
|
@ -120,55 +130,6 @@ that *wrote* the summary, e.g. `claude-haiku-4-5`, not the embedder).
|
||||||
(`claude-haiku-4-5`). `OLLAMA_URL` was **renamed** to `EMBED_PRIMARY_URL` in
|
(`claude-haiku-4-5`). `OLLAMA_URL` was **renamed** to `EMBED_PRIMARY_URL` in
|
||||||
Krok 2 — if an old `.env` sets `OLLAMA_URL`, it is ignored.
|
Krok 2 — if an old `.env` sets `OLLAMA_URL`, it is ignored.
|
||||||
|
|
||||||
## Deploy (PIHA)
|
|
||||||
|
|
||||||
0. Prerequisite: `ollama-piha` deployed and `bge-m3` pulled — see
|
|
||||||
`services/ollama-piha/README.md` (the pull is a **manual** deploy step).
|
|
||||||
1. `git pull` on PIHA (`~/homelab-codex-ws`).
|
|
||||||
2. `cp services/kb-query/env.example services/kb-query/.env` and fill in the
|
|
||||||
real `KB_DSN` password (the template already sets `EMBED_FALLBACK_URL`).
|
|
||||||
On an existing install: add `EMBED_FALLBACK_URL=http://192.168.31.5:11434`
|
|
||||||
to the existing `.env`.
|
|
||||||
3. ```
|
|
||||||
docker compose -f services/kb-query/docker-compose.yml \
|
|
||||||
-f hosts/piha/runtime/kb-query/docker-compose.override.yml up -d --build
|
|
||||||
```
|
|
||||||
4. Verify: `services/kb-query/healthcheck.sh`, then from PIHA:
|
|
||||||
`curl "http://192.168.31.5:8230/search?q=test"` and open
|
|
||||||
`http://192.168.31.5:8230/` in a browser.
|
|
||||||
|
|
||||||
## Fallback verification (execution: operator, after deploy)
|
|
||||||
|
|
||||||
- **Test A — SOLARIA online**: query via UI/`curl`; response has
|
|
||||||
`"embed_backend": "solaria"`, `docker logs kb-query` shows
|
|
||||||
`backend=solaria`, latency ~sub-second.
|
|
||||||
- **Test B — SOLARIA offline**: either wait for the nightly power-off, or
|
|
||||||
simulate: set `EMBED_PRIMARY_URL=http://192.0.2.1:11434` (TEST-NET, always
|
|
||||||
unreachable) in `.env` and `docker compose … up -d` again. Query still
|
|
||||||
works; response has `"embed_backend": "piha"`, log shows `backend=piha`
|
|
||||||
plus a `circuit open for 30s` warning on the first hit; latency visibly
|
|
||||||
higher (CPU + cold model load each time, `OLLAMA_KEEP_ALIVE=0`). Revert
|
|
||||||
`.env` afterwards if simulated.
|
|
||||||
- **Test C — SOLARIA returns**: after it is back up, within ≤30 s (one
|
|
||||||
health-cache TTL) responses show `"embed_backend": "solaria"` again, no
|
|
||||||
restart needed.
|
|
||||||
|
|
||||||
## Tests
|
|
||||||
|
|
||||||
```
|
|
||||||
pip install -e packages/kb-retrieval/
|
|
||||||
cd services/kb-query && pip install -r requirements.txt pytest pytest-asyncio && pytest
|
|
||||||
```
|
|
||||||
|
|
||||||
Unit tests mock the DB connection and Ollama HTTP session (no live DB/Ollama
|
|
||||||
required) — same style as `packages/kb-retrieval/tests/`. `tests/test_frontend.py`
|
|
||||||
drives `GET /`/`/static/*` through FastAPI's `TestClient` without entering it
|
|
||||||
as a context manager, so the DB-requiring `lifespan` never runs.
|
|
||||||
|
|
||||||
Frontend JS has its own pure-function tests (query-URL encoding, threshold
|
|
||||||
colouring, envelope grouping), run without a browser via Node's built-in
|
|
||||||
test runner: `node --test services/kb-query/tests/frontend/`.
|
|
||||||
|
|
||||||
## Ingress (`kb.kapala.org`, plan §8)
|
## Ingress (`kb.kapala.org`, plan §8)
|
||||||
|
|
||||||
Wired up 2026-07-23 (`docs/sessions/2026-07-23-kb-f4-ingress.md`), **no code
|
Wired up 2026-07-23 (`docs/sessions/2026-07-23-kb-f4-ingress.md`), **no code
|
||||||
35
kb/services/narty27.md
Normal file
35
kb/services/narty27.md
Normal file
|
|
@ -0,0 +1,35 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-31
|
||||||
|
links:
|
||||||
|
- ../runbooks/narty27-deploy.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# narty27
|
||||||
|
|
||||||
|
Static hosting for a single self-contained `viz.html` (narty 2027 / Saalbach KB
|
||||||
|
export) on PIHA. Plain `nginx:alpine` serving one Docker named volume — no build,
|
||||||
|
no database, no dependencies.
|
||||||
|
|
||||||
|
- URL: `http://192.168.31.5:8240/viz.html` (and `/` — same file, see below)
|
||||||
|
- Exposure: private (LAN/Tailscale only; no npm vhost, no public ingress)
|
||||||
|
- Volume: `narty27_narty27_content` → `/usr/share/nginx/html:ro`
|
||||||
|
|
||||||
|
## Content is personal and lives outside the repo
|
||||||
|
|
||||||
|
The visualisation is personal content. It is **never committed** — not to this
|
||||||
|
repo, not to any other. It exists in exactly two places:
|
||||||
|
|
||||||
|
1. the source on SOLARIA (`~/narty-2027/saalbach-kb/viz.html`), and
|
||||||
|
2. the `narty27_narty27_content` Docker volume on PIHA.
|
||||||
|
|
||||||
|
There is no backup job and no bind mount under `/opt/homelab/data/`. If the
|
||||||
|
volume is lost, re-run the update procedure below from SOLARIA.
|
||||||
|
|
||||||
|
Two copies of the same file are stored in the volume: `viz.html` (canonical name)
|
||||||
|
and `index.html` (so the bare root `http://192.168.31.5:8240/` works without a
|
||||||
|
path). Both must be refreshed together on every update.
|
||||||
|
|
||||||
|
|
@ -1,3 +1,13 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-30
|
||||||
|
links:
|
||||||
|
- ../runbooks/ollama-piha-deploy.md
|
||||||
|
---
|
||||||
|
|
||||||
# ollama-piha
|
# ollama-piha
|
||||||
|
|
||||||
Local CPU Ollama on **PIHA**, serving exactly one purpose: the **fallback embed
|
Local CPU Ollama on **PIHA**, serving exactly one purpose: the **fallback embed
|
||||||
|
|
@ -29,57 +39,6 @@ model is never resident — see below). Slower but alive beats fast but dead.
|
||||||
PIHA's uid pattern (host oskar=1004, containers uid 1000, setgid group pi)
|
PIHA's uid pattern (host oskar=1004, containers uid 1000, setgid group pi)
|
||||||
if it wrote to a shared bind directory.
|
if it wrote to a shared bind directory.
|
||||||
|
|
||||||
## Deploy (PIHA, master, after merge)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd ~/homelab-codex-ws && git pull
|
|
||||||
cp services/ollama-piha/env.example services/ollama-piha/.env # LAN_BIND_IP
|
|
||||||
docker compose -f services/ollama-piha/docker-compose.yml \
|
|
||||||
-f hosts/piha/runtime/ollama-piha/docker-compose.override.yml \
|
|
||||||
--env-file services/ollama-piha/.env up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
**Then pull the model — this does NOT happen automatically:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker exec ollama-piha ollama pull bge-m3
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
services/ollama-piha/healthcheck.sh # checks container + API + bge-m3 present
|
|
||||||
time curl -s http://127.0.0.1:11434/api/embeddings \
|
|
||||||
-d '{"model":"bge-m3","prompt":"test kalibracyjny"}' | head -c 80
|
|
||||||
```
|
|
||||||
|
|
||||||
(`deploy-node.sh` on PIHA also picks this service up from
|
|
||||||
`hosts/piha/services.yaml` once `.env` exists — the `ollama pull bge-m3` step
|
|
||||||
stays manual either way.)
|
|
||||||
|
|
||||||
## Calibration (plan §5 step 4 — gate, not formality)
|
|
||||||
|
|
||||||
Before trusting the fallback under load, on live PIHA at a normal (not
|
|
||||||
night-quiet) hour: run a few embeds as above while watching
|
|
||||||
`docker stats ollama-piha`, note peak RAM and wall time. Verdict per plan §5
|
|
||||||
step 5: keep as default fallback / tune `mem_limit` / fall back to explicit
|
|
||||||
503 degradation.
|
|
||||||
|
|
||||||
**Calibration status: measured 2026-07-27 — verdict GO** (live PIHA under
|
|
||||||
normal load, 3 consecutive `/api/embeddings` calls after `ollama pull bge-m3`;
|
|
||||||
full protocol in `docs/sessions/2026-07-27-kb-f4-fallback.md` §4):
|
|
||||||
|
|
||||||
- Latency: 5.25 s (cold start) / 4.41 s / 4.16 s — single seconds as expected,
|
|
||||||
no warm-up between calls by design (`OLLAMA_KEEP_ALIVE=0` releases the model
|
|
||||||
after every request, `ollama ps` shows nothing resident in between).
|
|
||||||
- RAM: idle ~66 MiB, burst peak ~983 MiB (`docker stats` sampled at 0.3 s) —
|
|
||||||
well inside the 2560m ceiling; host `available` never dropped below ~1.3 GiB.
|
|
||||||
- Kept as **default fallback** (no feature flag). The measurement was taken
|
|
||||||
against the same container configuration this repo deploys (image,
|
|
||||||
`OLLAMA_KEEP_ALIVE=0`, `mem_limit: 2560m`), so it carries over; only the
|
|
||||||
model storage differed (bind mount then, named volume now), which does not
|
|
||||||
affect RAM/latency.
|
|
||||||
|
|
||||||
## Relation to kb-query
|
## Relation to kb-query
|
||||||
|
|
||||||
kb-query's router (`services/kb-query/app/embed_router.py`) health-checks
|
kb-query's router (`services/kb-query/app/embed_router.py`) health-checks
|
||||||
|
|
@ -1,3 +1,13 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-07-12
|
||||||
|
links:
|
||||||
|
- ../runbooks/paperless-worker-deploy.md
|
||||||
|
---
|
||||||
|
|
||||||
# Paperless OCR worker (SOLARIA)
|
# Paperless OCR worker (SOLARIA)
|
||||||
|
|
||||||
Ciężki OCR filaru KB #2 (moduł 3, `docs/kb/modules/03-paperless-ocr-worker.md`).
|
Ciężki OCR filaru KB #2 (moduł 3, `docs/kb/modules/03-paperless-ocr-worker.md`).
|
||||||
|
|
@ -108,58 +118,3 @@ miała własny, lokalny, pusty `/tmp/paperless`. Fix: NFS-owy wolumen
|
||||||
katalog `/opt/homelab/data/paperless/scratch` (już istnieje na PIHA,
|
katalog `/opt/homelab/data/paperless/scratch` (już istnieje na PIHA,
|
||||||
`chown 1000:1000`), montowany na `/tmp/paperless` po obu stronach.
|
`chown 1000:1000`), montowany na `/tmp/paperless` po obu stronach.
|
||||||
|
|
||||||
## NFS: export na PIHA, mount na SOLARIA
|
|
||||||
|
|
||||||
Transfer idzie po **LAN** (PIHA `192.168.31.5` ↔ SOLARIA `192.168.31.70`,
|
|
||||||
1 Gb/s, ten sam switch) — NIE po Tailscale. Przepustowość nie jest wąskim
|
|
||||||
gardłem OCR.
|
|
||||||
|
|
||||||
### Host-side na PIHA (NIE w compose — krok przy deployu modułu 3)
|
|
||||||
|
|
||||||
```
|
|
||||||
# /etc/exports na PIHA — export TYLKO dla SOLARII:
|
|
||||||
/opt/homelab/data/paperless 192.168.31.70(rw,sync,no_subtree_check,no_root_squash)
|
|
||||||
```
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo apt install nfs-kernel-server # jesli brak
|
|
||||||
sudo exportfs -ra
|
|
||||||
```
|
|
||||||
|
|
||||||
`no_root_squash` jest potrzebne, bo entrypoint obrazu (root) robi `chown` na
|
|
||||||
katalogach przy starcie kontenera na SOLARII; export jest ograniczony do
|
|
||||||
jednego IP w zaufanym LAN.
|
|
||||||
|
|
||||||
### Strona SOLARII
|
|
||||||
|
|
||||||
Mounty definiuje compose jako named volumes z driverem NFS — **zero wpisów
|
|
||||||
w /etc/fstab**; jedyny host-side wymóg to pakiet klienta:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo apt install nfs-common
|
|
||||||
```
|
|
||||||
|
|
||||||
### UID mapping (krytyczne)
|
|
||||||
|
|
||||||
Pliki na exporcie mają numerycznego właściciela — NFS nie tłumaczy nazw.
|
|
||||||
Dlatego `USERMAP_UID/GID=1000` jest ustawione **w obu** compose (PIHA
|
|
||||||
i SOLARIA); zmiana po jednej stronie = worker traci dostęp do plików.
|
|
||||||
Weryfikacja po deployu: `./healthcheck.sh` robi test zapisu na mount.
|
|
||||||
|
|
||||||
## Cutover checklist (przy deployu modułu 3 — po działającym module 2)
|
|
||||||
|
|
||||||
1. ✅ `services/paperless/` działa na PIHA (healthcheck zielony).
|
|
||||||
2. ✅ Export NFS na PIHA (wyżej) + `showmount -e 192.168.31.5` z SOLARII.
|
|
||||||
3. ✅ `nfs-common` na SOLARII.
|
|
||||||
4. ✅ `.env` z `env.example` — sekrety SKOPIOWANE z PIHA, nie nowe.
|
|
||||||
5. ✅ `docker compose up -d` + `./healthcheck.sh`.
|
|
||||||
6. ✅ Test (2026-07-12): PDF-y wrzucone do consume na PIHA, część odebrana i
|
|
||||||
dokończona przez worker@SOLARIA (dowód w logach, zero File not found).
|
|
||||||
7. ⬜ Test fallbacku: stop workera → zadanie czeka/mieli PIHA → start → drenaż
|
|
||||||
(jeszcze niewykonany formalnie, ale mechanizm nie zmienił się tym fixem —
|
|
||||||
fallback na PIHA działał już wcześniej, patrz sekcja "Fallback" wyżej).
|
|
||||||
8. ⬜ **OTWARTE**: wpis `paperless-worker` w `hosts/solaria/services.yaml` +
|
|
||||||
`inventory/topology.yaml` (obecnie SOLARIA ma tam tylko `node-agent`) —
|
|
||||||
bez tego supervisor/observer nie widzą tego serwisu w desired-state, więc
|
|
||||||
drift między `hosts/solaria/services.yaml` a rzeczywistością nie jest
|
|
||||||
wykrywany. Patrz `docs/backlog.md`.
|
|
||||||
|
|
@ -1,3 +1,13 @@
|
||||||
|
---
|
||||||
|
okf: "0.1"
|
||||||
|
type: service
|
||||||
|
visibility: private
|
||||||
|
status: active
|
||||||
|
updated: 2026-05-27
|
||||||
|
links:
|
||||||
|
- ../runbooks/planner-agent-deploy.md
|
||||||
|
---
|
||||||
|
|
||||||
# planner-agent
|
# planner-agent
|
||||||
|
|
||||||
Asynchroniczny agent diagnozujący zdarzenia zdrowotne w homelabowej infrastrukturze.
|
Asynchroniczny agent diagnozujący zdarzenia zdrowotne w homelabowej infrastrukturze.
|
||||||
|
|
@ -110,99 +120,6 @@ w docker-compose.yml (nie jest w env_file — sekret injektowany przez operatora
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Deployment na SOLARIA
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Przygotuj .env na solaria
|
|
||||||
ssh oskar@100.100.231.104
|
|
||||||
mkdir -p /opt/homelab/config/planner-agent
|
|
||||||
cat > /opt/homelab/config/planner-agent/.env << 'EOF'
|
|
||||||
OLLAMA_HOST=http://host-gateway:11434
|
|
||||||
OLLAMA_MODEL=qwen2.5-coder:14b
|
|
||||||
REDIS_URL=redis://100.108.208.3:6379
|
|
||||||
NODE_NAME=solaria
|
|
||||||
COOLDOWN_SECONDS=300
|
|
||||||
RUNTIME_PATH=/opt/homelab
|
|
||||||
EOF
|
|
||||||
|
|
||||||
# 2. Deploy przez standardowy skrypt (z SATURN)
|
|
||||||
scripts/deploy/deploy.sh --service planner-agent
|
|
||||||
|
|
||||||
# 3. Weryfikacja
|
|
||||||
docker ps --filter name=planner-agent
|
|
||||||
docker logs planner-agent --tail 30
|
|
||||||
ls -la /opt/homelab/state/planner-agent.heartbeat
|
|
||||||
```
|
|
||||||
|
|
||||||
> **Uwaga:** stage `verify` w deploy.sh może failować bezpośrednio po starcie
|
|
||||||
> (race condition — heartbeat jest pisany co ~5 s). Kontener jest zdrowy gdy
|
|
||||||
> `docker ps` pokazuje `(healthy)`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Uruchamianie lokalne (bez Dockera)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd services/planner-agent
|
|
||||||
|
|
||||||
# Zainstaluj zależności
|
|
||||||
pip install -r requirements.txt
|
|
||||||
|
|
||||||
# Wyeksportuj zmienne (lub stwórz .env i użyj `export $(cat .env | xargs)`)
|
|
||||||
export REDIS_URL=redis://localhost:6379
|
|
||||||
export OLLAMA_HOST=http://localhost:11434
|
|
||||||
export OLLAMA_MODEL=qwen2.5:7b
|
|
||||||
export NODE_NAME=dev-local
|
|
||||||
export RUNTIME_PATH=/tmp/homelab-dev
|
|
||||||
export COOLDOWN_SECONDS=10
|
|
||||||
|
|
||||||
# Uruchom
|
|
||||||
python src/planner.py
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Testowanie
|
|
||||||
|
|
||||||
### Testy jednostkowe
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd services/planner-agent
|
|
||||||
pip install -r requirements.txt pytest pytest-asyncio
|
|
||||||
pytest tests/ -v
|
|
||||||
# 49 testów planner + 34 testy llm_router
|
|
||||||
```
|
|
||||||
|
|
||||||
### Ręczny end-to-end test przez Redis
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Opublikuj sztuczne zdarzenie unhealthy na kanale health_events
|
|
||||||
redis-cli -h 100.108.208.3 PUBLISH health_events '{
|
|
||||||
"type": "service_unhealthy",
|
|
||||||
"node": "piha",
|
|
||||||
"service": "mosquitto",
|
|
||||||
"severity": "error",
|
|
||||||
"payload": {"exit_code": 1, "reason": "OOMKilled"},
|
|
||||||
"timestamp": "2026-05-27T20:00:00Z"
|
|
||||||
}'
|
|
||||||
|
|
||||||
# Obserwuj logi planera
|
|
||||||
docker logs planner-agent -f
|
|
||||||
|
|
||||||
# Sprawdź czy propozycja trafiła do pending
|
|
||||||
ls /opt/homelab/actions/pending/
|
|
||||||
cat /opt/homelab/actions/pending/plan-piha-mosquitto-*.json
|
|
||||||
```
|
|
||||||
|
|
||||||
### Śledzenie metryk LLM
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Subskrybuj metryki routera w czasie rzeczywistym
|
|
||||||
redis-cli -h 100.108.208.3 SUBSCRIBE llm_router_metrics
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Healthcheck
|
## Healthcheck
|
||||||
|
|
||||||
Skrypt `healthcheck.sh` sprawdza czy plik heartbeat
|
Skrypt `healthcheck.sh` sprawdza czy plik heartbeat
|
||||||
Loading…
Reference in a new issue