refactor(kb-postgres): relokacja SOLARIA→PIHA — arm64, mem_limit 1g, tuning pod małą maszynę

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
oskar 2026-06-22 19:59:58 +02:00
parent bbfbb698f8
commit 2b3cb89144
8 changed files with 149 additions and 38 deletions

View file

@ -117,7 +117,7 @@ Pierwsza biblioteka: `packages/kb-mail/` — model koperty, helpery DB (asyncpg)
~~Etap 1 maili: zamroź kopertę + postaw Postgres+pgvector + szkielet repo.~~ **ZROBIONE** (2026-06-17). ~~Etap 1 maili: zamroź kopertę + postaw Postgres+pgvector + szkielet repo.~~ **ZROBIONE** (2026-06-17).
- ✅ `services/kb-postgres` — pgvector/pgvector:pg16 na SOLARIA (:5433), `init/001_envelope.sql` - ✅ `services/kb-postgres` — pgvector/pgvector:pg16 na PIHA (:5433, always-on), `init/001_envelope.sql`
- ✅ Zamrożona koperta — tabela `envelope` + `@dataclass Envelope` (tz-aware) - ✅ Zamrożona koperta — tabela `envelope` + `@dataclass Envelope` (tz-aware)
- ✅ `packages/kb-mail``envelope` / `db` (asyncpg) / `archive` (append-only .eml) - ✅ `packages/kb-mail``envelope` / `db` (asyncpg) / `archive` (append-only .eml)
- ✅ 15 testów unit + 5 integration (mark `integration`, wymaga KB_TEST_DSN) - ✅ 15 testów unit + 5 integration (mark `integration`, wymaga KB_TEST_DSN)

View file

@ -0,0 +1,102 @@
# PIHA-specific overrides for kb-postgres (KB spine).
#
# WHY PIHA: the KB store must answer queries 24/7. SOLARIA (GPU/compute) is
# powered down intermittently; PIHA (Raspberry Pi 5, always-on, mains power) is
# the right home for an always-available spine. Embeddings/models still run on
# SOLARIA's GPU — only the Postgres+pgvector store lives here.
#
# IMAGE / ARCH: pgvector/pgvector:pg16 is multi-arch and publishes a linux/arm64
# manifest, so it runs natively on the Pi 5 (arm64) — no emulation. We do NOT
# change the pg16 tag; arch is handled by the manifest list, not the tag.
#
# RESOURCE CONTEXT: PIHA has 8 GB RAM, but ~6 GB is already resident
# (Home Assistant, Immich, monitoring). Only ~2 GB is free (+4 GB swap). This is
# a RAM-bound box shared with Home Assistant — Postgres MUST NOT starve HA.
# Everything below is sized to keep kb-postgres's resident set near ~0.50.8 GB
# under normal load, with a hard 1 GB ceiling.
services:
kb-postgres:
# Hard cgroup ceiling. ~1 GB (not the 4 GB used on SOLARIA). If Postgres ever
# exceeds this, the cgroup OOM killer restarts the container via Docker —
# Postgres recovers cleanly via crash recovery — instead of letting the host
# OOM killer pick a victim (which could be Home Assistant). 1 GB comfortably
# covers the worst-case allocation below.
mem_limit: 1g
# Soft floor for the scheduler: reserve enough that shared_buffers (256 MB)
# plus connection/backend overhead is not constantly contended under memory
# pressure, without hard-pinning a full GB away from HA.
mem_reservation: 512m
# Postgres tuning for a tight, shared RAM budget. Passed as server args so we
# need no mounted postgresql.conf. Defaults (shared_buffers 128 MB, work_mem
# 4 MB, max_connections 100) assume a dedicated box — far too loose here.
#
# shared_buffers=256MB Postgres's own page cache. ~25% of the 1 GB
# ceiling — the standard rule of thumb. Bigger
# would crowd HA; smaller hurts cache hit rate
# for the envelope + pgvector working set.
# effective_cache_size=512MB Planner hint only (allocates nothing). Tells
# the planner how much OS+PG cache it can assume
# for this DB's share of the box, so it favours
# index scans appropriately. Conservative given
# the page cache is shared with HA/Immich.
# work_mem=8MB Per-sort/hash node. With max_connections=30 the
# worst case is bounded (~30 * a few nodes * 8MB);
# keeps a runaway analytic query from blowing the
# budget. Small enough for a Pi, big enough for
# typical KB lookups.
# maintenance_work_mem=64MB For VACUUM / CREATE INDEX (incl. building the
# pgvector ivfflat/hnsw index). One-at-a-time and
# transient, so a larger value than work_mem is
# safe and speeds index builds.
# max_connections=30 KB clients are a handful of agents/jobs, not a
# web fleet. Capping at 30 bounds per-backend RAM
# (each backend ~510 MB) and the work_mem blast
# radius. Raise only if a real client count needs it.
#
# Sanity check on the ceiling: 256 MB shared_buffers + ~30 backends * ~10 MB
# overhead (~300 MB) + bounded work_mem spikes stays well under mem_limit=1g.
command:
- "postgres"
- "-c"
- "shared_buffers=256MB"
- "-c"
- "effective_cache_size=512MB"
- "-c"
- "work_mem=8MB"
- "-c"
- "maintenance_work_mem=64MB"
- "-c"
- "max_connections=30"
# ---------------------------------------------------------------------------
# DATA PLACEMENT — must land on the NVMe (/home, ~170 GB free), NEVER the SD card.
#
# The base compose uses the Docker-managed named volume `kb_postgres_data`, which
# physically lives under Docker's data-root. On PIHA, Immich already stores its
# (large) photo library in Docker volumes here, which is only possible if the
# data-root sits on the NVMe — so the plain named volume should already land on
# NVMe and is the convention used by other PIHA services (e.g. vikunja).
#
# BEFORE FIRST DEPLOY, verify on PIHA:
# docker info -f '{{.DockerRootDir}}' # expect a path on the NVMe
# df -h "$(docker info -f '{{.DockerRootDir}}')" # confirm it's the NVMe fs
#
# If (and only if) the data-root is NOT on the NVMe, pin the volume explicitly to
# an NVMe path by uncommenting the block below. The official postgres entrypoint
# runs as root and chowns PGDATA to the in-container postgres user (uid 999) on
# startup, so PIHA's host-uid 1004-vs-1000 skew does not apply to PGDATA itself —
# but the bind *device* directory must pre-exist (Docker will not create it):
# sudo mkdir -p /home/oskar/homelab-data/kb-postgres
# sudo chown 1004:1004 /home/oskar/homelab-data/kb-postgres # host owner; PG re-chowns PGDATA to 999 inside
#
# volumes:
# kb_postgres_data:
# name: kb_postgres_data
# driver: local
# driver_opts:
# type: none
# o: bind
# device: /home/oskar/homelab-data/kb-postgres
# ---------------------------------------------------------------------------

View file

@ -57,3 +57,20 @@ services:
# secrets (.env) + OIDC config (config.yml) live alongside the compose file # secrets (.env) + OIDC config (config.yml) live alongside the compose file
config_path: services/vikunja config_path: services/vikunja
# data is in Docker named volumes: vikunja_vikunja_db, vikunja_vikunja_files # data is in Docker named volumes: vikunja_vikunja_db, vikunja_vikunja_files
kb-postgres:
role: kb-database # KB spine: Postgres 16 + pgvector (always-on)
deployment_model: docker-compose
exposure: local-only # Tailscale-accessible; no public ingress
offline_required: false
depends_on:
local: []
external: []
ports:
- name: postgres
host_port: 5433
protocol: tcp
runtime:
config_path: /opt/homelab/config/kb-postgres
# data is in Docker named volume kb_postgres_data — must land on the NVMe
# (Docker data-root on /home); see hosts/piha/runtime/kb-postgres override.

View file

@ -1,6 +0,0 @@
# SOLARIA-specific overrides for kb-postgres.
# SOLARIA has 64 GiB RAM; these limits are guard-rails against runaway growth,
# not a hard budget constraint like on VPS.
services:
kb-postgres:
mem_limit: 4g

View file

@ -1,22 +1,6 @@
host: solaria host: solaria
services: services:
kb-postgres:
role: kb-database
deployment_model: docker-compose
exposure: local-only # Tailscale-accessible; no public ingress
offline_required: false
depends_on:
local: []
external: []
ports:
- name: postgres
host_port: 5433
protocol: tcp
runtime:
config_path: /opt/homelab/config/kb-postgres
# data lives in Docker named volume: kb_postgres_data
node-agent: node-agent:
role: node-stability-monitor role: node-stability-monitor
deployment_model: docker-compose deployment_model: docker-compose

View file

@ -22,6 +22,7 @@ nodes:
- ha-diag-agent - ha-diag-agent
- brain-watchdog - brain-watchdog
- vikunja # Task management (vikunja + postgres), public via npm - vikunja # Task management (vikunja + postgres), public via npm
- kb-postgres # KB spine: Postgres 16 + pgvector, port 5433 (always-on)
solaria: solaria:
roles: roles:
@ -29,7 +30,6 @@ nodes:
- ai - ai
services: services:
- node-agent - node-agent
- kb-postgres # KB spine: Postgres 16 + pgvector, port 5433
vps: vps:
roles: roles:

View file

@ -1,32 +1,45 @@
# kb-postgres # kb-postgres
Postgres 16 + pgvector — KB spine on SOLARIA. Stores the frozen envelope schema shared by all KB pillars (mails, documents, photos, transactions). Postgres 16 + pgvector — KB spine on **PIHA** (Raspberry Pi 5, always-on). Stores the frozen envelope schema shared by all KB pillars (mails, documents, photos, transactions).
Port: **5433** on SOLARIA (Tailscale-accessible to other nodes). Runs here because the KB store must answer queries 24/7; SOLARIA (GPU/compute) is powered down intermittently. Embeddings/models still run on SOLARIA's GPU — only the Postgres+pgvector store lives on PIHA. The `pgvector/pgvector:pg16` image is multi-arch and runs natively on arm64 (the Pi 5).
Port: **5433** on PIHA (Tailscale-accessible to other nodes).
## Standard deploy (from SATURN) ## Standard deploy (from SATURN)
```bash ```bash
# On SATURN — pushes to master, then deploy.sh SSHes to SOLARIA and runs deploy-node.sh # On SATURN — pushes to master, then deploy.sh SSHes to PIHA and runs deploy-node.sh
git push origin master git push origin master
scripts/deploy/deploy.sh solaria scripts/deploy/deploy.sh piha
``` ```
`deploy-node.sh` on SOLARIA automatically picks up the per-host override: `deploy-node.sh` on PIHA automatically picks up the per-host override:
``` ```
docker compose \ docker compose \
-f services/kb-postgres/docker-compose.yml \ -f services/kb-postgres/docker-compose.yml \
-f hosts/solaria/runtime/kb-postgres/docker-compose.override.yml \ -f hosts/piha/runtime/kb-postgres/docker-compose.override.yml \
up -d --remove-orphans up -d --remove-orphans
``` ```
## First-time setup on SOLARIA (before first deploy) The PIHA override caps memory (`mem_limit: 1g`) and tunes Postgres for a tight,
HA-shared RAM budget. Data lives in the `kb_postgres_data` named volume, which
**must** land on the NVMe (Docker data-root on `/home`), never the SD card —
verify before first deploy (see the override file's DATA PLACEMENT note):
The `.env` file must exist at `services/kb-postgres/.env` in the SOLARIA repo checkout ```bash
# On PIHA
docker info -f '{{.DockerRootDir}}' # expect an NVMe path
df -h "$(docker info -f '{{.DockerRootDir}}')" # confirm it's the NVMe
```
## First-time setup on PIHA (before first deploy)
The `.env` file must exist at `services/kb-postgres/.env` in the PIHA repo checkout
(alongside the compose file — that's where `env_file: .env` resolves to): (alongside the compose file — that's where `env_file: .env` resolves to):
```bash ```bash
# On SOLARIA # On PIHA
cd ~/homelab-codex-ws cd ~/homelab-codex-ws
cp services/kb-postgres/env.example services/kb-postgres/.env cp services/kb-postgres/env.example services/kb-postgres/.env
# Edit .env: set POSTGRES_PASSWORD to something strong # Edit .env: set POSTGRES_PASSWORD to something strong
@ -37,10 +50,10 @@ cp services/kb-postgres/env.example services/kb-postgres/.env
## Manual one-off (debugging / first boot) ## Manual one-off (debugging / first boot)
```bash ```bash
# On SOLARIA, from repo root # On PIHA, from repo root
docker compose \ docker compose \
-f services/kb-postgres/docker-compose.yml \ -f services/kb-postgres/docker-compose.yml \
-f hosts/solaria/runtime/kb-postgres/docker-compose.override.yml \ -f hosts/piha/runtime/kb-postgres/docker-compose.override.yml \
up -d up -d
``` ```
@ -83,7 +96,8 @@ Future migrations go in `init/` as `002_*.sql`, `003_*.sql`, …. Postgres runs
## Connection string ## Connection string
``` ```
postgresql://kb:<POSTGRES_PASSWORD>@solaria:5433/kb postgresql://kb:<POSTGRES_PASSWORD>@piha:5433/kb
``` ```
Set `KB_TEST_DSN` to this value when running integration tests from `packages/kb-mail/`. Set `KB_TEST_DSN` to this value when running integration tests from `packages/kb-mail/`
(host = `piha` over Tailscale).

View file

@ -1,6 +1,6 @@
service: service:
name: kb-postgres name: kb-postgres
owner_node: solaria owner_node: piha
exposure: local-only # reachable on Tailscale; no public ingress exposure: local-only # reachable on Tailscale; no public ingress
dependencies: [] dependencies: []
ports: ports: