homelab-codex-ws/kb/services/kb-postgres.md

okf	type	visibility	status	updated	links
0.1	service	private	active	2026-06-22

kb-postgres

Postgres 16 + pgvector — KB spine on PIHA (Raspberry Pi 5, always-on). Stores the frozen envelope schema shared by all KB pillars (mails, documents, photos, transactions).

Runs here because the KB store must answer queries 24/7; SOLARIA (GPU/compute) is powered down intermittently. Embeddings/models still run on SOLARIA's GPU — only the Postgres+pgvector store lives on PIHA. The pgvector/pgvector:pg16 image is multi-arch and runs natively on arm64 (the Pi 5).

Port: 5433 on PIHA (Tailscale-accessible to other nodes).

Standard deploy (from SATURN)

# On SATURN — pushes to master, then deploy.sh SSHes to PIHA and runs deploy-node.sh
git push origin master
scripts/deploy/deploy.sh piha

deploy-node.sh on PIHA automatically picks up the per-host override:

docker compose \
  -f services/kb-postgres/docker-compose.yml \
  -f hosts/piha/runtime/kb-postgres/docker-compose.override.yml \
  up -d --remove-orphans

The PIHA override caps memory (mem_limit: 1g) and tunes Postgres for a tight, HA-shared RAM budget. Data lives in the kb_postgres_data named volume, which must land on the NVMe (Docker data-root on /home), never the SD card — verify before first deploy (see the override file's DATA PLACEMENT note):

# On PIHA
docker info -f '{{.DockerRootDir}}'                       # expect an NVMe path
df -h "$(docker info -f '{{.DockerRootDir}}')"            # confirm it's the NVMe

First-time setup on PIHA (before first deploy)

The .env file must exist at services/kb-postgres/.env in the PIHA repo checkout (alongside the compose file — that's where env_file: .env resolves to):

# On PIHA
cd ~/homelab-codex-ws
cp services/kb-postgres/env.example services/kb-postgres/.env
# Edit .env: set POSTGRES_PASSWORD to something strong

.env is gitignored (*.env rule in root .gitignore) — it will never be committed.

Manual one-off (debugging / first boot)

# On PIHA, from repo root
docker compose \
  -f services/kb-postgres/docker-compose.yml \
  -f hosts/piha/runtime/kb-postgres/docker-compose.override.yml \
  up -d

Verify after first boot

# Host-side healthcheck
./services/kb-postgres/healthcheck.sh

# Inside the container
docker exec kb-postgres psql -U kb -d kb -c '\d envelope'
docker exec kb-postgres psql -U kb -d kb \
  -c "SELECT extname FROM pg_extension WHERE extname = 'vector';"

Expected \d envelope output:

                    Table "public.envelope"
  Column  |           Type           | Nullable |  Default
----------+--------------------------+----------+-----------
 id       | text                     | not null |
 source   | text                     | not null |
 ts       | timestamp with time zone | not null |
 geo      | jsonb                    |          |
 raw_ref  | text                     | not null |
 entities | jsonb                    | not null | '[]'::jsonb
Indexes:
    "envelope_pkey" PRIMARY KEY, btree (id)
    "envelope_source_idx" btree (source)
    "envelope_ts_idx" btree (ts)

Schema contract

The envelope table is the frozen cross-source envelope (see kb/subsystems/kb-overview.md §Zasady przekrojowe). Adding columns is OK; removing or renaming existing ones is NOT.

Future migrations go in init/ as 002_*.sql, 003_*.sql, …. Postgres runs initdb scripts only on a fresh volume — for existing instances apply migrations with psql directly.

Applied so far: 001 envelope, 002 document_chunk, 003 chunk model key + excluded_reason, 004 document_summary, 005 mail_sync_state.

005_mail_sync_state.sql (2026-08-06) adds the per-folder IMAP sync cursor for jobs/mail-imap-sync, keyed (account, folder). A table rather than a file under /opt/homelab/state/ for one decisive reason: the cursor and the envelopes it describes must restore together or not at all. A state file surviving a DB restore would make the poller silently skip everything between the restored rows and the file's last_uid — a failure with no symptom. Apply it by hand on the live instance (kb/runbooks/mail-sync-run.md §4); the DDL is IF NOT EXISTS, so repeating it is safe.

Connection string

postgresql://kb:<POSTGRES_PASSWORD>@piha:5433/kb

Set KB_TEST_DSN to this value when running integration tests from packages/kb-mail/ (host = piha over Tailscale).