3755d7d6bb
Four audit findings (2026-07-14), each reproduced on crafted mboxes; the Takeout corpus has proven 8-bit header bytes, so all are real re-import risks. 1. Whole-run crash on 8-bit Message-ID. compat32 .get() returns an email.header.Header (not str) for raw 8-bit bytes; the old Header.strip() raised AttributeError. _message_id ran BEFORE the per-message try, so one bad header killed the entire import. Fix: str() + sanitize_surrogates() before strip; and move _message_id/_parse_date/_parse_attachments INSIDE the per-message try — a broken message is now errors += 1, never run death. 2. Poison batch. pending.clear() ran only AFTER a successful insert, so a failed flush (DB down / bad row) left pending intact and every later message re-flushed the doomed batch; the final flush sat in try/finally with no except and propagated out, losing all stats. Fix: _flush always clears pending and counts a failed insert as db_insert_failed; the run always reaches import_complete. 3. Stats didn't reconcile with the DB. imported counts archive writes, not DB rows, so a partial-insert drift was invisible. Fix: separate db_inserted/db_insert_failed counters; main() exits non-zero on any error, DB drift, or a processed = imported + skipped + errors imbalance. 4. 8-bit Date → needless epoch_fallback. parsedate_to_datetime(Header) raised even when str(header) parses fine. Fix: str() before the epoch fallback. Shared helper: _sanitize moved from gmail-header-backfill into packages/kb-mail (kb_mail.text.sanitize_surrogates) and used by both jobs; gmail-header-backfill now depends on kb-mail. Tests: regression coverage for all four findings in gmail-bulk-import (8-bit id/date, per-message guard, failed-insert non-poisoning, stats balance) plus kb_mail.text unit tests. Full suites green: kb-mail 27, gmail-bulk-import 33, gmail-header-backfill 43. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .claude/skills | ||
| backups/zigbee | ||
| docs | ||
| dotfiles | ||
| hosts | ||
| inventory | ||
| jobs | ||
| packages/kb-mail | ||
| scripts | ||
| services | ||
| .codex | ||
| .gitignore | ||
| CLAUDE.md | ||
| codex_context | ||
| codex_context.yaml | ||
| deploy_agent.py | ||
| ollama_client.py | ||
| README.md | ||
| start-aider.sh | ||
| start-codex.sh | ||
| sync-context.sh | ||
| tech-debt.md | ||
| update-context.md | ||
Homelab Codex
GitOps-lite orchestration for a distributed homelab environment.
Architecture
The homelab consists of several nodes connected via a Tailscale internal mesh.
| Host | Role | Description |
|---|---|---|
| SATURN | Primary Node | Development, orchestration, and git source of truth (commit node). |
| SOLARIA | Compute Node | GPU, inference, and heavy compute workloads. |
| PIHA | Infra Node | Core infrastructure services, automation, and monitoring. |
| VPS | Edge Node | Public ingress, reverse proxy, and edge services. |
Agent System
The homelab uses a multi-agent orchestration model with human-in-the-loop for destructive actions:
| Agent | Node | Role |
|---|---|---|
| stability-agent | all nodes | Per-node watchdog — monitors Docker, disk, Tailscale, MQTT; emits events |
| node-agent | all nodes | Publishes container health events to Redis pub/sub |
| observer | VPS | Synthesizes world state from events into /opt/homelab/world/*.json |
| supervisor | VPS | Detects drift between desired and actual state; writes pending actions |
| planner-agent | SOLARIA | LLM-powered diagnosis — listens to Redis, proposes remediation actions |
| executor | VPS | Executes actions only after operator approval |
| operator-ui + telegram-bot | VPS / PIHA | Operator reviews and approves/rejects pending actions |
Action approval flow: pending/ → operator approves → approved/ → executor runs.
Repository Structure
docs/: Infrastructure Standards and Deployment Conventions.hosts/: Host-specific configurations and service assignments.services/: Reusable Docker Compose service definitions.scripts/: Deployment and management scripts.
Getting Started
- Standardization: Follow the Infrastructure Standards.
- Deployment: See Deployment Conventions for how to roll out changes.
- SATURN: Remember that SATURN is the only node where commits should be made.
Documentation Index
- Infrastructure Standards
- Agent Operating Procedures (For AI/Non-Human Agents)
- Deployment Conventions
- Hardware
- Networking
- Services
- Node Capabilities
- Action Model
Note: This repository documents the state of the homelab. Runtime state lives outside the repository in /opt/homelab.