Adds documents-ingest-paperless: paginated GET /api/documents/, maps each
doc to a source='paperless' envelope per plan §4.2-4.3, reusing
kb_mail.Envelope/insert_envelope unchanged (packages/kb-mail not touched).
Cross-source link (source_mail entity) is a deterministic join of
original_file_name against the phase-1 registry.json consume_name index —
no heuristics, no correspondent guessing (plan decision 4). Stats always
balance (fetched = already_in_db + inserted + errors) and main() now also
exits non-zero on imbalance, not just on errors>0, matching the exit-code
convention already established in gmail-bulk-import.
Verified live on PIHA (rsync to /tmp, ~/kb/venv, PIHA checkout untouched):
dry-run then --apply inserted 186/186 paperless envelopes (0 errors,
180 source_mail links), a second --apply reported inserted=0/already_in_db=186
(idempotent), gmail rows stayed at 225030 and document_chunk stayed empty.
Rotated the kb-ingest Paperless API token after it was accidentally
partially echoed during recon (old token now dead).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Module 5 phase 1 (docs/kb/modules/05-documents-ingest.md): pulls a sample of
PDF attachments (>50KB, last year, LIMIT 150) out of the Gmail .eml archive
and drops them into Paperless consume/ for OCR, so the RAG layer has real
documents to work with before the full Paperless/Nextcloud envelope adapter
is built. sha256-first attachment matching (manifest filenames can carry raw
RFC 2047 encoded-word artifacts that don't byte-match what email.policy.default
decodes today — confirmed against live data, ~10% of candidates were affected).
Idempotent via a sha256-keyed JSON registry; dry-run by default, --apply to write.
Verified end-to-end on PIHA: dry-run + --apply both run against live
kb-postgres/archive, 185/222 candidate PDFs written to consume/ (37 in-run
duplicates correctly deduped), Paperless picked them up and started OCR
immediately.
