50 lines
1.6 KiB
Markdown
50 lines
1.6 KiB
Markdown
|
---
|
||
|
|
okf: "0.1"
|
||
|
|
type: runbook
|
||
|
|
visibility: private
|
||
|
|
status: active
|
||
|
|
updated: 2026-07-14
|
||
|
|
links:
|
||
|
|
- ../services/job-gmail-header-backfill.md
|
||
|
|
---
|
||
|
|
|
||
|
|
# gmail-header-backfill — uruchomienie i testy
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Dry run (default) — parse and count only, no DB writes:
|
||
|
|
gmail-header-backfill --dsn postgresql://kb:<pw>@localhost:5433/kb --limit 100
|
||
|
|
|
||
|
|
# Real run — apply the UPDATE for this slice:
|
||
|
|
gmail-header-backfill --dsn ... --limit 1000 --offset 0 --apply
|
||
|
|
|
||
|
|
# Next slice — offset is stable/deterministic (ORDER BY id), independent of
|
||
|
|
# how many rows in earlier slices were already backfilled:
|
||
|
|
gmail-header-backfill --dsn ... --limit 1000 --offset 1000 --apply
|
||
|
|
```
|
||
|
|
|
||
|
|
DSN can also come from the `KB_DSN` env var instead of `--dsn`.
|
||
|
|
|
||
|
|
`--limit`/`--offset` exist so the full 225 030-row backfill can be run in
|
||
|
|
verifiable partitions instead of one long unattended run (plan §5.3) — start
|
||
|
|
small (`--limit 100`), check the result in the DB, then widen.
|
||
|
|
|
||
|
|
## Tests
|
||
|
|
|
||
|
|
```bash
|
||
|
|
pip install -e jobs/gmail-header-backfill/
|
||
|
|
cd jobs/gmail-header-backfill && pytest
|
||
|
|
```
|
||
|
|
|
||
|
|
Pure unit tests, no DB or filesystem outside `tmp_path`/synthetic `.eml`
|
||
|
|
bytes — `run()` is tested by monkeypatching `asyncpg.connect` with an
|
||
|
|
in-memory fake connection. Covers: header parsing (multi-address `To`/`Cc`,
|
||
|
|
quoted display names with commas, multiple `Delivered-To` occurrences, RFC
|
||
|
|
2047 encoded-words including Polish diacritics, malformed encoded-words that
|
||
|
|
must not raise, missing/multiple `From`, `date_raw` preserving literal text
|
||
|
|
vs. `Date` header reformatting), idempotency (rows already carrying a
|
||
|
|
`headers` entity are skipped and never re-appended), batch flushing, and
|
||
|
|
`--limit`/`--offset` query shape.
|
||
|
|
|