kert 5930a295bd feat(llm): stack llm restamp — metadata-only anchor backfill on pgvector (refs #688) ...

Backfills codes/families/elements onto the 1.15M chunks already in
pgvector without re-embedding: anchor_metadata is a pure function of
chunk text, so it's recomputed and merged into cmetadata with a jsonb
`||`, one transaction per batch, keyset-paginated on id. Idempotent —
a row whose three keys already match the freshly computed values is
skipped; a row missing a key (the pre-backfill norm) is patched even
when the computed value is "", since the acceptance check is key
presence (`cmetadata ? 'families'`), not value equality.

`stack llm restamp --collection all|comments|rules|corpus [--batch]
[--dry-run]` mirrors `index`'s config/engine setup, runs migrate() and
ensure_hnsw() first so the GIN indexes over families/elements exist,
refreshes the derived family registry from the DuckDB replica when
present, and builds the code->family index once per run via
pfs.anchors.code_family_index.

Live backfill (rig, pgvector at LLM_PG_HOST):
  rules:  scanned=67,340  updated=67,340  (168.4s, then 12,389 on the
          A5 re-run after excluding single-code families)
  corpus: scanned=235,140 updated=235,140 (626.1s, ~376 rows/s)
  comments (~918k rows) launched in the background per Ruling A3.

2026-09-09 16:04:15 -04:00

1.5 KiB

Raw Blame History

View Raw