llm: chat and index use the derived code families (pfs.code_family) — refresh at service start, scale detect_codes/family_of, curate synonyms #699
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: homelab/stack#699
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Follow-up from the P49 slice-1 whole-branch review (spec docs/superpowers/specs/2026-09-09-code-family-longitudinal-design.md §Components
pfs/families.py).Gap. Slice 1 derives families into
pfs.code_familyand exposespfs.families.refresh_from(con)(in-place refresh of the liveFAMILIESdict, hand families kept as a minimum), but the only callers are thestack pfsCLI.llm/evidence.py(detect_codes) andllm/chunk.py(find_codes) still see the five hand families, so the chat never benefits from the derived registry. The spec's original wording (load at import when the replica is present) conflicts with the global constraint thatpfs.*must not touch DuckDB at import; the module docstring now says so.Work.
refresh_from(con)atllmservice start (the replica handle inllm/evidence.pyalready exists) and on replica mtime change.detect_codescompiles one regex per synonym per family (≈32k with ~16k derived families) andfamily_ofis a linear scan — build a code→family index and a single alternation regex (or a trie) at refresh time.GENE,ADM,REVERSEmust not become chat triggers; only hand families and reviewed derived families get synonyms (asynonymscolumn or an allow-list), the rest match by code only.Blocks: #691 (longitudinal chat).
Measurements from the P49 slice-2 whole-branch review (2026-09-09) that this issue must absorb:
refresh_from),detect_codescosts 0.50 s per call: it compiles one regex per synonym per family andfamily_ofis a linear scan. Precompile a single alternation at refresh time and build a code→family dict.load_familiesleavessynonyms=()for non-hand families); matching derived families by name is this issue's job — gate on ≥3-word titles or a curated allow-list, never bare heading words.refresh_fromis wired into the llm service here.Items 1–2 done in
da345bc: refresh_from runs whenever the chat opens or reopens the replica (llm.evidence._connect, plus warm() at service start); detect_codes uses a trie-compiled phrase alternation (name phrases only for CPT-named families ≥ 12 chars / ≥ 2 words) and a code→family index (family_of O(1)); detect_codes ~0.01–0.05 ms on the 8,518-key registry. New: Detection.wide and MAX_FAMILY_EXPAND=20 (a family with more codes is listed but does not expand its codes), per-turn cap chat_codes_max=24. Item 3 (synonym curation) remains open; also open: readers of FAMILIES do not take the registry lock (the swap is update-then-delete, so a reader sees old ∪ new, never empty).Closing: items 1–2 shipped in
da345bc(refresh_from on every replica open + warm() at startup; trie-compiled phrase regex and code→family index, detect_codes ~0.01–0.05 ms on 8,518 keys). Item 3 (synonym curation) is addressed by Ruling B1 rather than a curated list: a derived family's name is a chat trigger only when it is a real CPT heading (Family.cpt) of ≥ 12 characters and ≥ 2 words, so GENE/ADM/REVERSE-style keys never fire; stem-derived families are reachable by member code only. Wide families (> 20 codes) are listed but do not expand their codes (Detection.wide). If a curated synonym list is still wanted later, open a fresh issue against pfs.families.HAND_FAMILIES.