Files
stack/docs/superpowers/plans/2026-09-10-code-family-longitudinal-chat.md

28 KiB
Raw Blame History

Code-family longitudinal chat + golden evaluation — Implementation Plan (P49 slice 3)

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: A "history of X" question in the LLM chat gets a dated, anchored timeline for the detected codes and families (a lineage SSE event and a prompt block the model must cite), era-balanced retrieval so CY2015 and CY2027 both surface, the CPT manual's own heading text as a source, and a golden question set that runs nightly against the live chat and files an issue on regression — for ANY PFS-payable code, not only the five hand families.

Architecture: Mirrors P48's valuation flow. llm/evidence.py gains lineage_evidence(question, cfg) which reads pfs.code_event, pfs.code_element and pfs.code_guidance from the cached read-only replica handle (falling back to on-demand pfs.lineage.lineage(...) for codes with no precomputed rows), returns a payload for the SSE lineage event and a "Lineage" prompt block whose rows carry bracketed labels ([CY2021 PFS final ¶1578]). rag.retrieve(..., mode="timeline") groups hits by rule year and caps per era. chat.html renders the timeline between the answer and the sources drawer. stack pfs lineage --all-payable --write precomputes events for every A/R/T code with one inverted pass over fr_anchors. tests/llm/golden_lineage.yaml + dev/scripts/llm_golden.py check anchors, cited labels and forbidden claims against the live /chat endpoint, nightly, through the existing issue filer.

Tech Stack: Python 3.13, FastAPI SSE, DuckDB replica (read-only cached handle, per-call cursor), SQLite bib (fr_anchors, items), pgvector via SQLAlchemy text(), Ollama self-hosted pool, marimo notebook, Gitea Actions workflows generated by dev/scripts/gen_config.py from stack.toml.

Spec: docs/superpowers/specs/2026-09-09-code-family-longitudinal-design.md (§Decisions 56, §Components "longitudinal chat", §Evaluation). Tracker: #691 (chat), #692 (eval), #699 items 12 (derived families in chat), #705 item 1 (CPT manual as a chat source, Ruling A9 of the anchors slice).

Global Constraints

  • Self-hosted inference only (Ollama pool from config); no cloud LLM API anywhere in src/llm or src/pfs.
  • pfs.* modules never touch DuckDB at import time; the chat reads the replica through llm/evidence.py's cached handle (_connect, mtime-keyed reopen, per-call .cursor()); writes only through conf.connect.duckdb_batch + publish_replica, never inside slow work (build first, then open the batch).
  • Control question is byte-identical: for a question with no detected code, the SSE event sequence and every event payload are unchanged from main 4616d36 (token* → sources → done); for a coded question without history words the sequence is valuation? → token* → sources → done exactly as today plus the new lineage event first: lineage? → valuation? → token* → sources → done.
  • Evidence budget: lineage_evidence adds < 300 ms wall-clock on the live replica for ≤ 3 codes with precomputed rows; on-demand fallback runs for at most 3 codes per question and is skipped (logged) beyond that.
  • SSE payloads are JSON-serialisable dicts with a type key; the API wraps them as data: {json}\n\n (no event: field).
  • Bracketed labels in prompt blocks and payloads use the exact same string; the system prompt tells the model to cite them verbatim.
  • Every FR anchor link is built from (item_key, p_id) through bib.frlink (_para_link via resolve/place), eCFR links through bib.cfrlink.url.
  • Tests: no live DB or Ollama required (fakes as in tests/llm/test_rag.py and tests/llm/test_evidence.py); live checks skip without LLM_DB_PASSWORD / LLM_CHAT_URL.
  • Commit messages: conventional prefix, (refs #NNN); never add a Co-Authored-By or any trailer.
  • Docs: regenerate docs/docs/cli/ fully after CLI changes (uv run python docs/scripts/extract_cli.py), commit every changed page.

Task 1: Derived families in the chat (#699 items 12)

Files:

  • Modify: src/pfs/families.py (detect_codes, family_of, a compiled index built by refresh_from and on FAMILIES mutation), src/llm/evidence.py (_connect → call refresh_from when the replica (re)opens)
  • Test: tests/pfs/test_families.py, tests/llm/test_evidence.py

Interfaces:

  • Consumes: pfs.families.FAMILIES, Family(key, name, codes, synonyms), refresh_from(con) -> int, detect_codes(text) -> Detection(codes, families, explicit), code_family_index(families) in pfs.anchors.
  • Produces: pfs.families.rebuild_index() -> None (called by refresh_from and at import after HAND_FAMILIES seeding); detect_codes unchanged signature, same results for the five hand families (regression fixture), now also detecting derived families by member code and by name phrase (see rule below); family_of(code) -> Family | None O(1) via the code→family index (first key in sorted order when a code belongs to several families, hand families first); llm.evidence._connect(path) refreshes FAMILIES from pfs.code_family whenever it opens or reopens the replica (mtime change), logging the family count.

Rules (binding):

  • Family detection by synonym stays for hand families. A derived family is detected by name only when its name is ≥ 12 characters and ≥ 2 words and appears as a case-insensitive whole-phrase match; single-word CPT headings (GENE, ADM, REVERSE, Introduction) never match by name. Detection by member code applies to every family.

  • One compiled alternation regex over all synonym/name phrases (longest first, word-bounded) plus a dict[code, tuple[key, ...]] index, rebuilt in rebuild_index(); detect_codes must run in < 5 ms on a 300-character question with the live 8,518-key registry (measure in a test marked live, skipped without the replica).

  • refresh_from is idempotent and thread-safe enough for the chat: it builds the new dict/index first and swaps them under _REGISTRY_LOCK.

  • Step 1: Failing teststests/pfs/test_families.py: test_detect_derived_family_by_member_code (a fake FAMILIES entry SUTURE-REMOVAL with codes 15850 15851; "removal of sutures 15850" → families contains SUTURE-REMOVAL), test_derived_family_name_phrase_requires_two_words (GENE name never matches; "Chronic Care Management Services" matches by name), test_family_of_is_indexed (monkeypatch FAMILIES with 5,000 synthetic families; 10,000 family_of calls < 0.2 s), test_hand_families_regression (the existing five-family detections unchanged — reuse existing assertions). tests/llm/test_evidence.py: test_connect_refreshes_families_on_open_and_reopen (patch llm.evidence.refresh_from; assert called once on first _connect, not on a cached call, again after _mtime changes).

  • Step 2: Run, confirm failures.

  • Step 3: Implement rebuild_index, index-backed family_of, phrase regex in detect_codes, the refresh_from swap, and the _connect hook (refresh_from(con) inside the lock after a successful open; exceptions logged, never raised).

  • Step 4: Run uv run pytest tests/pfs/test_families.py tests/llm -q -p no:cacheprovider; ruff.

  • Step 5: Live check (env wrapper): uv run python -c "from pfs.families import detect_codes, FAMILIES, refresh_from; import duckdb; refresh_from(duckdb.connect('data/replica/aco.ro.duckdb', read_only=True)); import time; t=time.perf_counter(); d=detect_codes('history of chronic care management services and 99490'); print(len(FAMILIES), d, (time.perf_counter()-t)*1000, 'ms')" — report the ms.

  • Step 6: Commit feat(pfs,llm): derived families reach the chat — indexed detect_codes/family_of, refresh on replica open (refs #699).


Task 2: Precompute lineage for every payable code — stack pfs lineage --all-payable --write

Files:

  • Modify: src/pfs/lineage.py (inverted FR pass), src/cli/pfs.py (lineage command gains --all-payable), src/pfs/codetables.py (write_events_many(con, by_code) bulk writer if not present)
  • Test: tests/pfs/test_lineage.py, tests/cli/test_pfs_cli.py

Interfaces:

  • Consumes: lineage(con, store, code) -> list[EventRow], rvu_events, fr_events(store, code), cpt_events, pfs.families.find_codes, codes_in/code_pattern, pfs.valuation/pfs.rvu for the A/R/T universe (the same query stack pfs elements --all-payable uses — reuse its helper), write_events/read_all_events.
  • Produces: fr_events_bucketed(store, codes: Sequence[str]) -> dict[str, list[EventRow]] — ONE pass over fr_anchors joined to items (SQL prefilter: text LIKE any target code is too wide for 8.7k codes → instead stream all rows ordered by item_key, p_id and bucket with find_codes(text) ∩ targets), then the existing per-code event-verb logic applied to each code's bucket (factor the verb/anchor logic out of fr_events into _events_for(code, rows) so fr_events(store, code) == fr_events_bucketed(store, [code])[code]); lineage_all(con, store, codes) -> dict[str, list[EventRow]] merging rvu/fr/cpt per code with the same ±1-year anchoring; CLI stack pfs lineage --all-payable [--write] [--limit N] printing counts per source and unanchored count, writing per code (delete-then-insert per code inside one batch, opened only after the build — Ruling A13).

Rules: equality test fr_events_bucketed(store, [c])[c] == fr_events(store, c) for the seven fixture codes on the sqlite fixture; the full pass must finish in < 10 minutes on the live bib (193k paragraphs) — report the time; --limit N takes the first N target codes for smoke runs.

  • Step 1: Failing tests — bucketed == per-code equality on a fixture store with three rules and four codes; lineage_all marks rvu events anchored when an FR/CPT event is within ±1 year; CLI --all-payable --limit 3 --write calls the writer once per code after the build and publishes (indirections monkeypatched, call order asserted); --all-payable without --write never opens the batch.
  • Step 2: Run, confirm failures.
  • Step 3: Implement.
  • Step 4: Run uv run pytest tests/pfs/test_lineage.py tests/cli/test_pfs_cli.py -q -p no:cacheprovider; ruff.
  • Step 5: Live (env wrapper): stack pfs lineage --all-payable --limit 50 (dry) to time the pass, then stack pfs lineage --all-payable --write — report codes written, events per source, unanchored share, wall time; confirm pfs.code_event now has rows for G2211 and 99441 on the replica.
  • Step 6: Commit feat(pfs): lineage for every payable code — one inverted pass over fr_anchors, stack pfs lineage --all-payable (refs #691 #698).

Task 3: lineage_evidence — events, element diffs, guidance; SSE payload and prompt block

Files:

  • Create: src/llm/lineage.py
  • Modify: src/llm/evidence.py (export), src/llm/rag.py (stream_answer, build_messages, _SYSTEM), src/llm/config.py + stack.toml (lineage_max_rows = 25, lineage_on_demand_max = 3)
  • Test: tests/llm/test_lineage.py, tests/llm/test_rag.py

Interfaces:

  • Consumes: detect_codes, _connect(path)/_replica_path(cfg) from llm/evidence.py, read_events/read_all_events, read_elements, read_guidance, pfs.lineage.lineage(con, store, code) (on-demand fallback; the bib Store from conf.connect.bib()), pfs.descriptors.rule_year_of, bib.frlink.resolve(ref, store=..., item_key=...) with ref=f"p-{p_id}" for the FR URL, bib.cfrlink.url(parse_cite(locator)) for CFR guidance rows, Family.name.
  • Produces:
    • @dataclass(frozen=True) LineageEvent(code, year, kind, from_codes, to_codes, label, item_key, p_id, page, url, source, anchored, note).
    • @dataclass(frozen=True) LineageEvidence(codes, families, events: tuple[LineageEvent,...], element_diffs: tuple[ElementDiff,...], guidance: tuple[GuidanceRef,...]) with .payload() -> dict ({"type": "lineage", "codes", "families", "events": [asdict...], "element_diffs": [...], "guidance": [...]}) and .prompt_block() -> str.
    • ElementDiff(type, value, in_codes: tuple[str,...], not_in_codes: tuple[str,...], label, item_key, p_id) — for a detected family (or ≥ 2 detected codes) the element values present for some codes and absent for others, from pfs.code_element (newest year per code).
    • GuidanceRef(kind, locator, url, label, item_key_src, p_id_src) from pfs.code_guidance for the detected families (≤ 8 rows, CFR first, deduped by locator).
    • rule_label(title: str, date_published: str, p_id: int) -> str"CY2021 PFS final ¶1578" / "CY2027 PFS proposed ¶394" (proposed when the title contains "Proposed"; year via rule_year_of); CPT events label "CPT Changes 2022"; rvu-only events "PFS CY2020 RVU file".
    • lineage_evidence(question, cfg) -> LineageEvidence | NoneNone when no codes detected; events from read_events per code (≤ lineage_on_demand_max codes fall back to pfs.lineage.lineage(...) when a code has no rows, cached in-process by (replica mtime, code)); events collapsed to one row per (year, kind, from_codes, to_codes, code) preferring anchored FR rows, sorted by year then code, capped at lineage_max_rows for the prompt block (payload carries all collapsed rows); never raises (log + None).
    • build_messages(question, sources, evidence=None, lineage=None) inserts the lineage block after the excerpts and before the valuation block; _SYSTEM gains one paragraph: the Lineage section is authoritative for dates/predecessors/successors; cite its bracketed label for every dated claim; never invent a year without a label.
    • stream_answer yields lineage.payload() first (before valuation) when lineage_evidence returns non-None; when the lineage detected families, its codes are unioned into the code_cited_sources call's codes.

Rules: payload/prompt labels identical strings; the prompt block format is one line per event [label] YEAR KIND CODE (from → to) — note and one line per element diff [label] element type=value: in 99490, 99491; not in G0556; guidance lines [label] 42 CFR 410.78(a)(3) — cited by ….

  • Step 1: Failing teststests/llm/test_lineage.py: build events from a real temp DuckDB (pfs.code_event/code_element/code_guidance DDL via ensure_tables) with the 99490/G2058/99439 fixture rows above → collapsed row order, labels (CY2021 PFS final ¶1578 from a fake Store returning the rule title/date), element diff for CCM vs APCM fixture elements, guidance refs with CFR URLs; on-demand fallback called only for codes without rows and at most lineage_on_demand_max; payload() JSON-serialisable; prompt_block() exact text on a two-event fixture; None for a control question. tests/llm/test_rag.py: event order lineage → valuation → token* → sources → done; control question events byte-identical to a recorded baseline (json.dumps of every event with a fixed fake pool, compared against the same run with lineage_evidence patched to None); build_messages places the block correctly.
  • Step 2: Run, confirm failures.
  • Step 3: Implement.
  • Step 4: Run uv run pytest tests/llm -q -p no:cacheprovider; ruff.
  • Step 5: Live (env wrapper): time lineage_evidence("history of CCM coding and payment", cfg) ×5 on the live replica — report ms and the collapsed event rows; run "what replaced G2058 and why" and "does 99490 pass telehealth step 3" (elements present?).
  • Step 6: Commit feat(llm): lineage evidence — timeline events, element diffs and guidance as a lineage SSE event + cited prompt block (refs #691).

Task 4: Era-balanced retrieval — retrieve(..., mode="timeline"), trigger, API mode

Files:

  • Modify: src/llm/rag.py (retrieve, stream_answer(mode=...), is_history_question), src/llm/api.py (ChatRequest.mode), src/llm/config.py + stack.toml (timeline_per_era = 2, timeline_overfetch = 6)
  • Test: tests/llm/test_rag.py, tests/llm/test_api.py

Interfaces:

  • Consumes: _hits, filter_since, blend, Hit metadata (kind, title, date, docket), rule_year_of.
  • Produces: HISTORY_RE (history, evolution, "when did", "when was", replaced, predecessor, successor, timeline, "over the years", "since 20"); is_history_question(q) -> bool; era_of(hit) -> int (rules: rule_year_of(title, date); comments: docket year from docket → the rule year the docket belongs to when Store.dockets() maps it, else date[:4]; corpus: date[:4], 0 when unknown); era_balance(hits, *, per_era, top_n) -> list[Hit] — group by era, take the best per_era per era (score order), then fill to top_n by score from the remainder; retrieve(..., mode="auto"|"timeline"|"recent")timeline overfetches timeline_overfetch × k per kind, skips the recency blend and applies era_balance; auto = timeline when is_history_question(question) else recent (today's behaviour); stream_answer(..., mode="auto"); ChatRequest.mode: Literal["auto","timeline","recent"] = "auto", validated (400 otherwise).

Rules: with mode="recent" the output is byte-identical to main for every question; era_balance is deterministic (stable sort by (-score, label)).

  • Step 1: Failing testsis_history_question positives/negatives; era_of for the three kinds; era_balance picks CY2015 and CY2027 hits over three CY2026 hits with higher scores; retrieve(mode="timeline") uses the overfetch and skips blend (patch and assert); mode="recent" path unchanged (existing tests untouched); API accepts mode, rejects mode="x" with 400, forwards to stream_answer.
  • Step 24: run/implement/run; ruff.
  • Step 5: Live (env wrapper): retrieve("history of chronic care management payment", cfg=..., pool=..., mode="timeline") — report the eras represented vs mode="recent".
  • Step 6: Commit feat(llm): era-balanced retrieval for history questions — retrieve(mode="timeline"), auto trigger, ChatRequest.mode (refs #691).

Task 5: The CPT manual as a source; item_key/p_id on source dicts

Files:

  • Modify: src/llm/evidence.py (manual_sources), src/llm/links.py (as_source adds item_key, p_id, seq, section), src/llm/rag.py (merge after cited sources)
  • Test: tests/llm/test_evidence.py, tests/llm/test_links.py

Interfaces:

  • Consumes: FamilyRow.note = the CPT heading path (via pfs.codetables.read_family_rows or the pfs.code_family table: select distinct note from pfs.code_family where key = ?), pfs.cpt_years(con) newest edition and its item_key (pfs.cpt_section.item_key), corpus chunk metadata section (markdown heading) and item_key; _collect/text() binding style from code_cited_sources.
  • Produces: manual_sources(engine, con, families, *, per_family=1) -> list[dict] — for each detected family with a CPT heading, the corpus chunk(s) of the newest edition whose cmetadata->>'section' equals the heading's last path segment (case-insensitive; fall back to ILIKE '%<title>%'), first seq, labelled by as_source with title = "CPT <year> — <heading title>"; merged after the cited sources in stream_answer (dedupe by label; counts toward code_cited_max? No — manual rows are added after the cap, at most one per family); as_source carries item_key, p_id (rules), seq (others) and section so the eval can match anchors.

Rules: the P48 test_serves_chat_page… style assertions and every existing as_source test keep passing (new keys are additive); a family without a CPT heading yields nothing; SQL parameterised.

  • Step 1: Failing tests — fake engine returns a corpus row for CCM's heading → one source with kind="corpus", title starting CPT 2024 — ; no heading → empty; as_source new keys; stream_answer merges manual rows after cited rows (order asserted).
  • Step 24: run/implement/run; ruff.
  • Step 5: Live (env wrapper): manual_sources(engine, con, ["CCM"]) returns the GQGTPGYV chunk whose section is "Chronic Care Management Services" — print label/section/seq.
  • Step 6: Commit feat(llm): the CPT manual's heading text as a chat source; item_key/p_id on sources (refs #691 #705).

Task 6: UI — timeline table, element diffs, guidance, mode toggle

Files:

  • Modify: src/llm/web/chat.html
  • Test: tests/llm/test_api.py (TestIndex marker strings)

Interfaces:

  • Consumes: the lineage payload (Task 3), ChatRequest.mode (Task 4), the renderValuation/renderSources DOM pattern and .cite painting.

  • Produces: renderLineage(wrap, ev)<table class="lineage"> (Year, Event, Codes, Source) one row per event, Source cell = <a href=url>[label]</a> (span when url empty), rows with anchored=false get class unanchored and a title tooltip; below it a compact <ul class="elements"> of element diffs (type=value — in …; not in …) and a <ul class="guidance"> with CFR/IOM links; a <label class="mode"><select id="mode">auto/timeline/recent</select></label> beside the "last 12 months" checkbox, sent as mode in the POST body; the ask() dispatch gains else if (ev.type === 'lineage') renderLineage(wrap, ev) before the valuation branch; CSS table.lineage, .lineage-wrap, tr.unanchored.

  • Step 1: Failing testtest_serves_chat_page_with_lineage_renderer: "function renderLineage", "ev.type === 'lineage'", "table.lineage", "id=\"mode\"", "tr.unanchored" in the served HTML.

  • Step 24: run/implement/run.

  • Step 5: Headless checkuv run python -c that serves the page through the FastAPI TestClient and greps the markers; if playwright is available in the venv, load the page and post a canned SSE (skip otherwise, say so).

  • Step 6: Commit feat(llm): timeline table, element diffs and guidance in the chat UI; mode selector (refs #691).


Task 7: Golden evaluation — tests/llm/golden_lineage.yaml, runner, nightly workflow, filer

Files:

  • Create: tests/llm/golden_lineage.yaml, dev/scripts/llm_golden.py, tests/llm/test_golden.py
  • Modify: stack.toml ([ci.llm_golden] or the existing ci section the generator reads), dev/scripts/gen_config.py (emit .gitea/workflows/llm-golden.yml mirroring notebooks-integration.yml: docker exec llm …, --file-issues, report artifact, failure filer), regenerated .gitea/workflows/llm-golden.yml
  • Test: tests/llm/test_golden.py, tests/dev/test_gen_config.py (if such tests exist; otherwise a snapshot test that the generated workflow contains the docker exec line)

Interfaces:

  • Consumes: /chat SSE (data: {json} lines; events lineage, valuation, token, sources, done, error), source dicts with item_key/p_id/seq/label (Task 5), dev/scripts/nb_issue_filer.py (signature, cmd_report, MARKER, decide) — import it, do not copy it.

  • Produces:

    • YAML schema, one entry per question: id, question, mode (default auto), expect_anchors: [{item_key, p_id}] (must appear in sources), expect_labels: [regex] (must be cited in the answer prose, e.g. CY2021 PFS final ¶1578), forbid: [{pattern, unless_label: regex}] (e.g. pattern: "\\$\\d", unless_label: "Addendum B"), expect_events: [{code, kind, year}] (must be in the lineage payload), min_eras: 3 (distinct rule years among sources).
    • Seed entries (from #692, anchors verified on the corpus): (1) history of CCM coding and payment → DE2VH9PD ¶12491251, YBM4IZUS ¶1578, JJ6AM5HJ ¶1163; events 99490 created 2015, G2058 replaced_by 2021, G0556 created 2025; (2) what replaced G2058 and why → YBM4IZUS ¶1578, ¶2369; (3) how do APCM's elements differ from CCM's → JJ6AM5HJ ¶11641185 vs DE2VH9PD ¶12451247; (4) when were audio-only E/M codes payable and why did 9944199443 end → XFGGRBDH ¶489, event 99441 deleted|disappeared 2025; (5) what did commenters say about G2211 in 2023 vs 2025 → sources from dockets CMS-2023-0121 and CMS-2025-0304 (expect_dockets); (6) does 99490 pass telehealth Steps 13 → 2KVJ2HKX ¶394/¶396/¶398; (7) how did G2064/G2065 → 99424/99426 change the RHC/FQHC G0511 rate → JE7KYBW3 ¶1100/¶1111, MZ24MX5S ¶1305.
    • dev/scripts/llm_golden.py run --url http://llm:8000 --set tests/llm/golden_lineage.yaml --report report.json [--file-issues --source nightly-llm-golden] [--only id] — streams each question, evaluates the checks (check_anchors, check_labels, check_forbidden, check_events, check_eras, check_dockets — pure functions over the collected events, unit-tested), prints a per-question table, exit 1 on any failure, files/sweeps issues through nb_issue_filer with signature(question_id, check_name, detail).
    • tests/llm/test_golden.py: the checkers against a canned SSE transcript (pass and fail cases); YAML loads and every entry has the required keys; a live test skipped unless LLM_CHAT_URL is set that runs entry (2) and asserts pass.
    • Workflow: nightly 40 3 * * * after notebooks-integration, docker exec llm uv run python /tmp/golden/llm_golden.py run --url http://localhost:8000 … (copy the script and YAML in with docker cp like the notebooks job), GITEA_TOKEN from secrets.DEPLOY_TOKEN, failure filer step identical to the notebooks job.
  • Step 1: Failing tests (checkers, YAML shape, gen_config snapshot).

  • Step 24: run/implement/run; uv run python dev/scripts/gen_config.py regenerates workflows (commit them).

  • Step 5: Live (env wrapper; the chat service must be reachable — check LLM_CHAT_URL or the compose service http://localhost:8000; if unreachable, run the checkers against a transcript captured with stream_answer directly and say so): uv run python dev/scripts/llm_golden.py run --url … --set tests/llm/golden_lineage.yaml --report scratch/golden-report.json — report pass/fail per question with the failing checks; do NOT file issues from the worktree run.

  • Step 6: Commit feat(llm): golden longitudinal evaluation — yaml set, runner with anchor/label/forbidden checks, nightly workflow + issue filer (refs #692).


Task 8: Notebook section, docs, tracker

Files:

  • Modify: notebooks/code_families.py (new section "7b. What the chat sees" — lineage_evidence events/labels for the picked code, rendered as a table with the same labels the chat cites, plus the prompt_block() text in a collapsible), docs/docs/cli/* (regen), tests/notebooks/test_code_families_nb.py (banner assertion for the new section)
  • Steps: implement the cell (guarded like sections 57; uses the replica only — lineage_evidence with a config pointing at data/replica/aco.ro.duckdb), headless export with and without env, uv run pytest tests/notebooks -q, uv run python docs/scripts/extract_cli.py, commit feat(notebooks,docs): code_families 7b — the chat's lineage view; CLI docs (refs #691). The controller posts the live numbers on #691/#692 and closes #699 items 12 with a comment.

Self-review

Spec coverage. Decision 5 (era-balanced) → T4; Decision 6 (timeline like valuation) → T3 + T6; "lineage_evidence → events + element diffs" → T3; "system prompt: cite the event label" → T3; UI links (P40/P41) → T3 labels/urls + T6; Evaluation section → T7; "ANY PFS-payable code" (user) → T1 (families) + T2 (events for all codes); Ruling A9 (manual source) → T5; #699 items 12 → T1. Not in this slice: #699 item 3 (synonym curation beyond the name-phrase rule), #698 elements inversion (T2 inverts events only; elements stay 20 codes — the chat's element diffs therefore cover the fixture families until #698 lands; say so in the notebook cell), #703, #700.

Placeholders. Tasks give signatures, rules, test intents and live checks; implementers are mid-tier models working from prose (allowed by the SDD model-selection rule). Exact SQL for the manual source and era grouping is left to the implementer with the binding style references.

Type consistency. LineageEvidence.payload()["type"] == "lineage" (T3) is what T6 dispatches on and T7 reads; as_source keys added in T5 (item_key, p_id, seq) are what T7's check_anchors matches; ChatRequest.mode (T4) is what T6 posts and T7's YAML mode sends; rule_label (T3) strings are what T7's expect_labels regexes target.