llm: longitudinal chat — lineage SSE event + timeline table, era-balanced retrieval, cited history answers #691
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: homelab/stack#691
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Spec docs/superpowers/specs/2026-09-09-code-family-longitudinal-design.md (§Decisions 5–6;
llm/evidence.py,llm/rag.py,llm/web/chat.html).What. Mirrors P48's valuation flow:
detect_codes→lineage_evidence(question)readspfs.code_event,pfs.code_elementdiffs andpfs.code_guidancefrom the replica → SSElineageevent before the first token (year, kind, codes, label, FR/eCFR jump link) → prompt "Lineage" block whose rows carry bracketed labels ([CY2021 PFS final ¶1578]) the model must cite for any dated claim → UI timeline table between the answer and the sources drawer.Retrieval.
retrieve(..., mode="timeline"): era-balanced, capped per rule year, so CY2015 and CY2027 both surface for "history of CCM"; the recency blend (P34) stays for ordinary questions. Triggered by history words ("history", "evolution", "when did", "replaced", "predecessor") or an explicit UI toggle.Target answers (from the corpus). "History of CCM coding and payment" → G-code (CY2013–14) → 99490 CY2015 (
DE2VH9PD¶1251) → complex CCM 99487/99489 B→A CY2017 → 99491 CY2019 → G2058 CY2020 → 99439 CY2021 (YBM4IZUS¶1578) → 99437 CY2022 → APCM G0556–G0558 CY2025 (JJ6AM5HJ¶1163), with the valuation table (P48) beside it. "Does 99490 pass telehealth Step 3?" → walk the three steps (2KVJ2HKX¶394/¶396/¶398) against 99490's elements.Done when the event sequence is
lineage? → valuation? → token* → sources → done, a control question is byte-identical to today, and evidence adds < 300 ms.Merged to main as
da345bc(branch worktree-p49-chat, 21 commits). The chat now streams lineage → valuation → token* → sources → done: llm.lineage.lineage_evidence reads pfs.code_event (precomputed for 6,710 of 8,689 payable codes by stack pfs lineage --all-payable, 2m37s), collapses repeated mentions to the first occurrence per kind (final rule preferred over proposed), and hands the model a Lineage block whose labels (CY2021 PFS final 85 FR 84639 ¶1578) it must cite; the cited paragraphs are added to the sources so the model reads what it cites and the drawer links them. Retrieval: retrieve(mode="timeline") era-balanced with even era coverage, auto-triggered on history questions, ChatRequest.mode; comments windowed per docket in timeline mode. The CPT manual heading guideline is a deterministic source. UI: timeline table, element differences, guidance, mode selector. Derived families reach the chat (#699 items 1–2). Prompt budget: per-turn code cap 24, valuation rows cap, char budget with a drop order — a 58-code three-family question now builds a 16 kB prompt (was 55 kB). Control question byte-identical; lineage evidence ~23 ms live.Closeout notes (2026-09-10). Rolling to the live service: COMMIT_SHA=da345bc in .env, then docker compose build llm && docker compose up -d llm on the host (the container ran fhirworx/llm:ac06d7d, the P48 image). Try it in the chat page (the llm service, port 8000): mode selector auto/timeline/recent; questions that exercise the new path — 'What is the history of chronic care management coding and payment?', 'What replaced G2058 and why?', 'When were audio-only E/M codes payable, and why did 99441-99443 end?', 'What did commenters say about G2211 in 2023 vs 2025?', 'How did G2064/G2065 becoming 99424/99426 change the RHC/FQHC G0511 rate?'. Expect: a timeline table above the answer with linked paragraph labels (CY2021 PFS final 85 FR 84639 ¶1578), element differences and guidance below it, the valuation table as before, and the answer citing those labels. Golden nightly: llm-golden workflow at 03:40 files regressions with label llm. Known gaps: g2058 misses ¶1578 (outside the 8-paragraph cap), APCM-vs-CCM element paragraphs, 99490 telehealth Step 1–3 paragraphs. Ledger with rulings B1–B15 is in the session scratchpad (p49-chat-ledger.md).
Rolled 2026-09-10: fhirworx/llm:da345bc is live (healthy). One deployment fix was needed on top of the merge — the llm container mounted only the DuckDB replica, so it started with an empty bib.sqlite and every lineage label degraded to a raw item key; compose.yml now mounts ./data like the api service (
bb7d0e0). Verified inside the container: 'What replaced G2058 and why?' in timeline mode streams lineage → tokens → sources → done with resolved labels (CY2022 PFS final 86 FR 39174 ¶597) and federalregister.gov paragraph links. Follow-up: 91 FR 43896 (the CY2027 NPRM) is labelled 'final' — the proposed/final flag reads the bib title, which for that item lacks the word 'Proposed'; fix the title or derive the flag from the FR document type.Rolled fhirworx/llm:d8f6678 (2026-09-10, later). Two fixes after the first roll: (1)
6ed7ae5— proposed rules were labelled 'final': every PFS proposed rule since CY2018 (incl. the CY2025 and CY2027 NPRMs) because CMS titles no longer carry the word and the label read the title. bib.frlink.rule_kind now decides proposed/final/correction from the rule's own paragraphs ('this proposed rule'/'we propose' vs 'this final rule'/'we are finalizing'; a Correction title wins); the title is only a fallback and an undecidable rule is labelled 'rule', never 'final'. All 80 grabbed rules verified (30 proposed, 28 final, 22 corrections). Labels, the final-over-proposed tie-break and the golden audio-only entry use it. (2)d8f6678— chat answers render markdown (headings/lists/bold, text nodes only) and cited [labels] are painted as jump links to the FR paragraph / eCFR section using the URLs carried by the lineage/valuation/sources events (repainted when the sources event lands). Verified live: 'history of CCM' in timeline mode streams 808 tokens; labels show CY2027 PFS proposed / CY2022 PFS proposed and final / CY2025 proposed and final.Two more fixes rolled with
2de806d: (1) #700 — G0502/G0503 no longer contaminate CCM (see #700). (2) Acronym conflicts: the CY2027 NPRM (91 FR 43842 ¶330–331) calls its new cardiac contractility modulation codes 33X01–33X11 'the CCM code family'; similarity retrieval pulled those paragraphs into chronic-care-management answers. Family entries now carry conflict phrases (Family.conflicts; CCM: 'cardiac contractility modulation'), and both retrieval (full chunk text) and the cited/lineage/manual sources (snippets) drop excerpts that use the acronym the other way; the golden ccm-history entry forbids 'contractility'. Also in this build: lineage lines as sentences ending with their label, an end-of-prompt citation rule, temperature 0.2, junk FR page numbers filtered from event code lists (81797a5).