c1a199c878
Final fix-wave items C1, I1-I4 and B11 (refined), all landing in the
same handful of interconnected files (llm.lineage/llm.rag/llm.evidence
share the collapse/select/label/budget call paths, so they can't be
split into independently-working commits):
- C1 (critical): llm.lineage._store() was one process-global bib.Store
whose sqlite connection is check_same_thread=True — /chat runs each
turn in Starlette's threadpool, so every thread but the first got a
silently inert (or raising) store. Now threading.local(), one Store
opened lazily per thread. rag._docket_year uses the same accessor and
is now fully guarded (never escapes era_of). New
TestConcurrentLineage (8 threads x 40 calls, mirrors
TestConcurrentChats) asserts no cross-thread error and one Store
construction per thread.
- I1 (Ruling B12): _collapse's tie-break is now
(not anchored_fr, is_proposed, p_id, item_key) — a final rule beats a
tied proposed one, and a fully-tied pair is decided by item_key for a
deterministic total order.
- I2 (Ruling B13): rule_label carries "{vol} FR {page}" when the
paragraph resolves (via the cached item/paragraph lookups), making
labels unique across paragraphs that used to share one.
merge_sources dedupes rule-kind rows on (item_key, p_id) instead of
label, since two rule chunks for the same paragraph can now carry
different labels.
- I3 (Important): era_balance picks eras evenly across the range when
there are more distinct eras than top_n, so the round-robin (which
visits newest-first every round) doesn't silently drop the oldest
eras from a wide history question.
- I4 (Important): _SYSTEM's opener, refusal clause and recency
guidance are reworded to match what the prompt actually contains
(excerpts + optional Lineage + optional Valuation), and the Lineage
paragraph now precedes Valuation to match build_messages' order.
- B11 refined (whole-branch review): a hard per-turn code cap
(chat_codes_max=24, explicit codes then family order —
llm.evidence.cap_codes, shared by valuation_evidence and
lineage_evidence so both cap the same question identically),
a valuation-rows cap (valuation_rows_max=24, explicit then newest
vintage), a lineage-rows hard cap (2x lineage_max_rows, priority
rows capped at 4/code), element-diff code lists compacted past 8,
manual sources capped at 2, and build_messages(budget_chars=...)
which drops lineage-source excerpts>4, retrieved>6, cited>8,
manual>1, valuation rows>12, then lineage rows>lineage_max_rows in
that order until the assembled prompt fits — wired into
stream_answer as budget_chars=cfg.chat_num_ctx*3. The
_LINEAGE_SOURCES_HARD_CAP is now enforced in lineage_sources' own
event loop, not only its element-diff tail.
I5: TestLineageEvidenceLive now uses the shared restore_families
fixture (moved to tests/conftest.py) so opening the real replica
doesn't leak thousands of derived families into later tests.
Diagnosing this also turned up a second, pre-existing leak of the same
shape: TestStreamAnswer's control-question test ran the real
lineage_evidence against CFG's default (real, 3GB)
data/replica/aco.ro.duckdb, since llm.lineage.lineage_evidence calls
evidence.warm(cfg) unconditionally before checking for detected codes
— fixed by pointing that one test at a nonexistent replica path
(exactly the "without touching the replica" behavior its own docstring
already claimed).
Verified: uv run pytest tests/llm tests/pfs/test_families.py
tests/pfs/test_lineage.py tests/cli/test_pfs_cli.py tests/dev -q
-p no:cacheprovider -m "not live" — 583 passed. In-process golden run
against the live replica: 3/7 pass (ccm-history, audio-only-em-99441,
g2211-commenters); g2058-replacement/apcm-vs-ccm/99490-telehealth
unchanged documented gaps; g2064-g2065 newly misses one of its three
anchors (JE7KYBW3 p1111) specifically because of the new 4-per-code
lineage-row cap this commit adds (Ruling B11) — an accepted tradeoff
of the budget work, not a bug. Prompt-size check (budget 24,576
chars): "history of CCM coding and payment" 20,670 chars; the 58-code
three-family history question 16,135 chars (58 detected codes capped
to 24) — both under budget.
46 KiB
46 KiB