Vocabulary v2 adds four themes the CY2027 docket's letters made
unmissable: modifier-25-same-day, lactation-services, health-coaching,
radiation-therapy (57 themes).
tests/llm/golden_themes.yaml: 39 CMS-2026-2377 comments labelled by
reading their opening text — expected themes (must carry) and
acceptable ones (allowed, not required). llm.tageval scores
predictions per slug (tp / fp outside expected∪acceptable / fn),
micro precision/recall/F1 and the abstain rate, and gate() applies the
spec thresholds (micro-F1 ≥ 0.70; no theme with support ≥ 5 under 0.5
precision). Predictions come from what the last run stored on the bib
items (CI, no model calls) or --live through the chain.
stack llm eval-tags [--golden] [--live] [--limit] exits 1 when the gate
fails.
c354047e17