c354047e17
Vocabulary v2 adds four themes the CY2027 docket's letters made unmissable: modifier-25-same-day, lactation-services, health-coaching, radiation-therapy (57 themes). tests/llm/golden_themes.yaml: 39 CMS-2026-2377 comments labelled by reading their opening text — expected themes (must carry) and acceptable ones (allowed, not required). llm.tageval scores predictions per slug (tp / fp outside expected∪acceptable / fn), micro precision/recall/F1 and the abstain rate, and gate() applies the spec thresholds (micro-F1 ≥ 0.70; no theme with support ≥ 5 under 0.5 precision). Predictions come from what the last run stored on the bib items (CI, no model calls) or --live through the chain. stack llm eval-tags [--golden] [--live] [--limit] exits 1 when the gate fails.
1.5 KiB
1.5 KiB
title, sidebar_position
| title | sidebar_position |
|---|---|
| stack perf | 65 |
stack perf
Usage: stack perf [OPTIONS] COMMAND [ARGS]...
Pipeline telemetry utilities.
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ show Pretty-print spans from the local JSONL trace file. │
╰──────────────────────────────────────────────────────────────────────────────╯