kert c9b6a28f07 feat(comments): coordination + form-letter detection, no LLM required (refs #255) ...

rex.comments.coordination: hti5 methodology — Jaccard similarity on
character 5-gram shingles (threshold 0.45) over normalized text,
union-find grouping with stable group ids, form-letter flag at >=3
members. Short texts (<200 chars) are excluded: two short 'I oppose'
notes are agreement, not coordination.

rex.comments.table: shared load path for skin_subs.rulemaking_comments
— outer-joins the classification (#254) and coordination (#255) JSONL
caches onto the comment identity rows, so whichever pass runs first
populates the table and the other enriches it without clobbering.
classify_comments.py refactored onto it; new analyze_coordination.py
driver.

Run against CMS-2025-0304 (the CY2026 OPPS skin-sub docket): of the
384 skin-sub-relevant comments, 203 (53%) are form letters across 31
coordinated groups — largest campaigns 72 and 36 members. Table loaded:
384 rows, coordination columns filled, classification columns NULL
until the #254 LLM run (blocked on API credits).

59 comments tests green.

2026-07-10 22:17:42 -04:00

4.3 KiB

Raw Blame History

View Raw