feat: CMS rulemaking comment analysis module (rex.comments) #261

Closed

kert wants to merge 1 commits from feat/rulemaking-comments into main

pull from: feat/rulemaking-comments

merge into: homelab:main

homelab:main

homelab:feat/zot-fetch-z1

homelab:feat/zotero-fetch-connector

homelab:feat/zotero-ama-coding-collection

Conversation 0 Commits 1 Files Changed 8 +1449

kert commented 2026-03-25 19:50:49 +00:00

Owner

Copy Link

Summary

New rex.comments subpackage for downloading, extracting, classifying, and analyzing public comments on CMS rulemaking dockets from regulations.gov.

Reference: jacobmr/hti5 — ONC HTI-5 comment analysis methodology, adapted for CMS skin substitute rulemaking.

Modules

File	Purpose
models.py	Comment and Attachment dataclasses
client.py	regulations.gov v4 API client — fetch, paginate, download attachments, JSON cache
extract.py	PDF (pypdf) + DOCX (python-docx) text extraction from attachments
classify.py	5-point position scale + 11 policy theme tags via weighted regex
coordination.py	Stakeholder segmentation (9 types), Jaccard 5-gram form letter detection, provision mapping (7 provisions)
store.py	DuckDB skin_subs.rulemaking_comments table loader

Pipeline

regulations.gov API → download comments + PDF/DOCX attachments
  → extract full text from attachments (70%+ of substance)
  → classify position (strongly_oppose → strongly_support)
  → tag 11 policy themes (asp_methodology, fraud_waste_abuse, etc.)
  → segment stakeholders (manufacturer, provider, government, etc.)
  → detect coordinated form letter campaigns (Jaccard n-gram similarity)
  → map to 7 regulatory provisions
  → load to DuckDB for SQL analysis

Demo Results (15 synthetic comments)

• Position: 9 strongly oppose, 2 oppose, 2 neutral, 2 strongly support
• Stakeholders: 4 manufacturer, 2 government, 1 provider, 1 individual, 1 academic
• Coordination: 1 campaign detected (7 form letters = 46% of comments)
• Themes: flat_rate_design (7), asp_methodology (6), fraud_waste_abuse (3)

Target Dockets

• CMS-1834-P — CY 2026 OPPS proposed rule (skin sub reclassification, $9.4B savings)
• CMS-1809-FC — CY 2025 OPPS final rule
• CMS-1784-F — CY 2024 PFS final rule

Usage

# Demo mode (no API key needed):
uv run python dev/scripts/analyze_rulemaking_comments.py --demo

# Live fetch (requires REGULATIONS_GOV_API_KEY):
export REGULATIONS_GOV_API_KEY=your_key
uv run python dev/scripts/analyze_rulemaking_comments.py --docket CMS-1834-P

Test plan

• Demo mode runs end-to-end: fetch → classify → coordination → DuckDB
• Position classification produces expected distribution on synthetic comments
• Form letter detection correctly groups 5 explicit + 2 organic similar comments
• All 12,067 existing tests pass
• Live fetch with API key against CMS-1834-P docket
• PDF attachment extraction on real comment attachments

Closes #251, refs #252, #253, #254, #255, #256

🤖 Generated with Claude Code

## Summary New `rex.comments` subpackage for downloading, extracting, classifying, and analyzing public comments on CMS rulemaking dockets from regulations.gov. **Reference:** [jacobmr/hti5](https://github.com/jacobmr/hti5) — ONC HTI-5 comment analysis methodology, adapted for CMS skin substitute rulemaking. ## Modules | File | Purpose | |------|------| | `models.py` | `Comment` and `Attachment` dataclasses | | `client.py` | regulations.gov v4 API client — fetch, paginate, download attachments, JSON cache | | `extract.py` | PDF (pypdf) + DOCX (python-docx) text extraction from attachments | | `classify.py` | 5-point position scale + 11 policy theme tags via weighted regex | | `coordination.py` | Stakeholder segmentation (9 types), Jaccard 5-gram form letter detection, provision mapping (7 provisions) | | `store.py` | DuckDB `skin_subs.rulemaking_comments` table loader | ## Pipeline ``` regulations.gov API → download comments + PDF/DOCX attachments → extract full text from attachments (70%+ of substance) → classify position (strongly_oppose → strongly_support) → tag 11 policy themes (asp_methodology, fraud_waste_abuse, etc.) → segment stakeholders (manufacturer, provider, government, etc.) → detect coordinated form letter campaigns (Jaccard n-gram similarity) → map to 7 regulatory provisions → load to DuckDB for SQL analysis ``` ## Demo Results (15 synthetic comments) - **Position:** 9 strongly oppose, 2 oppose, 2 neutral, 2 strongly support - **Stakeholders:** 4 manufacturer, 2 government, 1 provider, 1 individual, 1 academic - **Coordination:** 1 campaign detected (7 form letters = 46% of comments) - **Themes:** flat_rate_design (7), asp_methodology (6), fraud_waste_abuse (3) ## Target Dockets - **CMS-1834-P** — CY 2026 OPPS proposed rule (skin sub reclassification, $9.4B savings) - **CMS-1809-FC** — CY 2025 OPPS final rule - **CMS-1784-F** — CY 2024 PFS final rule ## Usage ```bash # Demo mode (no API key needed): uv run python dev/scripts/analyze_rulemaking_comments.py --demo # Live fetch (requires REGULATIONS_GOV_API_KEY): export REGULATIONS_GOV_API_KEY=your_key uv run python dev/scripts/analyze_rulemaking_comments.py --docket CMS-1834-P ``` ## Test plan - [x] Demo mode runs end-to-end: fetch → classify → coordination → DuckDB - [x] Position classification produces expected distribution on synthetic comments - [x] Form letter detection correctly groups 5 explicit + 2 organic similar comments - [x] All 12,067 existing tests pass - [ ] Live fetch with API key against CMS-1834-P docket - [ ] PDF attachment extraction on real comment attachments Closes #251, refs #252, #253, #254, #255, #256 🤖 Generated with [Claude Code](https://claude.com/claude-code)

kert added 11 commits 2026-03-25 19:50:49 +00:00

data: PubMed systematic search + grey literature for skin substitutes (refs #235, refs #236) ... b822aa9443

PubMed E-utilities search across 3 domains:
- clinical efficacy: 7,445 articles (MeSH skin substitutes/biological dressings + outcomes)
- cost-effectiveness: 123 articles (economics, Medicare spending, ASP)
- fraud/waste/abuse: 356 articles (billing patterns, enforcement, compliance)
- 7,817 unique articles after dedup, stored in bib.sqlite with PRISMA counts

Grey literature catalogue: 20 curated documents
- 3 OIG reports (incl. Sept 2025 payment trends)
- 4 CMS rules (CY2024-2026 OPPS/PFS, Benefit Policy Manual)
- 4 DOJ/court filings (Jenson, Gehrke/King, Vohra, national takedown)
- 3 GAO/MedPAC reports
- 4 MAC LCDs (Noridian, CGS, First Coast, Palmetto)
- 2 industry position statements

All tagged module:skin-subs with source and type tags for downstream filtering.

data: evidence base builder + literature methods doc (refs #237, refs #244) ... c1f60e16ef

- Collection hierarchy: 20 sub-collections under "Skin Substitutes"
  (Clinical Evidence, CMS Policy, Enforcement, etc.)
- COI/funding enrichment: 2,862 PubMed articles tagged with
  coi:single-product (2,003), stance:skeptical (598),
  stance:favorable (510), coi:industry-linked (11)
- All 22 expected tag types verified present
- Methods documentation: search queries, PRISMA flow, tag schema,
  collection hierarchy, limitations

docs: add Research section to Docusaurus with skin substitutes pages ... fe1ea48da0

New sidebar section "Research" with skin substitutes sub-pages:
- Overview: hypothesis, background, key cases, data sources, phase status
- Literature Methods: search queries, PRISMA flow, grey lit catalogue
- Evidence Base: tag schema, collection hierarchy, COI enrichment, querying

Moves documentation from dev/docs/ into the Docusaurus site where it
belongs. Research section follows existing autogenerated sidebar pattern.

data: study characteristics, snowball citations, entity tags, expanded grey lit (refs #235, #236, #237, #244) ... aaa968bab0

- Study characteristics table: 7,817 rows in skin_subs.study_characteristics
  (pub type, products mentioned, sample size, COI flags)
- Forward snowball: 2,415 new relevant articles from 200 seeds via elink
- Entity tags: 2,130 items tagged with manufacturer mentions
  (Acell 1,868, Medline 164, Organogenesis 36, 3M 25, Integra 13)
- Grey lit expanded to 30 docs: +4 CMS rules (2014-2023), +3 MAC LCDs
  (WPS, Novitas, NGS), +2 court filings (Nasser, Patel), +1 OIG report
- Full evidence base loaded as skin_subs.evidence_base (10,262 rows)
- Docusaurus research pages updated with final counts

fix: bib.sync and XML parsers — Zotero key validation, creators, journalArticle, itertext ... d7c624b127

bib.sync:
- Keys validated against Zotero allowedKeyChars (no 0, 1, O, lowercase)
- Creators parsed from extra into Zotero creators/itemCreators tables
- PubMed articles pushed as journalArticle (typeID=22) not document
- DOI, PMID, Journal parsed into dedicated Zotero fields (8, 86, 41)
- Authors stripped from extra to avoid duplication

XML parsers (search_pubmed, snowball_citations):
- _text() uses itertext() instead of .text to handle mixed-content XML
  (sup, i, sub tags inside ArticleTitle) — fixes 20 blank titles

Verified: 10,262 items, 0 blank titles, 0 invalid keys, 60,866 creators,
9,658 DOIs, 10,232 PMIDs in proper Zotero fields.

fix: bib.sync — normalize timestamps to Zotero format (no T, no Z) ... 8b6e6ef1b3

Zotero expects 'YYYY-MM-DD HH:MM:SS' for date fields, not ISO 8601
with T separator and Z suffix. Added _normalize_zotero_date() and
applied it to accessDate, dateEnacted, and date fields in _set_field().
Also fixed _now_iso() for dateAdded/dateModified on item creation.

fix: compose notebooks mount — remove :ro from parent to allow nested loch.css bind ... c2fa7326c4

infra/marimo mounted :ro blocked the nested loch.css file mount from
creating its mountpoint. Removed :ro from the parent; child mounts
(loch.css, home-page-patched.js) remain :ro individually.

data: ASP trajectories, market segmentation, utilization metrics (refs #234, #238, #239) ... e5ec7a22a1

ASP pricing trajectories (skin_subs.asp_trajectories):
- 1,974 rows: QoQ % change, cumulative change, entry quarter
- 39 products with anomalous >50% QoQ spikes
- Jan 2026 flat-rate impact: 105 losers, 47 gainers
- XWRAP Dual: $5,893/cm² payment vs $127.28 flat rate

Market segmentation (skin_subs.products_enriched, skin_subs.manufacturers):
- 286 products enriched with ASP stats + claims utilization
- 68 manufacturers; Organogenesis, MiMedx, Smith+Nephew top 3
- Amniotic membrane category: 214 products, avg ASP $1,392/cm²

Utilization metrics (skin_subs.providers, beneficiaries, utilization_summary):
- 200 providers, 1,255 beneficiaries from synthetic claims
- Podiatry: 60% of claims ($1.02M) — consistent with DOJ cases
- Office setting: 68% of spend — lowest oversight environment
- Top provider: $55K paid, 13 patients, $4,247/patient (FL dermatology)

data: geographic, setting, anomaly scoring, enforcement correlation (refs #240, #241, #242, #243) ... 88025d5976

Geographic analysis (skin_subs.geographic_summary, mac_summary):
- 16 states, 8 MAC jurisdictions
- FL highest spend/bene ($2,142), First Coast MAC #1 overall
- Palmetto states (TN, VA, NC) have highest per-bene spend

Setting analysis (skin_subs.setting_analysis):
- Office setting: 68% of product spend, lowest oversight
- Podiatry×office: 45% of total spend — wound care mill pattern
- 169 setting×specialty×product combinations

Anomaly scoring (skin_subs.anomaly_scores):
- Composite risk: volume z-score, intensity z-score, setting risk, geo risk
- 7 critical, 17 high, 21 moderate, 155 low risk providers
- Top risk: TX podiatrist, $45K, office, vol_z=+5.69 — Jenson pattern

Enforcement correlation (skin_subs.provider_risk_profile):
- 23 providers match "jenson-like" pattern (TX podiatry, office)
- Benford chi² analysis (no synthetic data anomalies expected)
- Fraud-pattern matched providers slightly higher volume z-scores

17 tables in skin_subs schema.

feat: skin substitutes marimo notebook + snippet (refs #245) ... 32c6d3cc00

Interactive explorer with 6 tabs:
- ASP Pricing: product-level time series with $127.28 flat rate line
- Market: manufacturer revenue bar chart, product category breakdown
- Geographic: state spend/bene, MAC jurisdiction comparison
- Setting: specialty x setting spend analysis
- Anomaly: provider risk scatter (volume vs intensity z-scores)
- Evidence: source distribution, study characteristics summary

Snippet: skin-subs-connect.py for quick DuckDB table inventory.
Updated snippet count test from 10 to 11.

feat: rex.comments — CMS rulemaking comment analysis module (refs #251-#256) ... 8982633860

New subpackage: src/rex/comments/ for downloading, extracting, classifying,
and analyzing public comments on CMS rulemaking dockets.

Modules:
- models.py — Comment and Attachment dataclasses
- client.py — regulations.gov API v4 client (fetch, download, cache)
- extract.py — PDF (pypdf) and DOCX (python-docx) text extraction
- classify.py — 5-point position scale + 11 theme tags via weighted regex
- coordination.py — stakeholder segmentation, Jaccard n-gram form letter
  detection, provision mapping
- store.py — DuckDB skin_subs.rulemaking_comments table

Pipeline script: dev/scripts/analyze_rulemaking_comments.py
- --demo mode with 15 synthetic comments for development
- --docket for live regulations.gov fetch (requires API key)
- --cache for offline analysis from JSON

Demo results: 15 comments, 1 coordinated campaign (7 form letters),
position split 9 oppose / 2 neutral / 2 support.

Reference: jacobmr/hti5 methodology adapted for CMS skin sub dockets.

kert closed this pull request 2026-07-08 13:33:55 +00:00

kert deleted branch feat/rulemaking-comments 2026-07-08 13:33:55 +00:00

Some checks are pending

Hide all checks

CI / skinny-install (bcda) (push) Successful in 38s

Details

CI / skinny-install (conf) (push) Successful in 28s

Details

CI / skinny-install (perf) (push) Successful in 31s

Details

CI / skinny-install (pfs) (push) Successful in 30s

Details

CI / skinny-install (rex) (push) Successful in 33s

Details

Infra CI / docs (push) Failing after 6s

Details

Infra CI / api (push) Successful in 28s

Details

Infra CI / mc (push) Successful in 10s

Details

CI / skinny-install (aco) (push) Successful in 1m17s

Details

CI / skinny-install (api) (push) Successful in 37s

Details

CI / lint-test (push) Failing after 1m59s

Details

CI / skinny-install (bib) (push) Successful in 31s

Details

CI / skinny-install (bls) (push) Successful in 28s

Details

CI / skinny-install (ccw) (push) Successful in 31s

Details

CI / skinny-install (cms) (push) Successful in 28s

Details

CI / skinny-install (cli) (push) Successful in 39s

Details

Infra CI / zotero (push) Successful in 7s

Details

CI / lint-test (pull_request) Failing after 1m32s

Details

Infra CI / notebooks (push) Successful in 3m15s

Details

CI / skinny-install (aco) (pull_request) Successful in 49s

Details

CI / skinny-install (bcda) (pull_request) Successful in 32s

Details

CI / skinny-install (api) (pull_request) Successful in 41s

Details

CI / skinny-install (bls) (pull_request) Successful in 30s

Details

CI / skinny-install (bib) (pull_request) Successful in 35s

Details

CI / skinny-install (ccw) (pull_request) Successful in 27s

Details

CI / skinny-install (cli) (pull_request) Successful in 36s

Details

CI / skinny-install (cms) (pull_request) Successful in 24s

Details

CI / skinny-install (conf) (pull_request) Successful in 29s

Details

CI / skinny-install (perf) (pull_request) Successful in 49s

Details

CI / skinny-install (pfs) (pull_request) Failing after 45s

Details

CI / skinny-install (rex) (pull_request) Successful in 28s

Details

Infra CI / notebooks (pull_request) Successful in 16s

Details

Infra CI / zotero (pull_request) Successful in 6s

Details

Infra CI / api (pull_request) Successful in 6s

Details

Infra CI / mc (pull_request) Successful in 7s

Details

Infra CI / docs (pull_request) Successful in 44s

Details

ci/woodpecker/push/ci

Required

ci/woodpecker/pr/ci

Required

ci/woodpecker/push/infra-ci

Required

ci/woodpecker/pr/infra-ci

Required

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

api

REST API (FastAPI)

auth

Authentication and authorization

bib

Bibliography, Zotero, citations

ci

CI/CD pipelines and Woodpecker plugins

cli

CLI commands and UX

cloud

Cloud provider abstraction (AWS, GCP, Azure, self-hosted)

config

Configuration and wiring

data

Data loading, CCLF, BCDA, seeds

databricks

Databricks jobs and deployment

docs

Documentation and generated references

image:api

image:docs

image:mc

image:notebooks

image:zotero

infra

lakehouse

Nessie, Polaris, Trino, Iceberg

llm

llm module — local RAG over comments + Zotero corpus

notebooks

marimo notebook errors (auto-filed by nb-watcher/nb-integration)

packaging

Python packaging, optional deps, build system

pfs

pipeline

Pipeline execution and validation

pipeline:ci

pipeline:deploy

pipeline:harden

pipeline:infra-ci

pipeline:rebuild-all

pipeline:release

qpp

quality

Testing, coverage, error handling

step:build

step:deploy

step:hadolint

step:lint

step:publish

step:scan

step:test

step:validate

styles

Charts, figures, visual theming

tree

Project directory structure and file organization

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

kert renovate-bot

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: homelab/stack#261