homelab/stack
1
0
Fork 0
Code Issues 58 Pull Requests 1 Actions Packages Projects 1 Releases Wiki Activity

feat(notebooks): prior-year palliative comment exhibit — counts, table, decade themes (refs #660)

Browse Source
This commit is contained in:
kert
2026-08-19 12:31:32 -04:00
parent f7fac3fd96
commit 4bfe624892
2 changed files with 669 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

393
docs/superpowers/plans/2026-08-19-palliative-comment-tagging-p43.md Normal file
View File

@@ -0,0 +1,393 @@
# Palliative Comment Tagging + Notebook Summary Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Tag every prior-year PFS-rulemaking comment that references palliative care (`topic:palliative` in bib), make them citable (scoped Zotero sync), and add a summary section to `notebooks/palliative_care_rfi.py` (issue #660, P43).
**Architecture:** A lexical sweep script walks all regulations.gov comment items in the bib store (166,941 across ten dockets CMS-2017-0092…CMS-2026-2377); per item the text is the extracted `combined.md` body under `.state/comments/<docket>/<comment_id>/` when present, else the enriched abstract. Matches on the `palliativ` stem get the `topic:palliative` tag, mention count + sentence snippets in `extra_json`, and per-docket denominators land in `bib_meta` (the notebook container mounts `data/` but not `.state/`, so everything the notebook renders must be materialized into `bib.sqlite` at sweep time). The notebook reads bib.sqlite live via `conf.connect.bib()`.
**Tech Stack:** Python stdlib `re`/`json`; `bib.store.Store` (`add_tag`, `remove_tag`, `update`); `rex.comments.combine.parse_combined`; marimo + polars + altair for the notebook section.
**Spec:** Gitea issue #660 (https://git.fhirworx.io/homelab/stack/issues/660).
## Global Constraints
- The `llm` module and everything touching the comment corpus is local-only — no cloud LLM calls anywhere in this work.
- Notebook rules: never mention skin substitutes in `palliative_care_rfi.py`; cast year columns through `conf.display.plain_years` (marimo renders int years as "2,027"); FR/CFR cites go through `bib.frlink`/`bib.cfrlink` (regulations.gov comment links are plain URLs, not FR cites).
- `stack bib sync-zotero` must always be scoped `--tag topic:palliative` (unscoped sync is the P38 incident vector).
- Never `git stash` in this worktree (container-owned `data/zotero/**` strands the stash). Stage explicit paths only; run `git status --short` before every commit; ruff-format before committing; commits with the pre-commit hook need background/600s timeouts.
- No Claude co-author trailer on commits.
- The sweep must be idempotent: re-running refreshes tags/extras and removes `topic:palliative` from items that no longer match.
---
### Task 1: Sweep script with tested mention extraction
**Files:**
- Create: `dev/scripts/tag_palliative_comments.py`
- Test: `tests/dev/test_tag_palliative_comments.py`
**Interfaces:**
- Consumes: `bib.store.Store` (`add_tag(item_key, tag)`, `remove_tag(item_key, tag)`, `update(key, extra_json=...)`, `_con()`), `rex.comments.combine.parse_combined(text) -> (header, body)`.
- Produces: script CLI `uv run python dev/scripts/tag_palliative_comments.py [--root PATH] [--db PATH] [--dry-run]`; pure helpers `find_mentions(text, max_snippets=3) -> tuple[int, list[str]]` and `sweep(store, root, dry_run=False) -> dict`; bib side effects: tag `topic:palliative`, `extra_json` keys `palliative_mentions` (int) and `palliative_snippets` (list[str]), `bib_meta` key `palliative_sweep` = JSON `{"dockets": {docket: {"total": int, "with_text": int, "hits": int}}, "swept_at": iso8601}`.
- [ ] **Step 1: Write the failing tests**
```python
"""Tests for dev/scripts/tag_palliative_comments.py — mention extraction."""
from __future__ import annotations
import importlib.util
import sys
from pathlib import Path
_SCRIPT = (
Path(__file__).resolve().parents[2]
/ "dev"
/ "scripts"
/ "tag_palliative_comments.py"
)
_spec = importlib.util.spec_from_file_location("_tag_palliative", _SCRIPT)
assert _spec and _spec.loader
tpc = importlib.util.module_from_spec(_spec)
sys.modules["_tag_palliative"] = tpc
_spec.loader.exec_module(tpc)
class TestFindMentions:
def test_no_match(self):
assert tpc.find_mentions("Medicare payment policy comment.") == (0, [])
def test_counts_and_case(self):
text = (
"We support Palliative care. Our palliative program grew. "
"Palliative-trained nurses matter."
)
count, snippets = tpc.find_mentions(text)
assert count == 3
assert len(snippets) >= 1
assert all("alliativ" in s for s in snippets)
def test_snippets_are_sentences_with_context(self):
text = (
"Intro sentence. CMS should pay for palliative consultations "
"in the home. Closing sentence."
)
_, snippets = tpc.find_mentions(text)
assert snippets == [
"CMS should pay for palliative consultations in the home."
]
def test_snippet_cap(self):
text = " ".join(f"Point {i} on palliative care." for i in range(10))
count, snippets = tpc.find_mentions(text, max_snippets=3)
assert count == 10
assert len(snippets) == 3
def test_long_sentence_trimmed(self):
text = "x" * 400 + " palliative " + "y" * 400
_, snippets = tpc.find_mentions(text)
assert snippets and len(snippets[0]) <= 320
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `uv run pytest tests/dev/test_tag_palliative_comments.py -v`
Expected: FAIL — script file does not exist.
- [ ] **Step 3: Write the script**
`dev/scripts/tag_palliative_comments.py`:
```python
"""Tag PFS-rulemaking comments that reference palliative care (#660).
Sweeps every regulations.gov comment item in the bib store. Per item the
text is the extracted body (`.state/comments/<docket>/<comment_id>/combined.md`)
when present, else the enriched abstract. Items matching the ``palliativ``
stem get:
- tag ``topic:palliative``
- ``extra_json`` keys ``palliative_mentions`` / ``palliative_snippets``
Per-docket sweep stats (total / with_text / hits) land in ``bib_meta``
under key ``palliative_sweep`` so the notebook can disclose denominators
without access to ``.state``. Idempotent: re-running refreshes extras and
removes the tag from items that no longer match.
Usage:
uv run python dev/scripts/tag_palliative_comments.py [--dry-run]
"""
from __future__ import annotations
import argparse
import json
import re
from collections import defaultdict
from datetime import datetime, timezone
from pathlib import Path
_PATTERN = re.compile(r"palliativ\w*", re.IGNORECASE)
_COMMENT_URL_PREFIX = "https://www.regulations.gov/comment/"
_TAG = "topic:palliative"
_META_KEY = "palliative_sweep"
_SNIPPET_MAX_CHARS = 320
def find_mentions(text: str, max_snippets: int = 3) -> tuple[int, list[str]]:
"""Count ``palliativ*`` matches and pull up to *max_snippets* sentences."""
matches = list(_PATTERN.finditer(text))
if not matches:
return 0, []
snippets: list[str] = []
seen_spans: list[tuple[int, int]] = []
for m in matches:
if len(snippets) >= max_snippets:
break
# sentence-ish window: back to the previous ./!/?/newline, forward to the next
start = max(
text.rfind(". ", 0, m.start()),
text.rfind("! ", 0, m.start()),
text.rfind("? ", 0, m.start()),
text.rfind("\n", 0, m.start()),
)
start = start + 2 if start > 0 else 0
ends = [
text.find(". ", m.end()),
text.find("! ", m.end()),
text.find("? ", m.end()),
text.find("\n", m.end()),
]
ends = [e for e in ends if e != -1]
end = min(ends) + 1 if ends else len(text)
if any(s <= m.start() < e for s, e in seen_spans):
continue # sentence already captured
seen_spans.append((start, end))
snippet = " ".join(text[start:end].split())
if len(snippet) > _SNIPPET_MAX_CHARS:
cut = snippet.lower().find("palliativ")
lo = max(0, cut - _SNIPPET_MAX_CHARS // 2)
snippet = ("…" if lo else "") + snippet[lo : lo + _SNIPPET_MAX_CHARS - 2] + "…"
snippets.append(snippet)
return len(matches), snippets
def _comment_rows(store) -> list[tuple[str, str, str, str]]:
"""(key, url, docket, comment_id) for every comment item."""
rows = (
store._con()
.execute(
"SELECT key, url FROM items WHERE url LIKE ?",
(f"{_COMMENT_URL_PREFIX}%",),
)
.fetchall()
)
out = []
for key, url in rows:
comment_id = url[len(_COMMENT_URL_PREFIX) :]
docket = comment_id.rsplit("-", 1)[0]
out.append((key, url, docket, comment_id))
return out
def _merge_extra(store, key: str, mentions: int, snippets: list[str]) -> None:
row = (
store._con()
.execute("SELECT extra_json FROM items WHERE key = ?", (key,))
.fetchone()
)
extra = json.loads(row[0]) if row and row[0] else {}
extra["palliative_mentions"] = mentions
extra["palliative_snippets"] = snippets
store.update(key, extra_json=json.dumps(extra))
def _clear_extra(store, key: str) -> None:
row = (
store._con()
.execute("SELECT extra_json FROM items WHERE key = ?", (key,))
.fetchone()
)
if not (row and row[0]):
return
extra = json.loads(row[0])
if "palliative_mentions" in extra or "palliative_snippets" in extra:
extra.pop("palliative_mentions", None)
extra.pop("palliative_snippets", None)
store.update(key, extra_json=json.dumps(extra))
def sweep(store, root: Path, dry_run: bool = False) -> dict:
from rex.comments.combine import parse_combined
tagged_before = {
r[0]
for r in store._con().execute(
"SELECT i.key FROM items i JOIN item_tags it ON it.item_id = i.id "
"JOIN tags t ON t.id = it.tag_id WHERE t.name = ?",
(_TAG,),
)
}
stats: dict[str, dict[str, int]] = defaultdict(
lambda: {"total": 0, "with_text": 0, "hits": 0}
)
hits: set[str] = set()
for key, _url, docket, comment_id in _comment_rows(store):
stats[docket]["total"] += 1
combined = root / docket / comment_id / "combined.md"
text = ""
if combined.exists():
_, body = parse_combined(combined.read_text())
text = body
if not text.strip():
text = store.get(key).abstract or ""
if not text.strip():
continue
stats[docket]["with_text"] += 1
count, snippets = find_mentions(text)
if not count:
continue
stats[docket]["hits"] += 1
hits.add(key)
if not dry_run:
store.add_tag(key, _TAG)
_merge_extra(store, key, count, snippets)
stale = tagged_before - hits
if not dry_run:
for key in stale:
store.remove_tag(key, _TAG)
_clear_extra(store, key)
meta = {
"dockets": dict(stats),
"swept_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
}
store._con().execute(
"INSERT INTO bib_meta (key, value) VALUES (?, ?) "
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
(_META_KEY, json.dumps(meta)),
)
store._con().commit()
return {"stats": dict(stats), "hits": len(hits), "untagged": len(stale)}
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", default=".state/comments", type=Path)
parser.add_argument("--db", default="", help="bib.sqlite path (default: cfg)")
parser.add_argument("--dry-run", action="store_true")
args = parser.parse_args()
if args.db:
from bib.store import Store
store = Store(args.db)
else:
from conf.connect import bib
store = bib()
result = sweep(store, args.root, dry_run=args.dry_run)
for docket in sorted(result["stats"]):
s = result["stats"][docket]
print(
f" {docket}: {s['hits']} hits / {s['with_text']} with text "
f"/ {s['total']} total"
)
print(f"tagged={result['hits']} untagged_stale={result['untagged']}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
```
(Adjust `Store(args.db)` if the constructor signature differs — check `bib/store.py:49`. If `bib_meta` lacks a UNIQUE constraint on `key`, use `DELETE` + `INSERT` instead of upsert — check `PRAGMA index_list(bib_meta)` first.)
- [ ] **Step 4: Run tests to verify they pass**
Run: `uv run pytest tests/dev/test_tag_palliative_comments.py -v`
Expected: PASS. Iterate on `find_mentions` until the sentence-window tests pass exactly.
- [ ] **Step 5: Ruff + commit**
```bash
uv run ruff format dev/scripts/tag_palliative_comments.py tests/dev/test_tag_palliative_comments.py && uv run ruff check dev/scripts/tag_palliative_comments.py tests/dev/test_tag_palliative_comments.py
git status --short # foreign staged work check
git add dev/scripts/tag_palliative_comments.py tests/dev/test_tag_palliative_comments.py
git commit -m "feat(comments): palliative-mention sweep — topic:palliative tags + extra_json snippets (refs #660)"
```
### Task 2: Run the sweep on the live store
**Files:** none (operational).
- [ ] **Step 1: Dry run** — `uv run python dev/scripts/tag_palliative_comments.py --dry-run`; expect ~270 hits spread over the ten dockets, heaviest CMS-2018-0076 (~71) and CMS-2023-0121 (~44). Investigate anything wildly off (e.g. 0 hits = root path wrong).
- [ ] **Step 2: Real run** — same command without `--dry-run`.
- [ ] **Step 3: Verify** —
```bash
uv run python -c "
from conf.connect import bib
store = bib(); con = store._con()
print(con.execute(\"SELECT count(*) FROM items i JOIN item_tags it ON it.item_id=i.id JOIN tags t ON t.id=it.tag_id WHERE t.name='topic:palliative'\").fetchone())
print(con.execute(\"SELECT value FROM bib_meta WHERE key='palliative_sweep'\").fetchone()[0][:200])
"
```
Expected: count matches the run's `tagged=` output; meta JSON present.
### Task 3: Scoped Zotero sync
**Files:** none (operational).
- [ ] **Step 1:** Read `stack bib sync-zotero --help` and `src/cli/bib.py:472` to confirm flags; confirm whether it stops the zotero container itself or needs a manual stop (mail-pipeline memory says the nightly workflow stops zotero for the SQLite lock).
- [ ] **Step 2:** Run `uv run stack bib sync-zotero --tag topic:palliative` (with the container held per Step 1's finding). NEVER run it unscoped.
- [ ] **Step 3:** Verify in Zotero: `uv run python -c` query of `connect.zotero()` for items tagged `topic:palliative` — expect the same ~270.
### Task 4: Notebook section
**Files:**
- Modify: `notebooks/palliative_care_rfi.py` (append a new section before the methods/appendix cells; match existing section-header style)
**Interfaces:**
- Consumes: `conf.connect.bib()` (mounted `data/bib.sqlite`), tags `topic:palliative` / `reg-docket:*` / `year:*` / `rule:*`, `extra_json.palliative_mentions` / `.palliative_snippets`, `bib_meta.palliative_sweep`.
- Produces: marimo cells — (a) intro md, (b) polars frame of tagged comments, (c) altair bar chart of hits per rule year with with-text denominators, (d) table of referencing comments (year, rule, commenter, mentions, regulations.gov link), (e) thematic summary prose with inline comment citations.
- [ ] **Step 1: Load the `dataviz` skill before writing the chart cell** (repo palette imports in the notebook — `fhirworx` AMBER/GRAY/INDIGO_MED/PLUM/TEAL — take precedence for colors).
- [ ] **Step 2: Read the notebook's existing section/chart/table cell patterns** (`notebooks/palliative_care_rfi.py`) and mirror them exactly: same md header style, same altair theming, `plain_years` for year columns.
- [ ] **Step 3: Write the cells.** Query shape:
```python
_rows = connect.bib()._con().execute(
"""
SELECT i.key, i.title, i.url, i.institution, i.extra_json,
MAX(CASE WHEN t.name LIKE 'year:%' THEN substr(t.name, 6) END) AS year,
MAX(CASE WHEN t.name LIKE 'rule:%' THEN substr(t.name, 6) END) AS rule,
MAX(CASE WHEN t.name LIKE 'reg-docket:%' THEN substr(t.name, 12) END) AS docket
FROM items i
JOIN item_tags it ON it.item_id = i.id
JOIN tags t ON t.id = it.tag_id
WHERE i.id IN (
SELECT it2.item_id FROM item_tags it2
JOIN tags t2 ON t2.id = it2.tag_id WHERE t2.name = 'topic:palliative'
)
GROUP BY i.id
"""
).fetchall()
```
plus `bib_meta.palliative_sweep` for denominators. Chart: hits per rule year, with a `with_text` coverage line/label so the exhibit can't be read as "% of all comments". Table: sortable frame with markdown links `[comment_id](url)`. Summary prose: written from the actual snippets (read them during this task), citing 5–10 representative comments by linked comment id — themes to expect from the snippet scan: payment for interdisciplinary/home-based palliative care, ACP/care-management code adequacy, hospice-carve-out boundaries, workforce. The disclosure line must state the text-coverage denominator (~23.8k full texts + ~6.4k abstracts of 167k comments).
- [ ] **Step 4: Verify headlessly** — run the single-notebook nb_integration pass inside the notebooks container (nightly pattern): write a one-notebook `nb-tests.toml` (`[ci_smoke] notebooks = ["palliative_care_rfi.py"]`), `docker cp` script+config into the `notebooks` container, `docker exec` it, and check the report JSON for zero errors.
- [ ] **Step 5: Ruff + commit**
```bash
uv run ruff format notebooks/palliative_care_rfi.py && uv run ruff check notebooks/palliative_care_rfi.py
git status --short
git add notebooks/palliative_care_rfi.py
git commit -m "feat(notebooks): prior-year palliative comment exhibit — counts, table, themes (refs #660)"
```
### Task 5: Close out
- [ ] **Step 1:** Comment results on #660 (per-docket counts, notebook section name, Zotero sync count) and close it.
- [ ] **Step 2:** Update `palliative_rfi_project.md` memory: tag exists, sweep script path, notebook section, re-run note for when #615 (post-close re-farm) or new extractions land.

294
notebooks/palliative_care_rfi.py
View File

@@ -184,9 +184,7 @@ def _(connect, mo, pl):
OR t.name LIKE 'source:%') OR t.name LIKE 'source:%')
GROUP BY t.name ORDER BY t.name GROUP BY t.name ORDER BY t.name
""").fetchall() """).fetchall()
ident_comp = pl.DataFrame( ident_comp = pl.DataFrame([{"tag": name, "records": n} for name, n in _rows])
[{"tag": name, "records": n} for name, n in _rows]
)
mo.vstack( mo.vstack(
[ [
mo.md("**Identification composition (live from Zotero):**"), mo.md("**Identification composition (live from Zotero):**"),
@@ -506,9 +504,12 @@ def _(AMBER, RULES, TEAL, alt, family_pay, fr_md, mo, pl, proposed_for):
.then(pl.lit("Gain")) .then(pl.lit("Gain"))
.otherwise(pl.lit("Loss")) .otherwise(pl.lit("Loss"))
.alias("sign"), .alias("sign"),
(pl.col("hcpcs") + pl.when(pl.col("mssp_attribution")).then(pl.lit(" ●")).otherwise(pl.lit(""))).alias( (
"code_label" pl.col("hcpcs")
), + pl.when(pl.col("mssp_attribution"))
.then(pl.lit(" ●"))
.otherwise(pl.lit(""))
).alias("code_label"),
) )
_unpriced = family_pay.filter( _unpriced = family_pay.filter(
pl.col("dollar_2026final").is_null() | pl.col("dollar_2027proposed").is_null() pl.col("dollar_2026final").is_null() | pl.col("dollar_2027proposed").is_null()
@@ -518,8 +519,15 @@ def _(AMBER, RULES, TEAL, alt, family_pay, fr_md, mo, pl, proposed_for):
alt.Chart(_priced.to_pandas()) alt.Chart(_priced.to_pandas())
.mark_bar() .mark_bar()
.encode( .encode(
x=alt.X("pct_change:Q", title="% change, CY2026 final → CY2027 proposed (national, GPCI=1.0)"), x=alt.X(
y=alt.Y("code_label:N", title=None, sort=alt.SortField(field="pct_change", order="descending")), "pct_change:Q",
title="% change, CY2026 final → CY2027 proposed (national, GPCI=1.0)",
),
y=alt.Y(
"code_label:N",
title=None,
sort=alt.SortField(field="pct_change", order="descending"),
),
color=alt.Color( color=alt.Color(
"sign:N", "sign:N",
title=None, title=None,
@@ -527,7 +535,10 @@ def _(AMBER, RULES, TEAL, alt, family_pay, fr_md, mo, pl, proposed_for):
legend=alt.Legend(orient="top"), legend=alt.Legend(orient="top"),
), ),
tooltip=[ tooltip=[
"hcpcs", "family", "description", "mssp_attribution", "hcpcs",
"family",
"description",
"mssp_attribution",
alt.Tooltip("dollar_2026final:Q", format="$.2f"), alt.Tooltip("dollar_2026final:Q", format="$.2f"),
alt.Tooltip("dollar_2027proposed:Q", format="$.2f"), alt.Tooltip("dollar_2027proposed:Q", format="$.2f"),
alt.Tooltip("pct_change:Q", format="+.2f"), alt.Tooltip("pct_change:Q", format="+.2f"),
@@ -634,7 +645,9 @@ def _(AMBER, TEAL, alt, family_pay, mo, pl, q):
scale=alt.Scale(domainMid=0, range=[AMBER, "#EDEBE6", TEAL]), scale=alt.Scale(domainMid=0, range=[AMBER, "#EDEBE6", TEAL]),
), ),
tooltip=[ tooltip=[
"hcpcs", "family", "year:O", "hcpcs",
"family",
"year:O",
alt.Tooltip("total_rvu:Q", format=".2f"), alt.Tooltip("total_rvu:Q", format=".2f"),
alt.Tooltip("rvu_yoy_pct:Q", format="+.2f"), alt.Tooltip("rvu_yoy_pct:Q", format="+.2f"),
], ],
@@ -795,12 +808,25 @@ def _(mo, pl):
def _(family_pay, mo, pl, proposed_codes): def _(family_pay, mo, pl, proposed_codes):
# Live valuation anchors: what the named PFS analogs pay today and # Live valuation anchors: what the named PFS analogs pay today and
# under the CY2027 proposal (from Section 2's family_pay frame). # under the CY2027 proposal (from Section 2's family_pay frame).
_analog_codes = ["99497", "99498", "99495", "99490", "99487", "G0556", "G0557", "G0558"] _analog_codes = [
"99497",
"99498",
"99495",
"99490",
"99487",
"G0556",
"G0557",
"G0558",
]
analog_anchor = ( analog_anchor = (
family_pay.filter(pl.col("hcpcs").is_in(_analog_codes)) family_pay.filter(pl.col("hcpcs").is_in(_analog_codes))
.select( .select(
"hcpcs", "family", "description", "hcpcs",
"dollar_2026final", "dollar_2027proposed", "mssp_attribution", "family",
"description",
"dollar_2026final",
"dollar_2027proposed",
"mssp_attribution",
) )
.sort("family", "hcpcs") .sort("family", "hcpcs")
) )
@@ -833,7 +859,237 @@ def _(family_pay, mo, pl, proposed_codes):
return return
# ── 4. Provenance ────────────────────────────────────────────────────── # ── 4. Prior rulemaking comment record ─────────────────────────────────
@app.cell(hide_code=True)
def _(mo):
mo.md("""
## 4. Prior Rulemaking Comment Record — Palliative Care
CMS asks in the RFI what it should know about community-based
palliative care; part of the answer is that **commenters have been
telling CMS for a decade**. The bib store holds every comment from
the last ten PFS rulemaking dockets (CY2018-CY2027). A lexical
sweep (`dev/scripts/tag_palliative_comments.py`, stem
`palliativ*` over the extracted comment text, else the enriched
abstract) tags each referencing comment `topic:palliative` in bib
and Zotero, with mention counts and sentence snippets on the item.
""")
return
@app.cell(hide_code=True)
def _(connect, mo, pl):
import json as _json
# keep the Store referenced for the cell — its __del__ closes the con
_bstore = connect.bib()
_bcon = _bstore._con()
_rows = _bcon.execute("""
SELECT i.key, i.title, i.url, i.extra_json,
(SELECT substr(t.name, 6) FROM item_tags it
JOIN tags t ON t.id = it.tag_id
WHERE it.item_id = i.id AND t.name LIKE 'year:%') AS year,
(SELECT substr(t.name, 6) FROM item_tags it
JOIN tags t ON t.id = it.tag_id
WHERE it.item_id = i.id AND t.name LIKE 'rule:%') AS rule,
(SELECT substr(t.name, 12) FROM item_tags it
JOIN tags t ON t.id = it.tag_id
WHERE it.item_id = i.id AND t.name LIKE 'reg-docket:%') AS docket
FROM items i
WHERE i.id IN (
SELECT it2.item_id FROM item_tags it2
JOIN tags t2 ON t2.id = it2.tag_id
WHERE t2.name = 'topic:palliative'
)
ORDER BY year, i.title
""").fetchall()
pall_comments = pl.DataFrame(
[
{
"year": r["year"] or "",
"rule": r["rule"] or "",
"docket": r["docket"] or "",
"comment": r["title"] or "",
"url": r["url"] or "",
"mentions": (
_json.loads(r["extra_json"]).get("palliative_mentions", 0)
if r["extra_json"]
else 0
),
"snippet": (
(_json.loads(r["extra_json"]).get("palliative_snippets") or [""])[0]
if r["extra_json"]
else ""
),
}
for r in _rows
]
)
_meta_row = _bcon.execute(
"SELECT value FROM bib_meta WHERE key = 'palliative_sweep'"
).fetchone()
pall_sweep = _json.loads(_meta_row[0]) if _meta_row else {"dockets": {}}
return pall_comments, pall_sweep
@app.cell(hide_code=True)
def _(TEAL, alt, mo, pall_comments, pall_sweep, pl):
_per_year = (
pall_comments.group_by("year")
.agg(pl.len().alias("comments"), pl.col("mentions").sum().alias("mentions"))
.sort("year")
)
_covered = sum(d["with_text"] for d in pall_sweep["dockets"].values())
_total = sum(d["total"] for d in pall_sweep["dockets"].values())
pall_year_chart = (
alt.Chart(_per_year.to_pandas())
.mark_bar(color=TEAL)
.encode(
x=alt.X("year:N", title="Comment year"),
y=alt.Y("comments:Q", title="Comments referencing palliative care"),
tooltip=["year", "comments", "mentions"],
)
.properties(
title="Palliative-Care Mentions Across Ten PFS Comment Cycles (live)",
width=620,
height=220,
)
)
mo.vstack(
[
pall_year_chart,
mo.md(f"""
**Denominator disclosure:** the sweep covers the
{_covered:,} of {_total:,} docketed comments with locally
retrievable text (extracted attachment/inline text for the
ten PFS dockets, enriched abstracts otherwise) — counts
are a floor, not a census. The 2018 spike is the CY2019
E/M office-visit code-collapse proposal, which drew an
organized response from hospice-and-palliative-medicine
clinicians (see themes below).
"""),
]
)
return
@app.cell(hide_code=True)
def _(mo, pall_comments, pl):
_tbl = (
pall_comments.sort("mentions", descending=True)
.with_columns(
pl.format("[{}]({})", pl.col("comment"), pl.col("url")).alias(
"comment_link"
)
)
.select(
pl.col("year"),
pl.col("rule"),
pl.col("comment_link").alias("comment"),
pl.col("mentions"),
pl.col("snippet"),
)
)
mo.vstack(
[
mo.md(f"""
### The {pall_comments.height} referencing comments (live from bib)
Every comment tagged `topic:palliative`, most-mentions
first, with its first extracted snippet. Each link opens
the comment on regulations.gov; the same items are in
Zotero under the `topic:palliative` tag for citation from
the P44 comment draft.
"""),
mo.ui.table(
_tbl.to_pandas(),
page_size=15,
label="Prior-cycle comments referencing palliative care",
),
]
)
return
@app.cell(hide_code=True)
def _(mo):
mo.md("""
### What prior-cycle commenters told CMS
Reading the tagged set, five themes recur — each maps onto an RFI
part, and each is a decade-long thread, not a one-off:
1. **The specialty comments every cycle.** The American Academy of
Hospice and Palliative Medicine appears in all nine prior
dockets ([2017](https://www.regulations.gov/comment/CMS-2017-0092-1574),
[2018](https://www.regulations.gov/comment/CMS-2018-0076-15128),
[2019](https://www.regulations.gov/comment/CMS-2019-0111-40551),
[2020](https://www.regulations.gov/comment/CMS-2020-0088-28866),
[2021](https://www.regulations.gov/comment/CMS-2021-0119-35330),
[2022](https://www.regulations.gov/comment/CMS-2022-0113-24293),
[2023](https://www.regulations.gov/comment/CMS-2023-0121-20479),
[2024](https://www.regulations.gov/comment/CMS-2024-0256-5617),
[2025](https://www.regulations.gov/comment/CMS-2025-0304-12014)),
joined repeatedly by the National Coalition for Hospice and
Palliative Care
([2018](https://www.regulations.gov/comment/CMS-2018-0076-13835),
[2023](https://www.regulations.gov/comment/CMS-2023-0121-20746)),
CAPC ([2017](https://www.regulations.gov/comment/CMS-2017-0092-1374),
[2018](https://www.regulations.gov/comment/CMS-2018-0076-15030)),
NHPCO ([2021](https://www.regulations.gov/comment/CMS-2021-0119-34010),
[2023](https://www.regulations.gov/comment/CMS-2023-0121-21390)),
and nursing/PA counterparts
([HPNA 2019](https://www.regulations.gov/comment/CMS-2019-0111-8492),
[PAHPM 2019](https://www.regulations.gov/comment/CMS-2019-0111-29255)).
2. **Payment adequacy for long, complex visits — RFI part (d).**
The 2018 spike is HPM clinicians opposing the CY2019 E/M
code-collapse, citing AMA analysis of a ~20% office-visit cut
for the specialty
([physician letters](https://www.regulations.gov/comment/CMS-2018-0076-14483),
[UPenn program director](https://www.regulations.gov/comment/CMS-2018-0076-13362));
the same adequacy argument returns for the G2211-style
complexity add-on and Principal Illness Navigation in
[2024](https://www.regulations.gov/comment/CMS-2024-0256-6729),
asking CMS to make "palliative care in the community
sustainable under Part B."
3. **Team-based and non-physician delivery — RFI parts (c)/(d).**
Recurring asks to recognize interdisciplinary palliative teams
(chaplains, social workers, nurses) and PA/NP practice
([OHSU PA 2019](https://www.regulations.gov/comment/CMS-2019-0111-31124),
[NHPCO on PAs in hospice 2019](https://www.regulations.gov/comment/CMS-2019-0111-40582)).
4. **Quality measurement — RFI part (e).** From 2023 on,
commenters push MIPS measure #495 ("Ambulatory Palliative Care
Patients' Experience of Feeling Heard and Understood") and the
NCP Clinical Practice Guidelines / NASW standards as the
measurement backbone
([2023](https://www.regulations.gov/comment/CMS-2023-0121-20811),
[AGS 2024](https://www.regulations.gov/comment/CMS-2024-0256-6812),
[2025](https://www.regulations.gov/comment/CMS-2025-0304-13362)) —
the same instruments Section 3's PCM-3 safeguard cites.
5. **Access framing beyond fee-for-service — RFI parts (b)/(d).**
Telehealth equivalence for home-based palliative care
([2025](https://www.regulations.gov/comment/CMS-2025-0304-12156)),
palliative care as a preventive service per Hawaii's Medicaid
state plan
([2025](https://www.regulations.gov/comment/CMS-2025-0304-12096)),
and underutilization despite evidence
([2024](https://www.regulations.gov/comment/CMS-2024-0256-4853)).
For the P44 draft this exhibit supports one sentence CMS cannot
easily dismiss: the RFI's questions have standing, organized,
decade-deep constituencies on the record — and the record shows
exactly where existing codes fell short each time.
""")
return
# ── 5. Provenance ──────────────────────────────────────────────────────
@app.cell(hide_code=True) @app.cell(hide_code=True)
@@ -873,7 +1129,7 @@ def _(mo, q):
mo.vstack( mo.vstack(
[ [
mo.md(""" mo.md("""
## 4. Provenance ## 5. Provenance
Ingest-log rows for the lake tables Section 2 reads, plus Ingest-log rows for the lake tables Section 2 reads, plus
the evidence-base pipeline: identification — the evidence-base pipeline: identification —
@@ -882,9 +1138,11 @@ def _(mo, q):
and grey-literature standards documents; run 2026-08-18); and grey-literature standards documents; run 2026-08-18);
screening — `stack prisma screen palliative-rfi` with the screening — `stack prisma screen palliative-rfi` with the
project's Zotero criteria anchors; flow — `prisma.flow` project's Zotero criteria anchors; flow — `prisma.flow`
over live Zotero tags. `cms.primary_care_service_code` is over live Zotero tags. Section 4's comment record —
the aco module's attribution reference list (P39), not an `dev/scripts/tag_palliative_comments.py` (lexical sweep,
FR ingest. run 2026-08-19; stats in `bib_meta.palliative_sweep`).
`cms.primary_care_service_code` is the aco module's
attribution reference list (P39), not an FR ingest.
"""), """),
mo.ui.table( mo.ui.table(
provenance.to_pandas(), provenance.to_pandas(),