Files
stack/tests/rec/pricers/test_pfs.py
kert 2a2df70e21
Some checks failed
CI / skinny-install (aco) (push) Successful in 1m1s
CI / skinny-install (api) (push) Successful in 40s
CI / skinny-install (bcda) (push) Successful in 40s
CI / skinny-install (bib) (push) Successful in 45s
CI / skinny-install (bls) (push) Successful in 33s
CI / skinny-install (ccw) (push) Successful in 56s
CI / skinny-install (cli) (push) Successful in 1m20s
CI / lint-test (push) Successful in 6m31s
CI / skinny-install (cms) (push) Successful in 56s
CI / skinny-install (conf) (push) Successful in 1m29s
CI / skinny-install (opps) (push) Successful in 1m15s
CI / skinny-install (perf) (push) Successful in 55s
CI / skinny-install (pfs) (push) Successful in 55s
CI / skinny-install (rex) (push) Successful in 56s
Infra CI / notebooks (push) Successful in 13s
Infra CI / zotero (push) Successful in 13s
Infra CI / docs (push) Failing after 20s
Infra CI / api (push) Successful in 12s
Infra CI / mc (push) Successful in 12s
Deploy / build-scan-report (push) Failing after 3m37s
Harden / build-scan-report (push) Failing after 13m27s
Package Supply Chain / pkg-supply-chain (push) Failing after 58s
feat(rec): reconciliation module with PFS pricer — tracks #340
Build the `rec` module to drive calculated payments toward 1:1 parity
with CMS-published ground-truth files. The notebook at
notebooks/pfs_calcs.py revealed our pfs.calcs.payment output diverges
from pfs.carrier_locality for some year × locality × HCPCS combos;
rec lets us measure the gap, track it year-over-year, and extend to
OPPS / IPPS / ASC / DMEPOS without touching the engine.

Design — open/closed:

- src/rec/base.py, engine.py, report.py, cli.py are closed for
  modification. Adding a new payment system = one new file under
  src/rec/pricers/ implementing the Pricer protocol + one line in
  rec.pricers.__init__.PRICERS.

- Pricer is a @runtime_checkable Protocol with ClassVar
  system/description/join_keys/compare_cols and three methods
  (ground_truth, calculated, years_available). isinstance() is used
  by the engine to validate conformance without subclassing.

- reconcile(pricer, con, year) returns a frozen Reconciliation
  dataclass with scalar counts, a per-row deltas polars DataFrame,
  exact/near match counts, and non-fatal warnings. Matched rows are
  classified as exact (≤ tolerance_cents) or near (≤ 1¢). The delta
  table contains <col>_gt / <col>_calc / <col>_delta + abs_max_delta
  + is_exact / is_near flags, sorted by worst delta first.

- rec.report.as_markdown / as_json format the Reconciliation for
  the CLI. markdown includes counts table and top-N deltas.

- src/cli/rec.py is a Typer sub-app registered under `stack rec`:
    stack rec list                    # registered pricers
    stack rec pfs --year 2025         # single year
    stack rec pfs                     # all available years
    stack rec pfs --format json --output out.json
  src/rec/__main__.py mirrors the same via `python -m rec`.

PfsPricer implementation (src/rec/pricers/pfs.py):

- ground_truth: SELECT from pfs.carrier_locality WHERE year = ?
- calculated:
    1. RULES[year].conversion_factor replaces the stale
       pfs.rvu.conv_factor column (rvu file values are unreliable;
       the notebook already did this override).
    2. Filters pfs.rvu to status_code IN ('A', 'T') so carrier-priced
       codes (status 'C') show up as ground_truth_only instead of
       being flagged as bugs.
    3. Cross-joins rvu × distinct(mac, locality) from gpci, then
       calls pfs.calcs.payment.payment() twice (facility=False /
       True), rounds to cents, and joins the two halves.
- join_keys: [year, mac, locality, hcpcs, mod]
- compare_cols: [non_fac_fee, fac_fee] — limiting-charge columns
  are omitted because pfs/pipe.py synthesises them from
  non_fac_fee × 1.0925 at load time; reconciling against a value
  we computed ourselves is circular. Documented in the module
  docstring as a tracked gap (#340 gap #3).

Wiring:

- pyproject.toml: new rec optional dep group (stack[conf,pfs] +
  duckdb + narwhals + typer), added to stack[all], module-name
  list extended to include "rec".
- stack.toml [storage]: rec = "data/rec" for report artifacts.
- src/cli/__init__.py registers rec_app under `stack rec`.

Tests (37, all green):

- tests/rec/conftest.py seeds an in-memory DuckDB with four rows
  designed to exercise the full match matrix: exact, 1¢ near, GT-only
  (carrier-priced), calc-only. Also defines a FakePricer class for
  engine tests that don't need DuckDB.
- tests/rec/test_base.py: protocol conformance, dataclass shape,
  pct_exact / is_perfect derived properties, frozen immutability.
- tests/rec/test_engine.py: counts, exact vs near classification,
  tolerance promotion, delta table shape and ordering, rejection of
  non-conforming objects.
- tests/rec/test_report.py: markdown section headers, JSON
  round-trip.
- tests/rec/pricers/test_pfs.py: registration, protocol conformance,
  ground_truth row count, status-code filter, cent rounding,
  summary counts against the seeded fixture, tolerance promotion,
  missing-year KeyError.

Notebook:

- notebooks/pfs_reconciliation.py — marimo interactive runner with
  year dropdown, tolerance slider, rendered summary markdown, delta
  table, and warnings panel. Mirrors the structure of pfs_calcs.py.

Gaps surfaced by this work (not fixed in this commit, to be tracked
as follow-up issues):

1. pfs.calcs.payment.payment() ignores RuleYear.budget_neutrality_adjustor.
   Silently correct for 2014–2025 since the value is 1.0, but a
   future non-1 year will drift.
2. _LIMITING_CHARGE_FACTOR = 1.0925 is a magic constant inside
   pfs/pipe.py at load time. Should be extracted to pfs.calcs and
   the synthesized columns should be superseded by an actual CMS
   source if one exists.
3. pfs.rvu.conv_factor is stale / often NULL. Single source of
   truth should be pfs.rules.RULES; the column could be dropped.
4. Carrier file doesn't capture the header's stated CF so we can't
   cross-check RULES[year].conversion_factor matches the file at
   ingest time.
2026-04-10 20:24:31 -04:00

110 lines
3.7 KiB
Python

"""Tests for rec.pricers.pfs.PfsPricer with a seeded in-memory DuckDB."""
from __future__ import annotations
from rec.base import Pricer
from rec.engine import reconcile
from rec.pricers import PRICERS
from rec.pricers.pfs import PfsPricer
class TestRegistration:
def test_in_PRICERS(self) -> None:
assert "pfs" in PRICERS
assert PRICERS["pfs"] is PfsPricer
def test_conforms_to_protocol(self) -> None:
assert isinstance(PfsPricer(), Pricer)
def test_class_attributes(self) -> None:
assert PfsPricer.system == "pfs"
assert PfsPricer.join_keys == ["year", "mac", "locality", "hcpcs", "mod"]
assert PfsPricer.compare_cols == ["non_fac_fee", "fac_fee"]
class TestGroundTruth:
def test_row_count(self, con, test_year) -> None:
p = PfsPricer()
gt = p.ground_truth(con, test_year)
# Seeded 3 carrier rows (99213, 99214, 99999)
assert gt.height == 3
def test_columns(self, con, test_year) -> None:
p = PfsPricer()
gt = p.ground_truth(con, test_year)
for col in [
"year",
"mac",
"locality",
"hcpcs",
"mod",
"non_fac_fee",
"fac_fee",
]:
assert col in gt.columns
class TestCalculated:
def test_excludes_carrier_priced(self, con, test_year) -> None:
"""99999 has status_code='C' so calc side skips it."""
p = PfsPricer()
calc = p.calculated(con, test_year)
hcpcs = calc["hcpcs"].to_list()
assert "99999" not in hcpcs
def test_includes_algorithmic(self, con, test_year) -> None:
"""99213, 99214, 88888 all have status 'A'."""
p = PfsPricer()
calc = p.calculated(con, test_year)
hcpcs = set(calc["hcpcs"].to_list())
assert {"99213", "99214", "88888"}.issubset(hcpcs)
def test_returns_cents_rounded(self, con, test_year) -> None:
p = PfsPricer()
calc = p.calculated(con, test_year)
# All values should be round to 2 decimals (cents)
for fee in calc["non_fac_fee"].to_list() + calc["fac_fee"].to_list():
assert fee == round(fee, 2)
class TestReconcilePfs:
def test_summary_counts(self, con, test_year) -> None:
r = reconcile(PfsPricer(), con, test_year)
# Ground-truth rows: 99213, 99214, 99999 = 3
assert r.ground_truth_rows == 3
# Calculated rows: 99213, 99214, 88888 = 3 (99999 skipped)
assert r.calculated_rows == 3
# Matched: 99213, 99214 = 2
assert r.matched_rows == 2
# gt_only: 99999
assert r.ground_truth_only == 1
# calc_only: 88888
assert r.calculated_only == 1
def test_exact_and_near(self, con, test_year) -> None:
r = reconcile(PfsPricer(), con, test_year)
# 99213 seeded to match exactly; 99214 is 1¢ off on non_fac_fee.
assert r.exact_matches == 1
assert r.near_matches == 2
def test_tolerance_promotes(self, con, test_year) -> None:
r = reconcile(PfsPricer(), con, test_year, tolerance_cents=1)
assert r.exact_matches == 2
def test_delta_table_contains_hcpcs(self, con, test_year) -> None:
r = reconcile(PfsPricer(), con, test_year)
hcpcs = set(r.deltas["hcpcs"].to_list())
# All four distinct hcpcs from outer join
assert hcpcs == {"99213", "99214", "99999", "88888"}
def test_years_available(self, con, test_year) -> None:
p = PfsPricer()
assert p.years_available(con) == [test_year]
def test_missing_year_raises(self, con) -> None:
import pytest
p = PfsPricer()
with pytest.raises(KeyError, match="RuleYear missing"):
p.calculated(con, 1999)