Files
stack/src/bcda/__init__.py
kert d08401dc53 feat(bcda): BCDA v3 — client, store adoption and cancel retries, dictionary-driven v3 flatteners, streaming multi-part load
Client defaults to /api/v3; _typeFilter on start_export/export with
type_filter_tag() for the System-Type tag; jobs() sends the bulk-data headers
BCDA requires (Prefer: respond-async, Accept: application/fhir+json — the old
call 400'd on v2 and v3) and unwraps the Task Bundle; completed_jobs() lists
reusable exports; every non-2xx raises BcdaHttpError carrying the
OperationOutcome diagnostics (still an httpx.HTTPStatusError); a poll that
hits 404 'Job has been cancelled' raises JobCancelledError; /_version and
/_health.

Store.adopt(client, job_url) records and downloads a job this store did not
start (completed jobs stay downloadable 24 h and the public sandbox entity is
shared); export(cancel_retries=N) restarts after server-side cancellation,
recording each 'cancelled' transition.

bcda.express.v3 flattens C4BB 2.2.0 resources into the existing bcda.* tables,
table-driven from the v3 Data Dictionary: v3 moved every BlueButton code off
resources/variables/<name> onto fhir/StructureDefinition|CodeSystem/<CODE> and
consolidated amounts into adjudication/total/item.adjudication, so the v2
suffix match returned NULL for almost everything. Dispatch is per resource
(is_v3); v2 payloads are unchanged. Identity follows the migration guide:
bene_id = Patient.id (BENE_SK), current MBI = us-mbi identifier without
period.end, clm_id = C4BB 'uc', clm_cntl_num pairs SharedSystem with NCH.
Race crosswalked from US Core OMB codes; Coverage is one row per plan+period
(Part A/B/C/D + Dual) with v3-native columns; EOB gains System-Type /
Final-Action / EOB-TYPE, IDR dates, header totals and PPS/IME/DSH amounts;
items gain NDC quantity, revenue-centre indicators, ANSI codes, Part D/DME
amounts.

flatten_export streams <Type>.N.ndjson parts to Parquet with model-derived
Arrow schemas (a Large-entity EOB export is 192 files / 2.2 GB); load_bcda()
gathers the newest export dir per resource type (a v3 pull is three
single-type jobs). cclf.stg_beneficiary_demographics prefers the Part A row
and joins the Dual row; eligibility reads coalesce(dual_01, dual_stus_cd).

bcda package at 100% line coverage; tests run on CMS's official v3 samples.

refs #1139
2026-10-25 13:08:13 -04:00

100 lines
4.0 KiB
Python

"""bcda -- Beneficiary Claims Data API client and table models.
BCDA is the CMS Beneficiary Claims Data API, a FHIR R4-based
alternative to CCLF flat files for ACO claims data access. BCDA
delivers Medicare claims and beneficiary data via bulk FHIR ndjson
exports.
The table models represent the **flattened** FHIR R4 payload
structure — what you actually get in the ndjson lines — not the
CCLF field names.
Usage::
from bcda.client import Client
with Client(client_id="...", client_secret="...") as c:
files = c.export(types=["Patient", "ExplanationOfBenefit"])
Submodules
----------
``bcda.client``
HTTP client for the BCDA v3 bulk FHIR export API (``version="v2"``
still works for the legacy REACH sandbox entities). Handles auth,
``_typeFilter``, job management, polling, cancellation detection
and ndjson download; non-2xx responses carry the server's
OperationOutcome diagnostics.
``bcda.store``
Persistent storage and state management. Wraps the client
with append-only Parquet state tracking and configurable file
storage (local, network, S3, Azure Blob via fsspec).
Supports retry / resume of interrupted exports.
``bcda.log``
JSONL structured logging handler. Attaches to the ``bcda``
logger hierarchy to capture full operation detail into an
append-only ``.jsonl`` file.
``bcda.express``
``flatten`` turns the raw ndjson into the ``bcda.*`` tables,
detecting the payload dialect per resource: the v2
(``resources/variables``) flattener in ``flatten`` itself, the v3
(C4BB 2.2.0, ``fhir/StructureDefinition``) flattener in
``bcda.express.v3`` — table-driven from the BCDA v3 Data
Dictionary (``dev/vendor/bcda-site``, ``dev/seeds/bcda_v3_*.csv``).
``cclf`` holds the Narwhals expression functions that transform
those tables into input_layer (medical_claim, pharmacy_claim,
eligibility).
``bcda.pipe``
Pipeline definitions wiring express functions into executable
DAGs via ``aco.pipe.base.Pipeline``.
``bcda.table``
Pydantic ``SQLTable`` models for 8 BCDA data entities:
3 adjudicated FHIR R4 resources (Patient, ExplanationOfBenefit,
Coverage) plus EobItem for line items, and 4 partially
adjudicated tables (FISS/MCS Claim and ClaimResponse).
Adjudicated Claims — FHIR R4 Resources
----------------------------------------
Delivered via ``/api/{v2,v3}/{Resource}/$export`` bulk endpoints
(v3 since 2026-07-01; v1/v2 are discontinued on 2027-07-30):
::
| Table | FHIR R4 Resource | v3 additions |
|-------------------------|------------------------|----------------------------------|
| patient | Patient | contact language, source keys |
| explanation_of_benefit | ExplanationOfBenefit | System-Type/Final-Action tags, |
| | | claim control number, IDR totals |
| eob_item | ExplanationOfBenefit | consolidated line adjudication |
| coverage | Coverage | one row per plan + period, Dual |
``ExplanationOfBenefit`` is split into header-level fields
(``explanation_of_benefit``) and repeating line items
(``eob_item``) since the FHIR resource is hierarchical.
Partially Adjudicated Claims
-----------------------------
In v3 these arrive as ``ExplanationOfBenefit`` tagged
``System-Type|SharedSystem`` (request them with ``_typeFilter``, see
``bcda.client.type_filter_tag``); the tables below are the v2
``Claim``/``ClaimResponse`` shape, kept for v2 exports. Updated
daily (2-4 day lag). Sourced from FISS (institutional) and MCS
(professional) processing systems:
::
| Table | Fields | Source | FHIR Resource |
|----------------------|--------|--------|---------------|
| fiss_claim | 40 | FISS | Claim |
| fiss_claim_response | 14 | FISS | ClaimResponse |
| mcs_claim | 28 | MCS | Claim |
| mcs_claim_response | 9 | MCS | ClaimResponse |
"""