fix(pfs): CPT parser — chapter title parents its headings, numeric range containment, C9 winners verified on real editions (refs #687)
C10, two Important findings on C9 traced to one wrong structural
assumption: div.ch-title ("Surgery Guidelines", "Surgery", "Anesthesia
Guidelines", …) was being treated as a level-1 heading, so the next
real h1 popped it off the stack — every guideline-reprint heading's
path was flat (just its own title), never nested under its chapter.
1. ch-title is now the level-0 parent of every heading in its own
chapter-group and every later chapter-group that carries no banner
of its own — replaced only by the next ch-title, never popped by an
hN (level >= 1 never pops level 0). Most chapter files (a chapter
split across many numbered files for size, e.g. 2019's Surgery body
spans Chapter07-16 with the banner only on Chapter06) carry no
ch-title at all, so _group now takes/returns the active chapter
title and parse_epub threads it from one chapter-group's call into
the next's.
Fixing this exposed a second, previously-inert bug: _chapter_groups
returned groups in zf.namelist() order, which is book order in 2019
but arbitrarily scrambled in 2021/2022/2024 (verified directly).
That never mattered pre-C10 (groups were parsed independently); with
title-threading it corrupted paths across unrelated chapters (21089
landing under "Medicine Guidelines" in 2021, "Pathology and
Laboratory" in 2024). Groups are now sorted by chapter number.
2. _in_range now compares numerically on min(lo, hi)..max(lo, hi) — a
resequenced heading's own TOC span can print in display order rather
than numeric order (lo > hi), which the old plain string comparison
silently got wrong for.
3. C9's scoring formula is unchanged. Verified on all four real
editions: 21089 (Surgery) now resolves under "Surgery > Head > Head
Prosthesis > Other Procedures", its "Surgery Guidelines > Unlisted
Service or Procedure" copy lands in cpt_code_alt (reason
guidelines-reprint); 99100 resolves to Medicine's ranged "Qualifying
Circumstances for Anesthesia" (the book's only TOC-declared range for
that code), with Anesthesia's own "Qualifying Circumstances" as the
alternate — documented in parse_epub's docstring as the accepted
outcome, not a bug. 99490 (2024) path_key is now "Evaluation and
Management > Care Management Services > Chronic Care Management
Services", matching the brief's original example exactly.
guidelines-reprint > 0 in cpt_code_alt for every edition (2019: 132,
2021: 130, 2022: 38, 2024: 130).
4. Minors: losing entries' instructions/references are deduped by
(code, kind, text) before resolving alternates, so an exact-repeat
parenthetical note from a guideline reprint doesn't survive as a
spurious extra row; removed the redundant ensure_tables call in
cpt-ingest's write path (write_cpt_edition already calls it);
_in_range's docstring documents the resequenced-span reversal.
This commit is contained in:
@@ -317,7 +317,8 @@ def cpt_ingest_cmd(
|
|||||||
_print_cpt_counts(counts)
|
_print_cpt_counts(counts)
|
||||||
return
|
return
|
||||||
with _batch() as con:
|
with _batch() as con:
|
||||||
ensure_tables(con)
|
# write_cpt_edition (via cpt_ingest) already calls ensure_tables
|
||||||
|
# itself — no need to duplicate that here.
|
||||||
counts = cpt_ingest(store, con, years=years, dry_run=False)
|
counts = cpt_ingest(store, con, years=years, dry_run=False)
|
||||||
_print_cpt_counts(counts)
|
_print_cpt_counts(counts)
|
||||||
_publish()
|
_publish()
|
||||||
|
|||||||
@@ -279,8 +279,25 @@ def _category(code: str) -> str:
|
|||||||
|
|
||||||
|
|
||||||
def _group(
|
def _group(
|
||||||
events: list[tuple], *, year: int, source: str = ""
|
events: list[tuple], *, year: int, source: str = "", chapter_title: str = ""
|
||||||
) -> tuple[list[CptSection], list[CptCode], list[CptInstruction], list[CptReference]]:
|
) -> tuple[
|
||||||
|
list[CptSection], list[CptCode], list[CptInstruction], list[CptReference], str
|
||||||
|
]:
|
||||||
|
"""Group one chapter-group's flat event stream (see module docstring)
|
||||||
|
into sections/codes/instructions/references, plus the *active*
|
||||||
|
chapter title on exit (5th return value).
|
||||||
|
|
||||||
|
``chapter_title`` (C10) seeds ``heading_stack`` as a level-0 parent:
|
||||||
|
``div.ch-title`` is the book's own chapter banner ("Surgery
|
||||||
|
Guidelines", "Surgery", "Anesthesia Guidelines", …) and a real
|
||||||
|
``hN`` heading nests *under* it — it is popped only by the next
|
||||||
|
``ch-title``, never by an ``hN`` (level ≥ 1 never pops level 0). Most
|
||||||
|
chapter files carry no ``ch-title`` of their own at all (a chapter
|
||||||
|
split across many numbered files for size, e.g. 2019's Surgery body
|
||||||
|
spans Chapter07 through Chapter16 with the banner only on Chapter06)
|
||||||
|
— ``parse_epub`` threads the returned title from one chapter-group's
|
||||||
|
call into the next one's ``chapter_title`` argument so it keeps
|
||||||
|
parenting headings until a new banner replaces it."""
|
||||||
sections: list[CptSection] = []
|
sections: list[CptSection] = []
|
||||||
codes: list[CptCode] = []
|
codes: list[CptCode] = []
|
||||||
instructions: list[CptInstruction] = []
|
instructions: list[CptInstruction] = []
|
||||||
@@ -288,7 +305,10 @@ def _group(
|
|||||||
|
|
||||||
toc_ranges = _toc_ranges_from_events(events)
|
toc_ranges = _toc_ranges_from_events(events)
|
||||||
|
|
||||||
heading_stack: list[tuple[int, str]] = []
|
# C10: the (possibly inherited) chapter title is the level-0 base of
|
||||||
|
# the stack from the very start of this chapter-group, so the first
|
||||||
|
# real heading already nests under it.
|
||||||
|
heading_stack: list[tuple[int, str]] = [(0, chapter_title)] if chapter_title else []
|
||||||
current: dict | None = None
|
current: dict | None = None
|
||||||
guideline_buf: list[str] = []
|
guideline_buf: list[str] = []
|
||||||
in_guideline_zone = False
|
in_guideline_zone = False
|
||||||
@@ -461,27 +481,31 @@ def _group(
|
|||||||
continue # consumed by the toc_ranges pre-pass
|
continue # consumed by the toc_ranges pre-pass
|
||||||
|
|
||||||
is_heading = any(_HEADING_RE.match(c) for c in classes)
|
is_heading = any(_HEADING_RE.match(c) for c in classes)
|
||||||
# A few chapters (e.g. "Category III Codes") never carry a
|
# C10: div.ch-title is the book's own chapter banner ("Surgery
|
||||||
# div.hN at all — the chapter's own div.ch-title is the only
|
# Guidelines", "Surgery", …) — the level-0 parent of every
|
||||||
# heading before the code table. Treat it as a level-1 heading
|
# heading in this chapter (and every later chapter-group that
|
||||||
# too, so its codes land under a real, titled section instead
|
# doesn't carry its own banner, via the seeded heading_stack
|
||||||
# of collapsing onto sec_id="" (it's superseded immediately,
|
# above), replaced only by the next ch-title, never popped by a
|
||||||
# same level, the moment a real h1 shows up in chapters that
|
# real hN (level ≥ 1 never pops level 0). A chapter with no hN
|
||||||
# have both).
|
# of its own at all (e.g. "Category III Codes") still gets a
|
||||||
|
# real, titled section this way instead of collapsing onto
|
||||||
|
# sec_id="" — it's just level 0 doing that job now, not level 1.
|
||||||
is_chapter_title = "ch-title" in classes
|
is_chapter_title = "ch-title" in classes
|
||||||
if is_heading or is_chapter_title:
|
if is_heading or is_chapter_title:
|
||||||
finalize_entry()
|
finalize_entry()
|
||||||
finalize_section()
|
finalize_section()
|
||||||
if is_heading:
|
title = text.rstrip("*").strip()
|
||||||
|
if is_chapter_title:
|
||||||
|
level = 0
|
||||||
|
chapter_title = title
|
||||||
|
heading_stack = [(0, title)]
|
||||||
|
else:
|
||||||
level = int(
|
level = int(
|
||||||
next(m for c in classes if (m := _HEADING_RE.match(c))).group(1)
|
next(m for c in classes if (m := _HEADING_RE.match(c))).group(1)
|
||||||
)
|
)
|
||||||
else:
|
while heading_stack and heading_stack[-1][0] >= level:
|
||||||
level = 1
|
heading_stack.pop()
|
||||||
title = text.rstrip("*").strip()
|
heading_stack.append((level, title))
|
||||||
while heading_stack and heading_stack[-1][0] >= level:
|
|
||||||
heading_stack.pop()
|
|
||||||
heading_stack.append((level, title))
|
|
||||||
path = tuple(t for _, t in heading_stack)
|
path = tuple(t for _, t in heading_stack)
|
||||||
sec_id = id_
|
sec_id = id_
|
||||||
if not sec_id:
|
if not sec_id:
|
||||||
@@ -553,11 +577,11 @@ def _group(
|
|||||||
finalize_entry()
|
finalize_entry()
|
||||||
finalize_section()
|
finalize_section()
|
||||||
|
|
||||||
return sections, codes, instructions, references
|
return sections, codes, instructions, references, chapter_title
|
||||||
|
|
||||||
|
|
||||||
def parse_xhtml(
|
def parse_xhtml(
|
||||||
text: str, *, year: int, source: str = ""
|
text: str, *, year: int, source: str = "", chapter_title: str = ""
|
||||||
) -> tuple[list[CptSection], list[CptCode], list[CptInstruction], list[CptReference]]:
|
) -> tuple[list[CptSection], list[CptCode], list[CptInstruction], list[CptReference]]:
|
||||||
"""Parse one body-chapter xhtml string (optionally with its chapter
|
"""Parse one body-chapter xhtml string (optionally with its chapter
|
||||||
TOC snippet concatenated in, for lo/hi — see module docstring and
|
TOC snippet concatenated in, for lo/hi — see module docstring and
|
||||||
@@ -566,11 +590,20 @@ def parse_xhtml(
|
|||||||
``source`` (the chapter file's basename, when called from
|
``source`` (the chapter file's basename, when called from
|
||||||
``parse_epub``) seeds the synthetic id given to a heading that has
|
``parse_epub``) seeds the synthetic id given to a heading that has
|
||||||
none of its own (2019's headings mostly lack one) — see
|
none of its own (2019's headings mostly lack one) — see
|
||||||
``_group``'s ``heading_counter``."""
|
``_group``'s ``heading_counter``. ``chapter_title`` (C10) seeds the
|
||||||
|
level-0 chapter-banner parent of every heading in ``text`` — direct
|
||||||
|
callers (tests) default to none, matching a chapter's own first
|
||||||
|
``div.ch-title``; ``parse_epub`` threads it itself via ``_group``
|
||||||
|
directly (this wrapper's return stays a 4-tuple, dropping ``_group``'s
|
||||||
|
trailing updated-title element, since ``parse_epub`` is the only
|
||||||
|
caller that needs to carry it forward)."""
|
||||||
tok = _Tokenizer()
|
tok = _Tokenizer()
|
||||||
tok.feed(text)
|
tok.feed(text)
|
||||||
tok.close()
|
tok.close()
|
||||||
return _group(tok.events, year=year, source=source)
|
sections, codes, instructions, references, _chapter_title = _group(
|
||||||
|
tok.events, year=year, source=source, chapter_title=chapter_title
|
||||||
|
)
|
||||||
|
return sections, codes, instructions, references
|
||||||
|
|
||||||
|
|
||||||
# --- Appendices -------------------------------------------------------
|
# --- Appendices -------------------------------------------------------
|
||||||
@@ -659,7 +692,16 @@ def _chapter_groups(names: list[str]) -> list[list[str]]:
|
|||||||
sec_id="". Concatenating a chapter's files into one document before
|
sec_id="". Concatenating a chapter's files into one document before
|
||||||
calling ``parse_xhtml`` lets the same heading_stack/`current`
|
calling ``parse_xhtml`` lets the same heading_stack/`current`
|
||||||
machinery carry the last open heading across the file boundary,
|
machinery carry the last open heading across the file boundary,
|
||||||
with no change to ``_group`` itself."""
|
with no change to ``_group`` itself.
|
||||||
|
|
||||||
|
Groups are returned sorted by chapter *number*, not by
|
||||||
|
``names``/``zf.namelist()`` order — the zip's own member order is
|
||||||
|
not book order in every edition (verified: 2019's happens to match,
|
||||||
|
2021/2022/2024's don't, entries interleaved arbitrarily). This
|
||||||
|
didn't matter before C10 (each group was parsed independently), but
|
||||||
|
C10 threads the active chapter title from one group's call into the
|
||||||
|
next's, so processing them out of book order would parent headings
|
||||||
|
under the wrong chapter entirely."""
|
||||||
order: list[str] = []
|
order: list[str] = []
|
||||||
grouped: dict[str, list[str]] = {}
|
grouped: dict[str, list[str]] = {}
|
||||||
for name in names:
|
for name in names:
|
||||||
@@ -674,21 +716,43 @@ def _chapter_groups(names: list[str]) -> list[list[str]]:
|
|||||||
grouped[key] = []
|
grouped[key] = []
|
||||||
order.append(key)
|
order.append(key)
|
||||||
grouped[key].append(name)
|
grouped[key].append(name)
|
||||||
return [sorted(grouped[key]) for key in order]
|
|
||||||
|
def _chapter_sort_key(k: str) -> tuple[int, str]:
|
||||||
|
try:
|
||||||
|
return (int(k), "")
|
||||||
|
except ValueError:
|
||||||
|
return (10**9, k) # no Chapter<N> match at all: keep, sort last
|
||||||
|
|
||||||
|
return [sorted(grouped[key]) for key in sorted(order, key=_chapter_sort_key)]
|
||||||
|
|
||||||
|
|
||||||
def _has_range(sec: CptSection) -> bool:
|
def _has_range(sec: CptSection) -> bool:
|
||||||
return bool(sec.code_lo and sec.code_hi)
|
return bool(sec.code_lo and sec.code_hi)
|
||||||
|
|
||||||
|
|
||||||
|
def _range_key(v: str) -> tuple[int, str]:
|
||||||
|
"""Numeric sort key for a 5-char CPT code: the 4-digit numeric
|
||||||
|
prefix as an int, then the whole string as a tiebreaker. The prefix
|
||||||
|
alone already orders Category II/III (``NNNNF``/``NNNNT``) codes
|
||||||
|
correctly — the trailing letter is constant across one range, so it
|
||||||
|
never needs to enter the comparison."""
|
||||||
|
digits = v[:4]
|
||||||
|
if digits.isdigit():
|
||||||
|
return (int(digits), v)
|
||||||
|
return (-1, v) # malformed guard: sorts first, never matches a real range
|
||||||
|
|
||||||
|
|
||||||
def _in_range(code: str, lo: str, hi: str) -> bool:
|
def _in_range(code: str, lo: str, hi: str) -> bool:
|
||||||
"""5-char string comparison (the brief's own convention) — CPT
|
"""Numeric containment check (C10). A resequenced heading's own TOC
|
||||||
codes are always 5 characters, and this reads Category II/III
|
span can print in the book's *display* order rather than numeric
|
||||||
(``NNNNF``/``NNNNT``) ranges the same way as Category I ranges since
|
order — e.g. "Qualifying Circumstances* (99490-99427)" — so ``lo``
|
||||||
the letter suffix is constant within a range."""
|
is not guaranteed to be the smaller endpoint; this always resolves
|
||||||
|
the true numeric ``min``/``max`` of the pair before comparing,
|
||||||
|
rather than assuming ``lo <= hi``."""
|
||||||
if not lo or not hi:
|
if not lo or not hi:
|
||||||
return False
|
return False
|
||||||
return lo <= code <= hi
|
lo_key, hi_key = sorted((_range_key(lo), _range_key(hi)))
|
||||||
|
return lo_key <= _range_key(code) <= hi_key
|
||||||
|
|
||||||
|
|
||||||
def _is_guideline_path(path: tuple[str, ...]) -> bool:
|
def _is_guideline_path(path: tuple[str, ...]) -> bool:
|
||||||
@@ -772,6 +836,40 @@ def _resolve_alternates(
|
|||||||
return canonical, alternates
|
return canonical, alternates
|
||||||
|
|
||||||
|
|
||||||
|
def _dedupe_instructions(instructions: list[CptInstruction]) -> list[CptInstruction]:
|
||||||
|
"""Collapse exact-duplicate ``(code, kind, text)`` rows. A losing
|
||||||
|
guideline-reprint entry (C9/C10) and its real counterpart often
|
||||||
|
print the identical parenthetical note verbatim; there's no
|
||||||
|
per-entry ``sec_id`` on ``CptInstruction`` to attribute one row to
|
||||||
|
one specific ``CptCode`` occurrence, so this is the loser's
|
||||||
|
instructions being dropped in the one case that's unambiguous — an
|
||||||
|
exact repeat — leaving any of its instructions with genuinely
|
||||||
|
distinct text untouched."""
|
||||||
|
seen: set[tuple[str, str, str]] = set()
|
||||||
|
out = []
|
||||||
|
for i in instructions:
|
||||||
|
key = (i.code, i.kind, i.text)
|
||||||
|
if key in seen:
|
||||||
|
continue
|
||||||
|
seen.add(key)
|
||||||
|
out.append(i)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _dedupe_references(references: list[CptReference]) -> list[CptReference]:
|
||||||
|
"""Collapse exact-duplicate ``(code, kind, text)`` rows — see
|
||||||
|
``_dedupe_instructions``."""
|
||||||
|
seen: set[tuple[str, str, str]] = set()
|
||||||
|
out = []
|
||||||
|
for r in references:
|
||||||
|
key = (r.code, r.kind, r.text)
|
||||||
|
if key in seen:
|
||||||
|
continue
|
||||||
|
seen.add(key)
|
||||||
|
out.append(r)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
def parse_epub(path: Path, *, year: int | None = None) -> CptEdition:
|
def parse_epub(path: Path, *, year: int | None = None) -> CptEdition:
|
||||||
"""Parse one CPT Professional edition EPUB into a ``CptEdition``.
|
"""Parse one CPT Professional edition EPUB into a ``CptEdition``.
|
||||||
|
|
||||||
@@ -779,7 +877,22 @@ def parse_epub(path: Path, *, year: int | None = None) -> CptEdition:
|
|||||||
book-wide ``sec_id -> (lo, hi)`` map — a chapter's TOC can point
|
book-wide ``sec_id -> (lo, hi)`` map — a chapter's TOC can point
|
||||||
into a *different* chapter's body file, so ranges are matched by
|
into a *different* chapter's body file, so ranges are matched by
|
||||||
id across the whole book, not per source file — then parses every
|
id across the whole book, not per source file — then parses every
|
||||||
chapter body file and every appendix file (D/E/F/G/K/M/N/P/T)."""
|
chapter body file (in true chapter-number order, per
|
||||||
|
``_chapter_groups`` — the epub's own zip member order is *not* book
|
||||||
|
order in every edition) and every appendix file (D/E/F/G/K/M/N/P/T).
|
||||||
|
|
||||||
|
Accepted C9/C10 outcome, documented rather than "fixed" further: a
|
||||||
|
code that's genuinely cross-referenced at two different numeric
|
||||||
|
homes in the book resolves to whichever entry carries the TOC's own
|
||||||
|
declared numeric range, even when that isn't the chapter a reader
|
||||||
|
would expect. Example: 99100-99140 ("Qualifying Circumstances")
|
||||||
|
print both under Anesthesia's own guideline text *and* under
|
||||||
|
Medicine's "Qualifying Circumstances for Anesthesia" heading — only
|
||||||
|
the Medicine heading carries a TOC-declared range (99100-99140), so
|
||||||
|
it wins as canonical and the Anesthesia Guidelines copy becomes the
|
||||||
|
``CptAlternate``. This is the book's own numbering, not a parser
|
||||||
|
quirk — see ``TestReal2024.test_c10_99100_resolves_to_the_ranged_medicine_listing``
|
||||||
|
in tests/pfs/test_cpt_epub.py."""
|
||||||
path = Path(path)
|
path = Path(path)
|
||||||
if year is None:
|
if year is None:
|
||||||
year = _infer_year(path)
|
year = _infer_year(path)
|
||||||
@@ -801,6 +914,13 @@ def parse_epub(path: Path, *, year: int | None = None) -> CptEdition:
|
|||||||
codes: list[CptCode] = []
|
codes: list[CptCode] = []
|
||||||
instructions: list[CptInstruction] = []
|
instructions: list[CptInstruction] = []
|
||||||
references: list[CptReference] = []
|
references: list[CptReference] = []
|
||||||
|
# C10: most chapter-groups carry no div.ch-title of their own (a
|
||||||
|
# chapter split across many numbered files for size) — thread
|
||||||
|
# the active chapter title from one group's call into the
|
||||||
|
# next's so it keeps parenting headings until a real ch-title
|
||||||
|
# replaces it. Calls _group directly (not parse_xhtml, whose
|
||||||
|
# public 4-tuple return drops this) to carry it.
|
||||||
|
chapter_title = ""
|
||||||
for group in _chapter_groups(names):
|
for group in _chapter_groups(names):
|
||||||
bodies = [
|
bodies = [
|
||||||
_body_inner(zf.read(name).decode("utf-8", errors="replace"))
|
_body_inner(zf.read(name).decode("utf-8", errors="replace"))
|
||||||
@@ -808,7 +928,12 @@ def parse_epub(path: Path, *, year: int | None = None) -> CptEdition:
|
|||||||
]
|
]
|
||||||
combined = "<html><body>" + "\n".join(bodies) + "</body></html>"
|
combined = "<html><body>" + "\n".join(bodies) + "</body></html>"
|
||||||
source = group[0].rsplit("/", 1)[-1]
|
source = group[0].rsplit("/", 1)[-1]
|
||||||
s, c, i, r = parse_xhtml(combined, year=year, source=source)
|
tok = _Tokenizer()
|
||||||
|
tok.feed(combined)
|
||||||
|
tok.close()
|
||||||
|
s, c, i, r, chapter_title = _group(
|
||||||
|
tok.events, year=year, source=source, chapter_title=chapter_title
|
||||||
|
)
|
||||||
sections.extend(s)
|
sections.extend(s)
|
||||||
codes.extend(c)
|
codes.extend(c)
|
||||||
instructions.extend(i)
|
instructions.extend(i)
|
||||||
@@ -840,9 +965,14 @@ def parse_epub(path: Path, *, year: int | None = None) -> CptEdition:
|
|||||||
elif letter in _LIST_APPENDICES:
|
elif letter in _LIST_APPENDICES:
|
||||||
lists.extend(_parse_appendix_list(text, letter))
|
lists.extend(_parse_appendix_list(text, letter))
|
||||||
|
|
||||||
# C9: resolve any code that printed more than once (real descriptor
|
# C9/C10: resolve any code that printed more than once (real
|
||||||
# both times — never the C8 placeholder, already dropped) down to
|
# descriptor both times — never the C8 placeholder, already dropped)
|
||||||
# one canonical CptCode, keeping the rest as CptAlternate rows.
|
# down to one canonical CptCode, keeping the rest as CptAlternate
|
||||||
|
# rows. Dedupe (code, kind, text) first (minor #4) so an exact-repeat
|
||||||
|
# instruction/reference from a losing entry doesn't survive as a
|
||||||
|
# spurious duplicate row alongside the real entry's own copy.
|
||||||
|
instructions = _dedupe_instructions(instructions)
|
||||||
|
references = _dedupe_references(references)
|
||||||
canonical_codes, alternates = _resolve_alternates(fixed_sections, codes, references)
|
canonical_codes, alternates = _resolve_alternates(fixed_sections, codes, references)
|
||||||
|
|
||||||
return CptEdition(
|
return CptEdition(
|
||||||
|
|||||||
@@ -337,7 +337,7 @@ class TestReferences:
|
|||||||
assert refs[0].years == (2022,)
|
assert refs[0].years == (2022,)
|
||||||
|
|
||||||
|
|
||||||
# --- C9: guideline-reprint / cross-reference duplicates ------------------
|
# --- C9/C10: guideline-reprint / cross-reference duplicates --------------
|
||||||
#
|
#
|
||||||
# Unlike C8's resequenced placeholder (a bare pointer row with no real
|
# Unlike C8's resequenced placeholder (a bare pointer row with no real
|
||||||
# descriptor, dropped entirely by the parser), C9 handles a code that
|
# descriptor, dropped entirely by the parser), C9 handles a code that
|
||||||
@@ -347,19 +347,33 @@ class TestReferences:
|
|||||||
# the ``parse_epub`` (edition) level, since the resolution runs after
|
# the ``parse_epub`` (edition) level, since the resolution runs after
|
||||||
# every chapter has been parsed — so these tests build a tiny synthetic
|
# every chapter has been parsed — so these tests build a tiny synthetic
|
||||||
# EPUB rather than using ``cpt_sample.xhtml``/``parse_xhtml``.
|
# EPUB rather than using ``cpt_sample.xhtml``/``parse_xhtml``.
|
||||||
|
#
|
||||||
|
# C10 corrected a structural assumption: the fixture below now uses the
|
||||||
|
# book's own ``div.ch-title`` chapter banners (one per synthetic chapter
|
||||||
|
# file) rather than hand-nested ``h1``/``h2``, so it exercises the real
|
||||||
|
# mechanism — a chapter banner parents every heading in its own chapter
|
||||||
|
# and every later chapter that carries no banner of its own — the same
|
||||||
|
# mechanism the real book's "Surgery Guidelines" (Chapter05) / "Surgery"
|
||||||
|
# (Chapter06) / "Other Procedures" (Chapter07, no ch-title of its own)
|
||||||
|
# split relies on.
|
||||||
|
|
||||||
|
|
||||||
def _build_epub(path: Path, chapter_body: str) -> None:
|
def _build_epub(path: Path, chapters: dict[str, str]) -> None:
|
||||||
|
"""``chapters``: ``{"Chapter01.xhtml": body, "Chapter02.xhtml": body,
|
||||||
|
…}`` — each becomes its own ``_chapter_groups`` group (a distinct
|
||||||
|
``parse_epub`` chapter-group call), so ``chapter_title`` threading
|
||||||
|
across groups can be exercised."""
|
||||||
with zipfile.ZipFile(path, "w") as zf:
|
with zipfile.ZipFile(path, "w") as zf:
|
||||||
zf.writestr("mimetype", "application/epub+zip")
|
zf.writestr("mimetype", "application/epub+zip")
|
||||||
zf.writestr("OPS/Chapter01.xhtml", f"<html><body>{chapter_body}</body></html>")
|
for name, body in chapters.items():
|
||||||
|
zf.writestr(f"OPS/{name}", f"<html><body>{body}</body></html>")
|
||||||
|
|
||||||
|
|
||||||
_C9_CHAPTER = """
|
# Chapter01: the "Surgery Guidelines" banner — a guideline table
|
||||||
<div class="h1-toc"><a href="body.xhtml#sec_r"><span class="green">Sample Procedures* (54321-54329)</span></a></div>
|
# reprinting codes with no TOC range of their own.
|
||||||
|
_C9_CH1 = """
|
||||||
<div class="h1" id="sec_g1">Surgery Guidelines</div>
|
<div class="ch-title">Surgery Guidelines</div>
|
||||||
<div class="h2" id="sec_g2">Unlisted Service or Procedure</div>
|
<div class="h1" id="sec_g2">Unlisted Service or Procedure</div>
|
||||||
<div class="noindent">The “Unlisted Procedures” and accompanying codes are as follows:</div>
|
<div class="noindent">The “Unlisted Procedures” and accompanying codes are as follows:</div>
|
||||||
<table class="table1"><tbody>
|
<table class="table1"><tbody>
|
||||||
<tr>
|
<tr>
|
||||||
@@ -371,7 +385,12 @@ _C9_CHAPTER = """
|
|||||||
<td class="td"><div class="table-para1">Unlisted procedure, only ever printed in the guidelines</div></td>
|
<td class="td"><div class="table-para1">Unlisted procedure, only ever printed in the guidelines</div></td>
|
||||||
</tr>
|
</tr>
|
||||||
</tbody></table>
|
</tbody></table>
|
||||||
|
"""
|
||||||
|
|
||||||
|
# Chapter02: a *different* banner, "Surgery" — the real, ranged home.
|
||||||
|
_C9_CH2 = """
|
||||||
|
<div class="h1-toc"><a href="body.xhtml#sec_r"><span class="green">Sample Procedures* (54321-54329)</span></a></div>
|
||||||
|
<div class="ch-title">Surgery</div>
|
||||||
<div class="h1" id="sec_r">Sample Procedures</div>
|
<div class="h1" id="sec_r">Sample Procedures</div>
|
||||||
<div class="noindent">Guideline text for the real, ranged section.</div>
|
<div class="noindent">Guideline text for the real, ranged section.</div>
|
||||||
<table class="table1"><tbody>
|
<table class="table1"><tbody>
|
||||||
@@ -384,14 +403,57 @@ _C9_CHAPTER = """
|
|||||||
</tbody></table>
|
</tbody></table>
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
# Chapter03: carries *no* ch-title of its own at all — proves the
|
||||||
|
# "Surgery" banner from Chapter02 keeps parenting headings until a new
|
||||||
|
# banner replaces it (exactly 2019's Surgery body, Chapter07-16, which
|
||||||
|
# has no ch-title of its own and inherits Chapter06's "Surgery").
|
||||||
|
_C9_CH3 = """
|
||||||
|
<div class="h1" id="sec_o">Other Procedures</div>
|
||||||
|
<table class="table1"><tbody>
|
||||||
|
<tr>
|
||||||
|
<td class="td-w1" id="code_54323"><div class="table-para"><b>54323</b></div></td>
|
||||||
|
<td class="td"><div class="table-para1">Unlisted procedure, no ch-title of its own</div></td>
|
||||||
|
</tr>
|
||||||
|
</tbody></table>
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="module")
|
@pytest.fixture(scope="module")
|
||||||
def c9_edition(tmp_path_factory):
|
def c9_edition(tmp_path_factory):
|
||||||
path = tmp_path_factory.mktemp("c9") / "sample.epub"
|
path = tmp_path_factory.mktemp("c9") / "sample.epub"
|
||||||
_build_epub(path, _C9_CHAPTER)
|
_build_epub(
|
||||||
|
path,
|
||||||
|
{
|
||||||
|
"Chapter01.xhtml": _C9_CH1,
|
||||||
|
"Chapter02.xhtml": _C9_CH2,
|
||||||
|
"Chapter03.xhtml": _C9_CH3,
|
||||||
|
},
|
||||||
|
)
|
||||||
return parse_epub(path, year=2024)
|
return parse_epub(path, year=2024)
|
||||||
|
|
||||||
|
|
||||||
|
class TestChapterTitleParenting:
|
||||||
|
"""C10: div.ch-title is the level-0 parent of every heading in its
|
||||||
|
own chapter-group and every later one that carries no banner of its
|
||||||
|
own, replaced only by the next ch-title."""
|
||||||
|
|
||||||
|
def test_heading_nests_under_its_own_chapter_banner(self, c9_edition):
|
||||||
|
sec = next(s for s in c9_edition.sections if s.sec_id == "sec_g2")
|
||||||
|
assert sec.path == ("Surgery Guidelines", "Unlisted Service or Procedure")
|
||||||
|
|
||||||
|
def test_a_new_banner_replaces_the_previous_one(self, c9_edition):
|
||||||
|
sec = next(s for s in c9_edition.sections if s.sec_id == "sec_r")
|
||||||
|
assert sec.path == ("Surgery", "Sample Procedures")
|
||||||
|
|
||||||
|
def test_chapter_with_no_banner_of_its_own_inherits_the_previous_one(
|
||||||
|
self, c9_edition
|
||||||
|
):
|
||||||
|
# Chapter03 has no div.ch-title at all — "Surgery" (Chapter02's
|
||||||
|
# banner) must still parent its heading.
|
||||||
|
sec = next(s for s in c9_edition.sections if s.sec_id == "sec_o")
|
||||||
|
assert sec.path == ("Surgery", "Other Procedures")
|
||||||
|
|
||||||
|
|
||||||
class TestAlternates:
|
class TestAlternates:
|
||||||
def test_ranged_entry_wins_as_canonical(self, c9_edition):
|
def test_ranged_entry_wins_as_canonical(self, c9_edition):
|
||||||
matches = [c for c in c9_edition.codes if c.code == "54321"]
|
matches = [c for c in c9_edition.codes if c.code == "54321"]
|
||||||
@@ -505,6 +567,65 @@ class TestReal2024:
|
|||||||
assert len(edition.alternates) > 100
|
assert len(edition.alternates) > 100
|
||||||
print(f"2024 alternates: {len(edition.alternates)}")
|
print(f"2024 alternates: {len(edition.alternates)}")
|
||||||
|
|
||||||
|
def test_c10_21089_resolves_under_surgery(self, edition):
|
||||||
|
# C10 regression (structure only, no descriptor text): 21089 is
|
||||||
|
# printed twice — a bare "Unlisted procedure" row inside the
|
||||||
|
# Surgery Guidelines chapter's "Unlisted Service or Procedure"
|
||||||
|
# convenience table, and its real entry under its actual
|
||||||
|
# subsection. The chapter-title-parents-headings fix must land
|
||||||
|
# the canonical row under "Surgery > …", and the guideline copy
|
||||||
|
# in cpt_code_alt with reason "guidelines-reprint".
|
||||||
|
by_code = {c.code: c for c in edition.codes}
|
||||||
|
sec_by_id = {s.sec_id: s for s in edition.sections}
|
||||||
|
canonical = by_code["21089"]
|
||||||
|
sec = sec_by_id[canonical.sec_id]
|
||||||
|
assert sec.path[0] == "Surgery"
|
||||||
|
assert sec.path[-1] != "Unlisted Service or Procedure"
|
||||||
|
|
||||||
|
alts = [a for a in edition.alternates if a.code == "21089"]
|
||||||
|
assert len(alts) == 1
|
||||||
|
alt_sec = sec_by_id[alts[0].sec_id]
|
||||||
|
assert alt_sec.path == ("Surgery Guidelines", "Unlisted Service or Procedure")
|
||||||
|
assert alts[0].reason == "guidelines-reprint"
|
||||||
|
|
||||||
|
def test_c10_99100_resolves_to_the_ranged_medicine_listing(self, edition):
|
||||||
|
# C10 regression / documented accepted outcome (see parse_epub's
|
||||||
|
# docstring): 99100 (a Qualifying Circumstances for Anesthesia
|
||||||
|
# add-on) prints under Anesthesia's own "Qualifying
|
||||||
|
# Circumstances" guideline *and* under Medicine's "Qualifying
|
||||||
|
# Circumstances for Anesthesia" heading, which carries the
|
||||||
|
# book's own TOC-declared numeric range (99100-99140) — C9's
|
||||||
|
# scoring picks the ranged entry as canonical regardless of
|
||||||
|
# which chapter "feels" more natural, and that's intentional.
|
||||||
|
by_code = {c.code: c for c in edition.codes}
|
||||||
|
sec_by_id = {s.sec_id: s for s in edition.sections}
|
||||||
|
canonical = by_code["99100"]
|
||||||
|
sec = sec_by_id[canonical.sec_id]
|
||||||
|
assert sec.path == ("Medicine", "Qualifying Circumstances for Anesthesia")
|
||||||
|
|
||||||
|
alts = [a for a in edition.alternates if a.code == "99100"]
|
||||||
|
assert len(alts) == 1
|
||||||
|
alt_sec = sec_by_id[alts[0].sec_id]
|
||||||
|
assert alt_sec.path == ("Anesthesia Guidelines", "Qualifying Circumstances")
|
||||||
|
assert alts[0].reason == "guidelines-reprint"
|
||||||
|
|
||||||
|
def test_c10_99490_path_key(self, edition):
|
||||||
|
by_code = {c.code: c for c in edition.codes}
|
||||||
|
sec_by_id = {s.sec_id: s for s in edition.sections}
|
||||||
|
sec = sec_by_id[by_code["99490"].sec_id]
|
||||||
|
assert sec.path == (
|
||||||
|
"Evaluation and Management",
|
||||||
|
"Care Management Services",
|
||||||
|
"Chronic Care Management Services",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_c10_alternate_reasons_include_guidelines_reprint(self, edition):
|
||||||
|
from collections import Counter
|
||||||
|
|
||||||
|
reasons = Counter(a.reason for a in edition.alternates)
|
||||||
|
assert reasons["guidelines-reprint"] > 0
|
||||||
|
print(f"2024 cpt_code_alt reasons: {dict(reasons)}")
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.skipif(not EPUB_2022.exists(), reason="real 2022 CPT EPUB not on disk")
|
@pytest.mark.skipif(not EPUB_2022.exists(), reason="real 2022 CPT EPUB not on disk")
|
||||||
def test_real_2022_parses():
|
def test_real_2022_parses():
|
||||||
|
|||||||
Reference in New Issue
Block a user