stack/docs/superpowers/specs/2026-08-17-fr-jump-links-design.md

FR Jump Links — Page/Paragraph Deep Links for Every FR Citation (P40)

Date: 2026-08-17. Milestone: P40. Requested as: a utility "able to grab and place the paragraph link jump links from the web version of the federal register to any of the pages or paragraphs in the rule," context-aware for ANY FR citation instance, with page ↔ paragraph transmutation in both directions, Zotero child links, and :pincite: integration. Design presented in-session and approved ("yes that does it").

Verified ground truth (2026-08-17, CMS-1848-P / doc 2026-14327)

• The FR API's body_html_url serves the full rule body as one HTML file (4.2 MB for this 716-page rule) containing, in document order: 5,079 paragraph anchors id="p-N", 716 page anchors id="page-NNNNN", and — decisively — an explicit data-page="NNNNN" attribute on each paragraph element, so page association needs no positional inference.
• Deep links #p-N and #page-NNNNN work against the canonical html_url reader page.
• The API also returns start_page/end_page per document; bib rule items already carry fr_volume, fr_page, document_number in extra_json, and ~30 item_type='rule' items have federalregister.gov URLs.
• Sample verified: page 44218's paragraphs (p-3599…) are the §414.1425 QP passage the CY2027 notebook cites as "91 FR 44218".

Decisions (user-selected)

• Consumers: all. One context-aware resolver used by notebooks, the :pincite: graph/docs, Zotero (as child link attachments), and a CLI.
• Addressing: all modes. Quoted text, page + ordinal ("91 FR 44218 ¶3"), raw anchor id (p-3600), and plain page cites; page ↔ paragraph transmutation both directions.
• Persistence: in bib. Anchor maps stored in bib.sqlite; offline, reproducible; refresh on demand.
• Scope: all bib rules with FR URLs, backfilled.

Architecture

Home: new module src/bib/frlink.py — grabber, anchor store access, resolver, transmutations, markdown helper. Three thin hooks elsewhere: bib/pincite.py (FR locator grammar + URL resolution), bib/sync.py (child link attachments), src/cli/bib.py (two subcommands). Rationale: bib already owns the FR API client (federalregister.py), the citation formatter, the pincite pipeline, and the Zotero sync; frlink composes them. (Alternatives considered: extending federalregister.py — blurs a stateless API client with a stateful store; a new top-level package — overkill for citation infrastructure.)

Data model (bib/schema.sql, auto-applied via CREATE TABLE IF NOT

EXISTS on Store init)

• fr_anchors(id, item_key → items.key, p_id INTEGER, page INTEGER, ordinal INTEGER, text TEXT) — one row per paragraph anchor; ordinal is 1-based position on its printed page; text is the entity-decoded, tag-stripped paragraph text (for quote search). Unique on (item_key, p_id); indexed on (item_key, page).
• fr_anchor_docs(item_key PRIMARY KEY, document_number, html_url, body_html_url, start_page INTEGER, end_page INTEGER, fr_volume INTEGER, fetched_at, sha256, n_paragraphs, n_pages) — one row per grabbed rule; sha256 of the fetched body makes drift detectable.
• fr_links(id, item_key → items.key, p_id INTEGER NULL, page INTEGER, label TEXT, url TEXT, created_at) — placed links (the curated subset that gets carried into Zotero), unique on (item_key, url).

Grabber

grab(store, item_key, *, force=False) -> dict — resolves the item's document_number (from extra_json, falling back to parsing the FR URL), fetches body_html_url (one request; via the document API when the body URL must be discovered), parses anchors with a pure function parse_anchors(html) -> list[Anchor] (regex walk over id="p-N" / data-page / id="page-N", tag-strip + html.unescape for text), and replaces that item's fr_anchors partition + fr_anchor_docs row. Backfill = iterate item_type='rule' items whose URL matches federalregister.gov; skip already-grabbed unless --force.

Resolver (context-aware core)

resolve(ref, *, store, item_key=None) -> JumpLink where JumpLink carries url, item_key, page, p_id, ordinal, snippet. Accepted ref forms:

1. "91 FR 44218" → page link {html_url}#page-44218. The rule is found by volume + page-range containment against fr_anchor_docs (no item_key needed); ambiguity (overlapping ranges) raises with candidates listed.
2. "91 FR 44218 ¶3" (also para 3 / p.3 spellings) → 3rd paragraph anchor on that page → #p-N.
3. "p-3600" (requires item_key or an unambiguous single-rule context) → validated #p-3600.
4. A quoted passage ≥ 15 chars (anything not matching 1–3) → normalized substring search over fr_anchors.text (whitespace collapsed, entities already decoded); exactly-one match required, zero/many raise with diagnostics.

Transmutations: page_of(item_key, p_id) -> int and paragraphs_of("91 FR 44218") -> list[Anchor] (with ordinals + snippets) — page cites upgrade to paragraph cites and vice versa. md_link(ref, *, text=None) -> str renders [91 FR 44218 ¶3](url) for notebooks.

Placement

place(store, ref, *, label="") -> JumpLink — resolves, then upserts an fr_links row. bib/sync.py gains _sync_fr_links beside _sync_attachments: each fr_links row becomes a Zotero child link attachment (linkMode=3 linked_url, contentType text/html, itemData url + title = label), idempotent by existing child URL. Zotero carries the links actually cited — never all ~5k anchors per rule (explicit scope decision from the design review).

:pincite: integration: the locator grammar learns FR forms (91 FR 44218, 91 FR 44218 ¶3, p-3600); Pincite gains a jump_url(store) accessor used by the citation-graph/docs rendering; unresolvable FR locators degrade to no-URL (never an exception in graph rendering).

CLI

• stack bib fr-grab [--all | --key KEY] [--force] — grab/backfill.
• stack bib fr-jump REF [--key KEY] [--md] [--place [--label L]] — resolve, print URL (or markdown), optionally place.

First consumer

The CY2027 NPRM notebook's FR pincites (at minimum the 91 FR 44218 QP-alignment cite and the CAA-2026 revival cites 44218-44219, 44286) become live jump links via frlink.md_link, gated on the nb_integration.py single-notebook run. Placed links for those cites sync to Zotero as children of 5ITGVDJV.

Testing

• parse_anchors: synthetic body-HTML fixture (pages, paragraphs, data-page, entities like § , footnote-free edge cases); ordinal assignment; idempotent re-grab (partition replace).
• Resolver: every ref form; ambiguity errors (multi-rule page range, multi-hit quote); transmutations both ways; md_link output.
• Placement: tmp bib + tmp zotero db (existing test_sync fixtures) — linkMode=3 child created once, idempotent on re-sync.
• Live smoke: parse the already-downloaded 4.2 MB CY2027 body HTML — 5,079 paragraphs / 716 pages expected; resolve("91 FR 44218 ¶2") lands inside the §414.1425 passage.

Out of scope

• Anchors for non-rule item types (manuals, comments).
• Bulk-placing every anchor into Zotero.
• eCFR/edition permalinks (govinfo PDFs) — FR web version only.
• Auto-rewriting existing docstring pincites; the grammar accepts FR locators, adoption is incremental.

P40 build outcomes (2026-08-17)

Issues #634–#638, milestone P40 (6 commits, 5ea78cf..d7adf59, direct on main). All TDD; the pre-commit hook ran the bib-targeted suite green at each step (4,200+ tests).

• #634 (5ea78cf): bib/frlink.py grabber + fr_anchors / fr_anchor_docs / fr_links tables + fr-grab CLI. Ground-truth correction: the CY2027 body carries 4,856 paragraph anchors (4,194 <p> + 662 <li>), not the 5,079 stated above — that figure was a grep -c line-count artifact that also matched id="page- lines. The parser handles both element kinds (close-tag aware) and captures 100% of real anchors; 431 of the 716 printed pages carry paragraph anchors (the rest are tables/addenda), and the resolver reports that distinction explicitly on ¶-of-page lookups.
• #635 (01e5d95): resolve() — page / page+¶-ordinal / raw anchor / quote refs, volume+page-range rule inference, ambiguity diagnostics; page_of/paragraphs_of transmutations; md_link. Live: "91 FR 44218 ¶2" → #p-3601 (the Participation-List ¶).
• #636 (eda1887): locator types fr_para/fr_anchor added (bare cites keep the backward-compatible fr); Pincite.jump_url (degrades to "", never raises); CitationEdge.jump_url populated by build_citation_graph.
• #637 (e5e5eb8): place() → fr_links; _sync_fr_links in the Zotero push (linkMode=3 linked_url children, url+title itemData, idempotent by child URL); fr-jump CLI with --md/--place.
• #638 (d7adf59): backfill grabbed 78 rules / 192,011 anchors, zero errors; CY2027 notebook cites are live links via a graceful-fallback fr_md() helper (gate pass=1, resolution proven in-container); three placed links synced to Zotero as child link attachments on 5ITGVDJV (incl. the §414.1425 QP-alignment paragraph pinned by quote → #p-3600).
• Deviation: none of substance; the plan's ~30-rule estimate undercounted (bib holds 78 FR-linked rules; all grabbed).