stack/docs/superpowers/specs/2026-08-17-fr-jump-links-design.md

# FR Jump Links — Page/Paragraph Deep Links for Every FR Citation (P40)

Date: 2026-08-17. Milestone: P40. Requested as: a utility "able to grab
and place the paragraph link jump links from the web version of the
federal register to any of the pages or paragraphs in the rule,"
context-aware for ANY FR citation instance, with page ↔ paragraph
transmutation in both directions, Zotero child links, and `:pincite:`
integration. Design presented in-session and approved ("yes that does
it").

## Verified ground truth (2026-08-17, CMS-1848-P / doc 2026-14327)

- The FR API's `body_html_url` serves the full rule body as one HTML
  file (4.2 MB for this 716-page rule) containing, in document order:
  5,079 paragraph anchors `id="p-N"`, 716 page anchors
  `id="page-NNNNN"`, and — decisively — an explicit
  `data-page="NNNNN"` attribute **on each paragraph element**, so
  page association needs no positional inference.
- Deep links `#p-N` and `#page-NNNNN` work against the canonical
  `html_url` reader page.
- The API also returns `start_page`/`end_page` per document; bib rule
  items already carry `fr_volume`, `fr_page`, `document_number` in
  `extra_json`, and ~30 `item_type='rule'` items have
  federalregister.gov URLs.
- Sample verified: page 44218's paragraphs (p-3599…) are the
  §414.1425 QP passage the CY2027 notebook cites as "91 FR 44218".

## Decisions (user-selected)

- **Consumers: all.** One context-aware resolver used by notebooks,
  the `:pincite:` graph/docs, Zotero (as child link attachments), and
  a CLI.
- **Addressing: all modes.** Quoted text, page + ordinal ("91 FR
  44218 ¶3"), raw anchor id (`p-3600`), and plain page cites; page ↔
  paragraph transmutation both directions.
- **Persistence: in bib.** Anchor maps stored in `bib.sqlite`;
  offline, reproducible; refresh on demand.
- **Scope: all bib rules with FR URLs**, backfilled.

## Architecture

**Home: new module `src/bib/frlink.py`** — grabber, anchor store
access, resolver, transmutations, markdown helper. Three thin hooks
elsewhere: `bib/pincite.py` (FR locator grammar + URL resolution),
`bib/sync.py` (child link attachments), `src/cli/bib.py` (two
subcommands). Rationale: `bib` already owns the FR API client
(`federalregister.py`), the citation formatter, the pincite pipeline,
and the Zotero sync; `frlink` composes them. (Alternatives considered:
extending `federalregister.py` — blurs a stateless API client with a
stateful store; a new top-level package — overkill for citation
infrastructure.)

### Data model (`bib/schema.sql`, auto-applied via CREATE TABLE IF NOT
EXISTS on Store init)

- `fr_anchors(id, item_key → items.key, p_id INTEGER, page INTEGER,
  ordinal INTEGER, text TEXT)` — one row per paragraph anchor;
  `ordinal` is 1-based position on its printed page; `text` is the
  entity-decoded, tag-stripped paragraph text (for quote search).
  Unique on `(item_key, p_id)`; indexed on `(item_key, page)`.
- `fr_anchor_docs(item_key PRIMARY KEY, document_number, html_url,
  body_html_url, start_page INTEGER, end_page INTEGER, fr_volume
  INTEGER, fetched_at, sha256, n_paragraphs, n_pages)` — one row per
  grabbed rule; sha256 of the fetched body makes drift detectable.
- `fr_links(id, item_key → items.key, p_id INTEGER NULL, page
  INTEGER, label TEXT, url TEXT, created_at)` — **placed** links (the
  curated subset that gets carried into Zotero), unique on
  `(item_key, url)`.

### Grabber

`grab(store, item_key, *, force=False) -> dict` — resolves the item's
`document_number` (from `extra_json`, falling back to parsing the FR
URL), fetches `body_html_url` (one request; via the document API when
the body URL must be discovered), parses anchors with a pure function
`parse_anchors(html) -> list[Anchor]` (regex walk over `id="p-N"` /
`data-page` / `id="page-N"`, tag-strip + `html.unescape` for text),
and replaces that item's `fr_anchors` partition + `fr_anchor_docs`
row. Backfill = iterate `item_type='rule'` items whose URL matches
federalregister.gov; skip already-grabbed unless `--force`.

### Resolver (context-aware core)

`resolve(ref, *, store, item_key=None) -> JumpLink` where `JumpLink`
carries `url, item_key, page, p_id, ordinal, snippet`. Accepted
`ref` forms:

1. `"91 FR 44218"` → page link `{html_url}#page-44218`. The rule is
   found by volume + page-range containment against `fr_anchor_docs`
   (no `item_key` needed); ambiguity (overlapping ranges) raises with
   candidates listed.
2. `"91 FR 44218 ¶3"` (also `para 3` / `p.3` spellings) → 3rd
   paragraph anchor on that page → `#p-N`.
3. `"p-3600"` (requires `item_key` or an unambiguous single-rule
   context) → validated `#p-3600`.
4. A quoted passage ≥ 15 chars (anything not matching 1–3) →
   normalized substring search over `fr_anchors.text` (whitespace
   collapsed, entities already decoded); exactly-one match required,
   zero/many raise with diagnostics.

Transmutations: `page_of(item_key, p_id) -> int` and
`paragraphs_of("91 FR 44218") -> list[Anchor]` (with ordinals +
snippets) — page cites upgrade to paragraph cites and vice versa.
`md_link(ref, *, text=None) -> str` renders `[91 FR 44218 ¶3](url)`
for notebooks.

### Placement

`place(store, ref, *, label="") -> JumpLink` — resolves, then upserts
an `fr_links` row. `bib/sync.py` gains `_sync_fr_links` beside
`_sync_attachments`: each `fr_links` row becomes a Zotero **child
link attachment** (`linkMode=3` linked_url, `contentType text/html`,
itemData `url` + `title` = label), idempotent by existing child URL.
Zotero carries the links actually cited — never all ~5k anchors per
rule (explicit scope decision from the design review).

`:pincite:` integration: the locator grammar learns FR forms
(`91 FR 44218`, `91 FR 44218 ¶3`, `p-3600`); `Pincite` gains a
`jump_url(store)` accessor used by the citation-graph/docs rendering;
unresolvable FR locators degrade to no-URL (never an exception in
graph rendering).

### CLI

- `stack bib fr-grab [--all | --key KEY] [--force]` — grab/backfill.
- `stack bib fr-jump REF [--key KEY] [--md] [--place [--label L]]` —
  resolve, print URL (or markdown), optionally place.

### First consumer

The CY2027 NPRM notebook's FR pincites (at minimum the 91 FR 44218
QP-alignment cite and the CAA-2026 revival cites 44218-44219, 44286)
become live jump links via `frlink.md_link`, gated on the
`nb_integration.py` single-notebook run. Placed links for those cites
sync to Zotero as children of `5ITGVDJV`.

## Testing

- `parse_anchors`: synthetic body-HTML fixture (pages, paragraphs,
  data-page, entities like `§ `, footnote-free edge
  cases); ordinal assignment; idempotent re-grab (partition replace).
- Resolver: every ref form; ambiguity errors (multi-rule page range,
  multi-hit quote); transmutations both ways; `md_link` output.
- Placement: tmp bib + tmp zotero db (existing test_sync fixtures) —
  linkMode=3 child created once, idempotent on re-sync.
- Live smoke: parse the already-downloaded 4.2 MB CY2027 body HTML —
  5,079 paragraphs / 716 pages expected; `resolve("91 FR 44218 ¶2")`
  lands inside the §414.1425 passage.

## Out of scope

- Anchors for non-rule item types (manuals, comments).
- Bulk-placing every anchor into Zotero.
- eCFR/edition permalinks (govinfo PDFs) — FR web version only.
- Auto-rewriting existing docstring pincites; the grammar accepts FR
  locators, adoption is incremental.

## P40 build outcomes (2026-08-17)

Issues #634–#638, milestone P40 (6 commits, 5ea78cf..d7adf59, direct on
main). All TDD; the pre-commit hook ran the bib-targeted suite green at
each step (4,200+ tests).

- **#634 (5ea78cf)**: `bib/frlink.py` grabber + `fr_anchors` /
  `fr_anchor_docs` / `fr_links` tables + `fr-grab` CLI. **Ground-truth
  correction**: the CY2027 body carries **4,856** paragraph anchors
  (4,194 `<p>` + 662 `<li>`), not the 5,079 stated above — that figure
  was a `grep -c` line-count artifact that also matched `id="page-`
  lines. The parser handles both element kinds (close-tag aware) and
  captures 100% of real anchors; 431 of the 716 printed pages carry
  paragraph anchors (the rest are tables/addenda), and the resolver
  reports that distinction explicitly on ¶-of-page lookups.
- **#635 (01e5d95)**: `resolve()` — page / page+¶-ordinal / raw
  anchor / quote refs, volume+page-range rule inference, ambiguity
  diagnostics; `page_of`/`paragraphs_of` transmutations; `md_link`.
  Live: `"91 FR 44218 ¶2"` → `#p-3601` (the Participation-List ¶).
- **#636 (eda1887)**: locator types `fr_para`/`fr_anchor` added
  (bare cites keep the backward-compatible `fr`); `Pincite.jump_url`
  (degrades to `""`, never raises); `CitationEdge.jump_url` populated
  by `build_citation_graph`.
- **#637 (e5e5eb8)**: `place()` → `fr_links`; `_sync_fr_links` in the
  Zotero push (linkMode=3 linked_url children, url+title itemData,
  idempotent by child URL); `fr-jump` CLI with `--md`/`--place`.
- **#638 (d7adf59)**: backfill grabbed **78 rules / 192,011 anchors,
  zero errors**; CY2027 notebook cites are live links via a
  graceful-fallback `fr_md()` helper (gate `pass=1`, resolution proven
  in-container); three placed links synced to Zotero as child link
  attachments on `5ITGVDJV` (incl. the §414.1425 QP-alignment
  paragraph pinned by quote → `#p-3600`).
- Deviation: none of substance; the plan's ~30-rule estimate
  undercounted (bib holds 78 FR-linked rules; all grabbed).