feat: pydoc pincite — structured Zotero citations in docstrings #293

Merged

kert merged 2 commits from feat/pydoc-pincite into main 2026-03-27 04:12:59 +00:00

Conversation 0 Commits 2 Files Changed 7 +1197

kert commented 2026-03-27 03:48:30 +00:00

Owner

Copy Link

Summary

Centerpiece feature: machine-parseable :pincite: directives in function docstrings that link to specific pages, sections, or paragraphs of Zotero bibliography items.

@nw.narwhalify
def pharmacy_claims(core__medical_claim):
    """Filter Part D claims to valid NDCs.

    :pincite:`JX46GQ9L p.14` — NDC validation rules
    :pincite:`9ASETLJ4 §3.2` — exclusion criteria
    """

Architecture

4 layers in src/bib/pincite.py:

• Parse: regex extraction + AST discovery of @nw.narwhalify functions
• Store: pincites table in bib.sqlite with upsert/query/tag sync
• Graph: CitationGraph with mermaid + networkx export
• Inject: non-destructive docstring modification via AST

New/modified files

• src/bib/pincite.py — core module (480 lines)
• src/bib/schema.sql — pincites table
• src/bib/tag.py — Tag.pin() factory
• src/bib/store.py — pincite CRUD methods
• tests/bib/test_pincite.py — 39 tests

refs #246

## Summary Centerpiece feature: machine-parseable `:pincite:` directives in function docstrings that link to specific pages, sections, or paragraphs of Zotero bibliography items. ```python @nw.narwhalify def pharmacy_claims(core__medical_claim): """Filter Part D claims to valid NDCs. :pincite:`JX46GQ9L p.14` — NDC validation rules :pincite:`9ASETLJ4 §3.2` — exclusion criteria """ ``` ## Architecture 4 layers in `src/bib/pincite.py`: - **Parse**: regex extraction + AST discovery of @nw.narwhalify functions - **Store**: pincites table in bib.sqlite with upsert/query/tag sync - **Graph**: CitationGraph with mermaid + networkx export - **Inject**: non-destructive docstring modification via AST ## New/modified files - `src/bib/pincite.py` — core module (480 lines) - `src/bib/schema.sql` — pincites table - `src/bib/tag.py` — `Tag.pin()` factory - `src/bib/store.py` — pincite CRUD methods - `tests/bib/test_pincite.py` — 39 tests refs #246

kert added 1 commit 2026-03-27 03:48:30 +00:00

feat: pydoc pincite — structured Zotero citations in docstrings ... 5be2749d8f

Centerpiece feature: machine-parseable :pincite: directives in function
docstrings that link to specific pages, sections, or paragraphs of
bibliography items, enabling a knowledge graph from regulatory source
to function to table to column.

Format: :pincite:`JX46GQ9L p.14` — NDC validation rules

New module src/bib/pincite.py with 4 layers:
- Parse: regex extraction of :pincite: directives, AST-based discovery
  of @nw.narwhalify functions, locator classification (page/section/cfr/
  fr/chapter/paragraph)
- Store: pincites table in bib.sqlite with upsert, query, tag sync
  (pin: and fn: namespaces), foreign key validation
- Graph: CitationGraph with nodes (functions, items), edges (pincite
  rows), functions_citing/items_cited_by queries, mermaid and
  networkx export
- Inject: format_pincite_block for References sections,
  inject_pincites_into_source for AST-located non-destructive
  docstring modification

Schema: new pincites table with unique index on (fn_path, item_key,
locator), FK cascade to items.

Tag: new Tag.pin(key, locator) factory -> pin:KEY/locator

Store: upsert_pincite, list_pincites, delete_pincites on Store class.

39 tests covering parse, classify, store, graph, inject, and check.

kert added 1 commit 2026-03-27 04:02:42 +00:00

feat: fetch Federal Register PDF + TXT for all bib rules ... 8a0992576c

Script downloads the full text (PDF from govinfo.gov, TXT from
federalregister.gov) for all 47 PFS rules in bib.sqlite and attaches
them to their items via store.attach_file().

Constructs URLs from the FR document number embedded in each rule's
federalregister.gov URL.  Falls back to FR API metadata lookup when
the URL is ambiguous.  Skips items that already have attachments.

Usage:
  uv run python dev/scripts/fetch_fr_attachments.py --dry-run
  uv run python dev/scripts/fetch_fr_attachments.py
  uv run python dev/scripts/fetch_fr_attachments.py --key ZJ3YCYYF

refs #246

kert merged commit 037eae10bc into main 2026-03-27 04:12:59 +00:00

kert referenced this issue from a commit 2026-03-27 04:12:59 +00:00

Merge pull request 'feat: pydoc pincite — structured Zotero citations in docstrings' (#293) from feat/pydoc-pincite into main

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

api

REST API (FastAPI)

auth

Authentication and authorization

bib

Bibliography, Zotero, citations

ci

CI/CD pipelines and Woodpecker plugins

cli

CLI commands and UX

cloud

Cloud provider abstraction (AWS, GCP, Azure, self-hosted)

config

Configuration and wiring

data

Data loading, CCLF, BCDA, seeds

databricks

Databricks jobs and deployment

docs

Documentation and generated references

image:api

image:docs

image:mc

image:notebooks

image:zotero

infra

lakehouse

Nessie, Polaris, Trino, Iceberg

llm

llm module — local RAG over comments + Zotero corpus

notebooks

marimo notebook errors (auto-filed by nb-watcher/nb-integration)

packaging

Python packaging, optional deps, build system

pfs

pipeline

Pipeline execution and validation

pipeline:ci

pipeline:deploy

pipeline:harden

pipeline:infra-ci

pipeline:rebuild-all

pipeline:release

qpp

quality

Testing, coverage, error handling

step:build

step:deploy

step:hadolint

step:lint

step:publish

step:scan

step:test

step:validate

styles

Charts, figures, visual theming

tree

Project directory structure and file organization

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

kert renovate-bot

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: homelab/stack#293