kert 1256ae1c04 feat: headless PDF retrieval — Unpaywall, PMC, Semantic Scholar, SciHub (refs #257) ...

Terminal-based PDF retrieval pipeline that does not require Zotero GUI:
- Phase 1: Unpaywall (legal, 100K/day, ~4% coverage for wound care lit)
- Phase 2: PMC ID Converter + OA PDF download (batch 200 DOIs)
- Phase 3: Semantic Scholar batch endpoint (500 DOIs/request)
- Phase 4: SciHub with configurable proxy (--proxy socks5://host:port)

All sources attach PDFs directly to Zotero items in the SQLite DB.

Current blocker: SciHub mirrors blocked from this network (DNS + timeout).
OA coverage is ~4% for this corpus. Need SOCKS proxy or VPN for SciHub.
Script is ready — just needs: SCIHUB_PROXY=socks5://host:port

2026-03-25 18:01:12 -04:00

..

backends

fix: skinny install uses --no-dev to avoid pulling marimo/dev deps

2026-03-25 02:11:09 -04:00

add_carrier_to_zotero.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

add_zipcode_to_zotero.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

bootstrap_certs.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

bootstrap_secrets.py

P18: eliminate remaining hardcoded values across the stack

2026-03-23 20:04:32 -04:00

build_product_susceptibility.py

data: product susceptibility scoring for kickback/fraud risk (refs #260)

2026-03-25 15:57:34 -04:00

build_skin_subs_analytics.py

data: ASP trajectories, market segmentation, utilization metrics (refs #234, #238, #239)

2026-03-25 14:53:34 -04:00

build_skin_subs_anomaly.py

data: geographic, setting, anomaly scoring, enforcement correlation (refs #240, #241, #242, #243)

2026-03-25 15:02:05 -04:00

build_skin_subs_evidence_base.py

data: evidence base builder + literature methods doc (refs #237, refs #244)

2026-03-25 11:33:32 -04:00

build_skin_subs_geo_setting.py

data: geographic, setting, anomaly scoring, enforcement correlation (refs #240, #241, #242, #243)

2026-03-25 15:02:05 -04:00

check_pdf_coverage.py

feat: scipdf plugin + PDF coverage tracker for skin-subs retrieval (refs #257)

2026-03-25 16:14:08 -04:00

collect_grey_lit_skin_subs.py

data: study characteristics, snowball citations, entity tags, expanded grey lit (refs #235, #236, #237, #244)

2026-03-25 11:56:49 -04:00

coverage_badge.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

enrich_entity_tags_and_load_duckdb.py

data: study characteristics, snowball citations, entity tags, expanded grey lit (refs #235, #236, #237, #244)

2026-03-25 11:56:49 -04:00

extract_study_characteristics.py

data: study characteristics, snowball citations, entity tags, expanded grey lit (refs #235, #236, #237, #244)

2026-03-25 11:56:49 -04:00

fetch_pdfs.py

feat: headless PDF retrieval — Unpaywall, PMC, Semantic Scholar, SciHub (refs #257)

2026-03-25 18:01:12 -04:00

find_databricks_workspace.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

fix_zotero_dates.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

fix_zotero_keys.py

fix Zotero "key is not valid" errors blocking annotation saves

2026-03-21 23:31:00 -04:00

gen_config.py

feat: databricks-bundles Python resource loader — typed DAB resources

2026-03-25 00:29:13 -04:00

generate_alr_filenames.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_alr_reference_tables.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

generate_alr_table_models.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_asr_table_models.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_bls_express.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

generate_cclf_docs.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

generate_cclf_filenames.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

generate_cclf_models.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_ccw_docs.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_ccw_models.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_cms_express.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_expressions.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_models.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_quality_measure_docs.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_reach_docs.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_reach_models.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_reach_participants.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

generate_skin_sub_claims.py

data: synthetic skin sub claims — 2,000 encounters, product + application codes

2026-03-25 10:09:20 -04:00

generate_ssp_county_models.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

generate_zot_tables.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

ingest_asp.py

data: ASP ingestion — strict skin sub filter, full provenance, 86 canonical URLs

2026-03-25 09:44:18 -04:00

install_certs.sh

feat: skinny packages, tree reorg, cloud abstraction (refs #182–#201)

2026-03-24 15:00:26 -04:00

migrate_zotero.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

pkg_drift.py

feat: package supply chain — inventory, mirrors, drift, vuln scanning (refs #159–#168)

2026-03-24 17:02:30 -04:00

pkg_inventory.py

fix: pkg inventory parser — filter self-refs, parse optional-dependencies

2026-03-24 22:33:42 -04:00

pkg_issues.py

feat: package supply chain — inventory, mirrors, drift, vuln scanning (refs #159–#168)

2026-03-24 17:02:30 -04:00

pkg_mirror_sync.py

fix: mirror sync also skips local project name

2026-03-24 22:45:42 -04:00

populate_quality_measure_value_sets.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

register_seeds.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

scrape_bls_data.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

scrape_cms_data.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

search_pubmed_skin_subs.py

fix: bib.sync and XML parsers — Zotero key validation, creators, journalArticle, itertext

2026-03-25 14:04:32 -04:00

seed_col_tags.py

implement P6 bibliography: col tags, bib sync, generate models, lineage (fix #22, fix #23, fix #24, fix #25)

2026-03-12 17:25:30 -04:00

setup_unity_catalog.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

snowball_citations.py

fix: bib.sync and XML parsers — Zotero key validation, creators, journalArticle, itertext

2026-03-25 14:04:32 -04:00

test_databricks_connection.py

reorg dev/ into scripts/ and seeds/, add PY2024 value sets

2026-03-03 22:05:49 -05:00

test_network_isolation.sh

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00

unity_catalog_example.py

chore: clean sweep — lint, format, stale refs, generated artifacts

2026-03-24 17:33:55 -04:00