Files
stack/infra/cloudflared/worker/wrangler.toml
kert 88901d0d17
Some checks failed
CI / skinny-install (aco) (push) Successful in 2m19s
CI / lint-test (push) Failing after 3m13s
CI / skinny-install (api) (push) Successful in 1m45s
CI / skinny-install (bcda) (push) Successful in 1m41s
CI / skinny-install (bib) (push) Successful in 2m2s
CI / skinny-install (bls) (push) Successful in 2m1s
CI / skinny-install (ccw) (push) Successful in 1m37s
CI / skinny-install (cli) (push) Successful in 2m7s
CI / skinny-install (cms) (push) Successful in 1m40s
CI / skinny-install (conf) (push) Successful in 1m57s
CI / skinny-install (perf) (push) Successful in 1m41s
CI / skinny-install (rex) (push) Successful in 1m46s
CI / skinny-install (pfs) (push) Successful in 2m37s
Deploy / build-scan-report (push) Failing after 2m34s
feat: Cloudflare Worker PDF proxy + headless batch retrieval (fixes #273)
Cloudflare Worker (pdf-proxy) deployed to bypass ISP-level SciHub blocking:
- Worker fetches PDFs from SciHub mirrors on Cloudflare's edge network
- Authenticated via X-Proxy-Key header
- Routes: /pdf?doi=, /fetch?url=, /health
- Deployed at pdf-proxy.lite7889.workers.dev

fetch_pdfs.py updated with 5-phase waterfall:
1. Unpaywall (legal OA, ~4% for this corpus)
2. PMC (free, PMCID-based)
3. Semantic Scholar (batch 500)
4. CF Worker → SciHub (72% hit rate on first 50, running full batch)
5. SciHub direct (fallback with proxy)

Also: cloudflared tunnel (stack-proxy) with WARP routing in compose.yml,
CoreDNS sci-hub domain routing via unfiltered DNS.

Test: 36/50 PDFs (72%) retrieved in first batch via Worker.
2026-03-25 21:54:44 -04:00

9 lines
149 B
TOML

name = "pdf-proxy"
main = "worker.js"
compatibility_date = "2026-03-25"
[vars]
# SECRET is set via `wrangler secret put SECRET`
workers_dev = true