kert dddd325153 fix(bib): tag attachment:gone when CDN serves 200 + empty body ...

regulations.gov sometimes returns 200 OK with Content-Length: 0 for
attachment URLs whose files have been removed from object storage but
whose comment metadata still references them. download_attachment was
silently writing the 0-byte response to disk, registering it as a real
attachment, and refusing to refetch on retry (cache hit on the empty
file). Five comments in bib.sqlite ended up with phantom attachments.

- download_attachment raises new AttachmentMissing on 0-byte body and
  rejects cached 0-byte files instead of returning the stale path
- download_attachments_batch returns (paths, missing_urls); transient
  HTTP/transport failures are still untracked and get retried next run
- backfill_details tags affected items 'attachment:gone' (mirrors the
  existing 'enriched:gone' convention for 404'd comments) and counts
  them in a new att_gone stats field

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-04-27 15:04:58 -04:00

28 KiB

Raw Blame History

View Raw