regulations.gov sometimes returns 200 OK with Content-Length: 0 for
attachment URLs whose files have been removed from object storage but
whose comment metadata still references them. download_attachment was
silently writing the 0-byte response to disk, registering it as a real
attachment, and refusing to refetch on retry (cache hit on the empty
file). Five comments in bib.sqlite ended up with phantom attachments.
- download_attachment raises new AttachmentMissing on 0-byte body and
rejects cached 0-byte files instead of returning the stale path
- download_attachments_batch returns (paths, missing_urls); transient
HTTP/transport failures are still untracked and get retried next run
- backfill_details tags affected items 'attachment:gone' (mirrors the
existing 'enriched:gone' convention for 404'd comments) and counts
them in a new att_gone stats field
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dddd325153