a03801fb63
All checks were successful
CI / lint (push) Successful in 34s
CI / notebooks-smoke (push) Successful in 1m28s
Deploy / notebooks (push) Has been skipped
Deploy / zotero (push) Has been skipped
Deploy / docs (push) Has been skipped
Deploy / api (push) Has been skipped
Deploy / llm (push) Has been skipped
Deploy / mc (push) Has been skipped
Infra CI / notebooks (push) Successful in 3m33s
Infra CI / zotero (push) Successful in 23s
Infra CI / docs (push) Successful in 1m34s
Infra CI / api (push) Successful in 1m5s
Infra CI / llm (push) Successful in 49s
Infra CI / mc (push) Successful in 13s
Deploy / report (push) Successful in 15s
CI / test (push) Successful in 17m10s
Harden / build-scan-report (push) Successful in 18m38s
Notebooks Integration / notebooks-integration (push) Successful in 7m29s
Renovate / renovate (push) Successful in 18s
Zotero Sync / zotero-sync (push) Successful in 1m13s
Package Supply Chain / pkg-supply-chain (push) Successful in 1m0s
An unscoped 'bib backfill-comments' walks every un-enriched reg.gov stub in the store (~167k) at the 970/hr rate cap — a week-long crawl that buried the freshly farmed CMS-2026-2377 comments (highest item ids, ORDER BY i.id) at the back of the queue and blocked the P37 extract/index chain behind it. backfill_details now takes docket=, filtering on the store's reg-docket:<id> tag, and the CLI forwards --docket. The September refarm script uses it, and no longer calls comments extract-ocr, a phase-2 stub that always exits 2 (#253) and would have logged a spurious failed step every run. refs #615
55 lines
2.4 KiB
Bash
Executable File
55 lines
2.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Post-comment-close re-farm for docket CMS-2026-2377 (CY2027 PFS NPRM,
|
|
# CMS-1848-P). Scheduled via host crontab for 2026-09-15 (comment period
|
|
# closes 2026-09-14 per 91 FR 43842 DATES block). Tracker: issue #615.
|
|
#
|
|
# Every step is resumable/incremental — safe to re-run on partial failure.
|
|
set -uo pipefail
|
|
cd "$(dirname "$0")/../.."
|
|
export PATH="$HOME/.local/bin:$PATH"
|
|
set -a; source .env; set +a
|
|
|
|
mkdir -p .state/comments
|
|
LOG=.state/comments/refarm-CMS-2026-2377.log
|
|
exec >>"$LOG" 2>&1
|
|
echo "=== re-farm run $(date -Is) ==="
|
|
|
|
FAILURES=0
|
|
|
|
uv run stack bib fetch-pfs-comments || { echo "WARN: fetch-pfs-comments rc=$?"; FAILURES=$((FAILURES+1)); }
|
|
uv run stack bib backfill-comments --docket CMS-2026-2377 || { echo "WARN: backfill rc=$?"; FAILURES=$((FAILURES+1)); }
|
|
uv run stack comments extract --docket CMS-2026-2377 || { echo "WARN: extract rc=$?"; FAILURES=$((FAILURES+1)); }
|
|
# extract-ocr is a phase-2 stub (always exits 2, no --docket) — see #253.
|
|
# ocr_needed attachments stay flagged in combined.md frontmatter for now.
|
|
uv run stack llm index --collection comments --docket CMS-2026-2377 \
|
|
|| { echo "WARN: index rc=$?"; FAILURES=$((FAILURES+1)); }
|
|
|
|
COUNTS=$(uv run python - <<'EOF'
|
|
from conf import connect
|
|
s = connect.bib()
|
|
con = s._con()
|
|
total, enriched = con.execute(
|
|
"SELECT count(*), sum(CASE WHEN coalesce(abstract,'')<>'' THEN 1 ELSE 0 END) "
|
|
"FROM items WHERE url LIKE 'https://www.regulations.gov/comment/CMS-2026-2377-%'"
|
|
).fetchone()
|
|
print(f"comments={total} enriched={enriched}")
|
|
s.close()
|
|
EOF
|
|
) || { echo "WARN: counts rc=$?"; FAILURES=$((FAILURES+1)); }
|
|
echo "$COUNTS"
|
|
|
|
if [ "$FAILURES" -eq 0 ]; then
|
|
RESULT_MSG="Automated post-close re-farm ran $(date -I): $COUNTS."
|
|
else
|
|
RESULT_MSG="Automated post-close re-farm ran $(date -I) — completed with $FAILURES failed steps — check log. $COUNTS."
|
|
fi
|
|
|
|
curl -sf -X POST \
|
|
"https://git.fhirworx.io/api/v1/repos/homelab/stack/issues/615/comments" \
|
|
-H "Authorization: token $GITEA_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-H "User-Agent: curl/8.5.0" \
|
|
-d "{\"body\": \"$RESULT_MSG Log: .state/comments/refarm-CMS-2026-2377.log. Review counts, spot-check indexing, then close this issue and milestone P37.\n\nIf complete: close #615, close milestone P37, and REMOVE the crontab line (re-fires annually).\"}" \
|
|
|| echo "WARN: gitea comment failed"
|
|
echo "=== done $(date -Is) ==="
|