llm: tag write-back to bib + zotero-sync integration #576

New Issue

kert · 2026-07-16T19:26:09Z

kert commented

2026-07-16 19:26:09 +00:00

Tags land where comments are read: the bib Store, then Zotero via the existing nightly sync.

Scope

write llm:<tag> tags to bib Store; idempotent — re-tagging replaces prior llm: tags, never touches hand tags
verify nightly zotero-sync picks them up; scoped sync with --tag llm: tested
provenance note per tagged item (model, vocab version, confidence) in extra_json or notes

Acceptance

re-running the tagger yields no duplicate tags; hand tags provably untouched; tags visible in Zotero after sync

Spec: docs/superpowers/specs/2026-07-16-llm-module-design.md

Tags land where comments are read: the bib Store, then Zotero via the existing nightly sync. **Scope** - [ ] write `llm:<tag>` tags to bib Store; idempotent — re-tagging replaces prior `llm:` tags, never touches hand tags - [ ] verify nightly zotero-sync picks them up; scoped sync with `--tag llm:` tested - [ ] provenance note per tagged item (model, vocab version, confidence) in extra_json or notes **Acceptance** - re-running the tagger yields no duplicate tags; hand tags provably untouched; tags visible in Zotero after sync Spec: [`docs/superpowers/specs/2026-07-16-llm-module-design.md`](https://git.fhirworx.io/homelab/stack/src/branch/main/docs/superpowers/specs/2026-07-16-llm-module-design.md)

kert added this to the P35: LLM Tagging — closed-vocab RAG tagging into bib/Zotero milestone 2026-07-16 19:26:09 +00:00

kert added the bib llm labels 2026-07-16 19:26:09 +00:00

kert commented

2026-08-13 23:46:55 +00:00

Status audit (2026-08-13, repo-verified): NOT DONE. No tag write-back path in src/llm or bib sync integration (P35 not started).

**Status audit (2026-08-13, repo-verified):** NOT DONE. No tag write-back path in src/llm or bib sync integration (P35 not started).

kert referenced this issue from a commit

2026-09-22 20:55:01 +00:00

feat(llm): theme tagging chain + docket runner + write-back — llm.tagger, Store.merge_extra, stack llm tag (refs #575, #576)

kert referenced this issue from a commit

2026-09-22 20:55:01 +00:00

merge: #574/#575 — P35 theme vocabulary + tagging chain, runner and write-back (refs #574, #575, #576)

kert referenced this issue from a commit

2026-09-22 21:08:07 +00:00

merge: #577 — golden-set gate, eval-tags, vocab v2 and tuned tagger defaults (refs #574, #575, #576, #577)

kert commented

2026-09-22 21:09:23 +00:00

Write-back is in (merged 5e37641, llm.tagger.apply_tags + Store.merge_extra): a run makes the item's llm:* tags exactly the accepted set — hand tags (topic:, stance:, reg-docket: …) are never touched (tested), re-runs add nothing (idempotent), and provenance (vocab version, model, confidence, evidence chunk) sits in extra_json.llm_tags. Pilot: stack llm tag --docket CMS-2026-2377 --limit 300 is writing tags now.

Open, and it is a scope decision rather than a defect: the nightly zotero-sync job syncs only --tag source:email and --tag source:federal-register items, so regulations.gov comment items — and therefore their llm: tags — never reach Zotero today. Bringing them in means either a third nightly step (stack bib sync-zotero --tag llm: --no-hold, which would hydrate every tagged comment into Zotero: 300 now, 43k for the docket, ~250k corpus-wide) or a narrower rule (only comments in a curated collection). Leaving this issue open for that call; the bib side is done.

**Write-back is in (merged 5e37641, `llm.tagger.apply_tags` + `Store.merge_extra`)**: a run makes the item's `llm:*` tags exactly the accepted set — hand tags (`topic:`, `stance:`, `reg-docket:` …) are never touched (tested), re-runs add nothing (idempotent), and provenance (vocab version, model, confidence, evidence chunk) sits in `extra_json.llm_tags`. Pilot: `stack llm tag --docket CMS-2026-2377 --limit 300` is writing tags now. **Open, and it is a scope decision rather than a defect**: the nightly `zotero-sync` job syncs only `--tag source:email` and `--tag source:federal-register` items, so regulations.gov comment items — and therefore their `llm:` tags — never reach Zotero today. Bringing them in means either a third nightly step (`stack bib sync-zotero --tag llm: --no-hold`, which would hydrate every tagged comment into Zotero: 300 now, 43k for the docket, ~250k corpus-wide) or a narrower rule (only comments in a curated collection). Leaving this issue open for that call; the bib side is done.

kert referenced this issue from a commit

2026-09-22 21:09:36 +00:00

docs(llm): record the P35 golden-gate results and the Zotero scope caveat in the tagging spec (refs #577, #576)

kert referenced this issue from a commit

2026-09-22 21:09:36 +00:00

merge: P35 spec results section (refs #577, #576)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: homelab/stack#576