b8b44e5b95
Some checks failed
CI / lint (push) Successful in 54s
Infra CI / zotero (push) Successful in 25s
Infra CI / notebooks (push) Successful in 1m12s
CI / notebooks-smoke (push) Successful in 1m52s
CI / test (push) Failing after 2m53s
Deploy / notebooks (push) Has been skipped
Deploy / zotero (push) Has been skipped
Deploy / docs (push) Has been skipped
Deploy / api (push) Has been skipped
Deploy / llm (push) Has been skipped
Deploy / mc (push) Has been skipped
Infra CI / docs (push) Successful in 1m49s
Infra CI / api (push) Successful in 1m27s
Infra CI / mc (push) Failing after 50s
Infra CI / llm (push) Successful in 1m13s
Deploy / report (push) Has been cancelled
llm.themes stamps the P35 vocabulary onto indexed chunks with one normalised matrix product per batch against the theme cards (no model calls): the top themes clearing --min-score land in cmetadata as themes / theme_scores, with themes_vocab recording the vocabulary version so a vocabulary bump re-stamps. Default scope is the CMS manual sections (corpus, doctype manual); any collection/doctype/key can be stamped. /search takes theme=<slug> (a Python-side match over the comma-joined slugs, like year) and every source carries themes. Also: the thread-local Store test compared against the executor's worker cap (8) although the executor spawns threads lazily, so under load it failed with 7; it now compares against the threads that ran.
5.5 KiB
5.5 KiB
title, sidebar_position
| title | sidebar_position |
|---|---|
| stack llm | 48 |
stack llm
Usage: stack llm [OPTIONS] COMMAND [ARGS]...
Local RAG — index, search, tag.
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ index Embed comments/rules/corpus into pgvector (incremental, │
│ resumable). │
│ restamp Backfill codes/families/elements onto already-indexed chunks. │
│ hosts Show the Ollama fleet: declared VRAM, liveness, models, and │
│ which │
│ host + model would answer a chat right now. │
│ serve Serve the SSO-guarded chat UI (llm.api:app). │
│ vocab The closed theme vocabulary for comment tagging (#574): │
│ validate it │
│ and list its slugs, or show one theme's definition, synonyms │
│ and the FR │
│ section stems it was seeded from. │
│ tag Closed-vocabulary theme tagging of one docket's comments │
│ (P35): │
│ shortlist by similarity to the theme cards, judge each │
│ candidate │
│ yes/no on the largest live host with the scoring chunk as │
│ evidence, │
│ record state in the item's extra_json, and replace its llm: │
│ tags. │
│ Resumable — unchanged comments are skipped unless --force. │
│ eval-tags Score the theme tagger against the golden set (P35 gate, │
│ #577): │
│ per-theme precision/recall, micro-F1 and the abstain rate, │
│ then the │
│ fan-out gate (micro-F1 ≥ 0.70; no theme with support ≥ 5 under │
│ 0.5 │
│ precision). Default reads the tags the last run stored on each │
│ item │
│ (no model calls); --live re-tags each golden comment now. Exit │
│ 1 when │
│ the gate fails. │
│ prune-rules Remove Federal Register rules from the corpus collection and │
│ stamp │
│ proposed/final/correction on their rules chunks. Rules are │
│ indexed as │
│ anchored FR paragraphs in `rules`; the PDF copies that also │
│ landed in │
│ `corpus` surfaced the same text as 'corpus' with a weaker │
│ link. Safe │
│ to re-run (idempotent); `stack llm index` no longer adds them │
│ back. │
│ stamp-themes Categorise indexed chunks by the P35 theme vocabulary — by │
│ default │
│ the CMS manual sections — with one cosine pass against the │
│ theme │
│ cards (no model calls). Writes `themes` / `theme_scores` / │
│ `themes_vocab` into the chunk metadata; `/search?theme=<slug>` │
│ and the │
│ chat's manual-section filters read them. │
╰──────────────────────────────────────────────────────────────────────────────╯