prisma: parallel multi-host screening via the fleet pool #655

New Issue

kert · 2026-08-18T19:24:20Z

kert commented

2026-08-18 19:24:20 +00:00

prisma.screen.run is a sequential single-provider loop (one item at a time against one base URL). With the generation pool in place:

Worker pool keyed to fleet hosts: N workers, each acquire()s the idlest host per item; per-item db.commit() already makes crashes lose at most in-flight items.
SQLite is the write bottleneck concern — writes are tiny (tags + one note) so a single writer thread consuming a results queue is enough.
Same for prisma.eligible (stage-3, full-text prompts — the stage that actually NEEDS the 4090's prompt-processing throughput) and prisma.extract.
Benchmark note from the 2026-08-18 palliative run (qwen2.5:14b-ctx8k, ~2.5-3.5k-token prompts, ~150-token tool-call outputs): rack 3060 alone ≈ 15-25 items/min. Expected: 4090 ≈ 3-4x (bandwidth 1008 vs 360 GB/s + ~4x tensor compute), 5070 Ti ≈ 2.5-3.5x (896 GB/s GDDR7, Blackwell), fleet ≈ 6-8x → a 5k-record stage-2 pass in ~30-45 min.
Optional quality lever: the 4090's 24GB fits qwen2.5:32b Q4 (~19GB) — consider routing stage-3 eligibility to the 32B on the 4090 while stage-2 triage stays on 14B; needs a small bake-off against a human-reviewed sample first.

`prisma.screen.run` is a sequential single-provider loop (one item at a time against one base URL). With the generation pool in place: - Worker pool keyed to fleet hosts: N workers, each `acquire()`s the idlest host per item; per-item `db.commit()` already makes crashes lose at most in-flight items. - SQLite is the write bottleneck concern — writes are tiny (tags + one note) so a single writer thread consuming a results queue is enough. - Same for `prisma.eligible` (stage-3, full-text prompts — the stage that actually NEEDS the 4090's prompt-processing throughput) and `prisma.extract`. - Benchmark note from the 2026-08-18 palliative run (qwen2.5:14b-ctx8k, ~2.5-3.5k-token prompts, ~150-token tool-call outputs): rack 3060 alone ≈ 15-25 items/min. Expected: 4090 ≈ 3-4x (bandwidth 1008 vs 360 GB/s + ~4x tensor compute), 5070 Ti ≈ 2.5-3.5x (896 GB/s GDDR7, Blackwell), fleet ≈ 6-8x → a 5k-record stage-2 pass in ~30-45 min. - Optional quality lever: the 4090's 24GB fits qwen2.5:32b Q4 (~19GB) — consider routing stage-3 eligibility to the 32B on the 4090 while stage-2 triage stays on 14B; needs a small bake-off against a human-reviewed sample first.

kert added this to the P45: GPU Fleet — pooled inference across rack 3060 / rig 4090 / laptop 5070 Ti milestone 2026-08-18 19:24:20 +00:00

kert added the bib llm labels 2026-08-18 19:24:20 +00:00

kert referenced this issue from a commit

2026-08-18 19:54:26 +00:00

feat(prisma): fleet screening — native Ollama provider + multi-host parallel driver (refs #654, #655)

kert referenced this issue

2026-08-18 20:15:23 +00:00

fleet: expose + register rig 4090 and laptop 5070 Ti Ollama endpoints #653

kert referenced this issue from a commit

2026-08-18 22:44:23 +00:00

docs(notebooks): disclose 32B uncertain-adjudication pass in palliative PRISMA methods (refs #653, #655)

kert referenced this issue

2026-09-11 21:20:24 +00:00

fleet: expose + register rig 4090 and laptop 5070 Ti Ollama endpoints #653

kert referenced this issue from a commit

2026-09-11 22:23:17 +00:00

perf(zot): create_db runs the schema script as one transaction

kert commented

2026-09-11 22:37:32 +00:00

Done in 49a2f6f (merged to main). prisma.fleet.run_pool now owns the multi-host thread-pool loop that screen had inline: at most --per-host in-flight requests per Ollama host, only the LLM call runs on the pool, every SQLite touch and per-item commit stays on the calling thread, and a host that fails twice in a row is retired for the rest of the run so its queued items go to the surviving hosts (a dead host loses at most its in-flight items). screen, eligibility and extract each supply prepare/apply/on_error glue; eligibility items with no retrievable full text resolve as uncertain/unavailable-fulltext without a fleet call. CLI: prisma eligible and prisma extract gain --hosts / --fleet-model / --num-ctx / --per-host matching prisma screen (native /api/chat with per-request num_ctx, bypassing PRISMA_LLM_*). Docs regenerated; tests cover the driver (slot accounting, retirement, prepare-skips, error stages) and all three stages end to end with fake providers. Not exercised here: a live multi-host run — the rig 4090 is still pending registration (#653 closed earlier today with the laptop live).

Done in 49a2f6f (merged to main). prisma.fleet.run_pool now owns the multi-host thread-pool loop that screen had inline: at most --per-host in-flight requests per Ollama host, only the LLM call runs on the pool, every SQLite touch and per-item commit stays on the calling thread, and a host that fails twice in a row is retired for the rest of the run so its queued items go to the surviving hosts (a dead host loses at most its in-flight items). screen, eligibility and extract each supply prepare/apply/on_error glue; eligibility items with no retrievable full text resolve as uncertain/unavailable-fulltext without a fleet call. CLI: prisma eligible and prisma extract gain --hosts / --fleet-model / --num-ctx / --per-host matching prisma screen (native /api/chat with per-request num_ctx, bypassing PRISMA_LLM_*). Docs regenerated; tests cover the driver (slot accounting, retirement, prepare-skips, error stages) and all three stages end to end with fake providers. Not exercised here: a live multi-host run — the rig 4090 is still pending registration (#653 closed earlier today with the laptop live).

kert closed this issue

2026-09-11 22:37:32 +00:00

kert referenced this issue from a commit

2026-09-11 22:37:42 +00:00

feat(prisma): fleet mode for eligibility and extract via a shared run_pool driver (refs #655)

kert referenced this issue from a commit

2026-09-11 22:37:42 +00:00

merge: #655 — fleet mode for prisma eligibility and extract via a shared run_pool driver (refs #655)

kert referenced this issue

2026-09-11 22:45:13 +00:00

comments: extract author/org metadata from attachment text #416

Sign in to join this conversation.