52ce69acf1
Some checks failed
CI / notebooks-smoke (push) Successful in 1m34s
CI / test (push) Failing after 2m19s
Infra CI / zotero (push) Successful in 16s
Infra CI / docs (push) Successful in 17s
Infra CI / api (push) Successful in 1m7s
CI / lint (push) Successful in 38s
Infra CI / llm (push) Successful in 1m10s
Infra CI / mc (push) Failing after 41s
CI / test (pull_request) Failing after 2m27s
Infra CI / notebooks (push) Successful in 7m51s
CI / notebooks-smoke (pull_request) Successful in 1m25s
Infra CI / zotero (pull_request) Successful in 14s
Infra CI / docs (pull_request) Successful in 19s
Infra CI / notebooks (pull_request) Successful in 51s
CI / lint (pull_request) Successful in 32s
Infra CI / api (pull_request) Successful in 21s
Infra CI / llm (pull_request) Successful in 17s
Infra CI / mc (pull_request) Failing after 1m8s
The excerpt budget was a flat chat_num_ctx * 3 characters on the user
message alone. On FR/regulatory text the instruct model's tokenizer yields
2.5–2.9 chars per token, and the system prompt and the 1,200-token reply
reservation were not counted, so a routine answer rendered 9.4k–10k prompt
tokens into an 8,192-token context. Ollama's context-shift truncation keeps
the first 4 tokens and the tail half: the system prompt and the best-ranked
excerpts were dropped on every such answer, the only trace a WARN in the
ollama log. The nightly golden set has reported missing anchors since.
- prompt_budget_chars(cfg): (chat_num_ctx - chat_max_tokens - template
reserve) * chat_chars_per_token - len(system prompt); new [llm] knob
chat_chars_per_token = 2.4 (measured minimum 2.50 on the golden set)
- /api/chat is sent with truncate: false; the host's 400 for an over-long
prompt (exceed_context_size_error with n_prompt_tokens / n_ctx, or the
older bare message) rebuilds the messages under a proportionally tighter
budget and resends; when the trimming floors cannot shorten the prompt,
or on the final attempt, host truncation is allowed as the last resort
- stack_llm_chat_prompt_overflow_total{resolution} counts rebuilt /
host_truncated; each overflow is logged with the host's token counts