Kill extract-wave truncation: 65k ceiling, hard thinking budget, reasoning-token telemetry
Docker Release / build-and-push (push) Successful in 57s
Docker Release / release (push) Skipped

Job 98194fa8d215 showed every extract call hitting the 32k cap with only
~20k chars visible despite reasoning effort=low - Gemini 2.5 Pro still
burned ~25k thinking tokens per sheet.

- EXTRACT_MAX_TOKENS default 32768 -> 65536 (model output ceiling)
- new EXTRACT_REASONING_MAX_TOKENS (default 2048): OpenRouter reasoning
  max_tokens / Gemini thinking_budget; takes precedence over effort
- log per-call reasoning token counts (usage.completion_tokens_details)
  and include thinking count in the finish_reason=length marker
This commit is contained in:
2026-08-09 08:17:41 -05:00
parent 76e0a52658
commit 3d7fce7bf9
4 changed files with 53 additions and 16 deletions
+10 -3
View File
@@ -74,13 +74,20 @@ MAX_PAGES = int(os.getenv("MAX_PAGES", "60"))
MAX_DIMENSION = int(os.getenv("MAX_DIMENSION", "2400")) # px cap on the long edge
LLM_TIMEOUT = int(os.getenv("LLM_TIMEOUT", "180")) # seconds per call
# Gemini 2.5 Pro counts thinking tokens against max_tokens, so the visible
# JSON budget is well under this number on dense sheets. 32768 leaves real
# headroom; raise via env if a set still truncates.
EXTRACT_MAX_TOKENS = int(os.getenv("EXTRACT_MAX_TOKENS", "32768"))
# JSON budget is well under this number on dense sheets. 65536 is the model's
# output ceiling - give thinking all the room it wants so visible JSON never
# truncates; the thinking budget itself is capped separately below.
EXTRACT_MAX_TOKENS = int(os.getenv("EXTRACT_MAX_TOKENS", "65536"))
# Reasoning effort for the per-sheet extractor (OpenRouter reasoning knob).
# Extraction is perceptive, not deliberative - "low" keeps thinking tokens
# from eating the output budget. Empty string disables the parameter.
EXTRACT_REASONING_EFFORT = os.getenv("EXTRACT_REASONING_EFFORT", "low").strip()
# Hard thinking-token budget for the extractor (OpenRouter reasoning
# max_tokens -> Gemini thinking_budget). "low" effort alone still let Gemini
# burn ~25k thinking tokens per sheet (job 98194fa8d215); a hard cap forces
# the budget into visible output. 0 disables -> falls back to the effort knob.
# Mutually exclusive with effort when set (OpenRouter rejects both together).
EXTRACT_REASONING_MAX_TOKENS = int(os.getenv("EXTRACT_REASONING_MAX_TOKENS", "2048"))
REASON_MAX_TOKENS = int(os.getenv("REASON_MAX_TOKENS", "4096"))
# -- QAQC stage knobs (Stages 0-1, 3, 6-11) -------------------------