Kill extract-wave truncation: 65k ceiling, hard thinking budget, reasoning-token telemetry
Job 98194fa8d215 showed every extract call hitting the 32k cap with only ~20k chars visible despite reasoning effort=low - Gemini 2.5 Pro still burned ~25k thinking tokens per sheet. - EXTRACT_MAX_TOKENS default 32768 -> 65536 (model output ceiling) - new EXTRACT_REASONING_MAX_TOKENS (default 2048): OpenRouter reasoning max_tokens / Gemini thinking_budget; takes precedence over effort - log per-call reasoning token counts (usage.completion_tokens_details) and include thinking count in the finish_reason=length marker
This commit is contained in:
+10
-3
@@ -74,13 +74,20 @@ MAX_PAGES = int(os.getenv("MAX_PAGES", "60"))
|
||||
MAX_DIMENSION = int(os.getenv("MAX_DIMENSION", "2400")) # px cap on the long edge
|
||||
LLM_TIMEOUT = int(os.getenv("LLM_TIMEOUT", "180")) # seconds per call
|
||||
# Gemini 2.5 Pro counts thinking tokens against max_tokens, so the visible
|
||||
# JSON budget is well under this number on dense sheets. 32768 leaves real
|
||||
# headroom; raise via env if a set still truncates.
|
||||
EXTRACT_MAX_TOKENS = int(os.getenv("EXTRACT_MAX_TOKENS", "32768"))
|
||||
# JSON budget is well under this number on dense sheets. 65536 is the model's
|
||||
# output ceiling - give thinking all the room it wants so visible JSON never
|
||||
# truncates; the thinking budget itself is capped separately below.
|
||||
EXTRACT_MAX_TOKENS = int(os.getenv("EXTRACT_MAX_TOKENS", "65536"))
|
||||
# Reasoning effort for the per-sheet extractor (OpenRouter reasoning knob).
|
||||
# Extraction is perceptive, not deliberative - "low" keeps thinking tokens
|
||||
# from eating the output budget. Empty string disables the parameter.
|
||||
EXTRACT_REASONING_EFFORT = os.getenv("EXTRACT_REASONING_EFFORT", "low").strip()
|
||||
# Hard thinking-token budget for the extractor (OpenRouter reasoning
|
||||
# max_tokens -> Gemini thinking_budget). "low" effort alone still let Gemini
|
||||
# burn ~25k thinking tokens per sheet (job 98194fa8d215); a hard cap forces
|
||||
# the budget into visible output. 0 disables -> falls back to the effort knob.
|
||||
# Mutually exclusive with effort when set (OpenRouter rejects both together).
|
||||
EXTRACT_REASONING_MAX_TOKENS = int(os.getenv("EXTRACT_REASONING_MAX_TOKENS", "2048"))
|
||||
REASON_MAX_TOKENS = int(os.getenv("REASON_MAX_TOKENS", "4096"))
|
||||
|
||||
# -- QAQC stage knobs (Stages 0-1, 3, 6-11) -------------------------
|
||||
|
||||
Reference in New Issue
Block a user