Verbose per-call LLM logging, raw request/response dumps, and end-of-log cost summary.
- [LLM] line per call: stage, model, prompt size, output size, cost, parsed item counts - LLM_RAW_DUMP: full prompt/response JSON per call under outputs/<job>/llm_raw/ - Cost block at tail of job.log (per-stage, per-model, cached vs live) - Agent mode: reset llm cost counters per job; review finalization now teed into job.log + dumps
This commit is contained in:
@@ -102,6 +102,14 @@ CLUSTER_MAX = int(os.getenv("CLUSTER_MAX", "120"))
|
||||
LLM_CACHE = os.getenv("LLM_CACHE", "false").strip().lower() in ("1", "true", "yes")
|
||||
LLM_CACHE_DIR = os.getenv("LLM_CACHE_DIR", os.path.join(_BASE_DIR, ".llm_cache"))
|
||||
|
||||
# Verbose LLM observability. Per call, one line lands in the job log (model,
|
||||
# backend, prompt size, response size, parsed-item counts, per-call cost) and
|
||||
# the full request/response is dumped to <out_dir>/llm_raw/ (base64 image
|
||||
# payloads excluded; image count recorded instead) so missed or hallucinated
|
||||
# items can be traced back to exactly what the model saw and returned.
|
||||
LLM_VERBOSE = os.getenv("LLM_VERBOSE", "true").strip().lower() in ("1", "true", "yes")
|
||||
LLM_RAW_DUMP = os.getenv("LLM_RAW_DUMP", "true").strip().lower() in ("1", "true", "yes")
|
||||
|
||||
# Parallelism (ThreadPoolExecutor workers)
|
||||
EXTRACT_CONCURRENCY = int(os.getenv("EXTRACT_CONCURRENCY", "4"))
|
||||
REASON_CONCURRENCY = int(os.getenv("REASON_CONCURRENCY", "4"))
|
||||
|
||||
Reference in New Issue
Block a user