Compare commits
3
Commits
afa1089311
...
f7e1b6bb7c
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f7e1b6bb7c | ||
|
|
bf508bfdf6 | ||
|
|
a6b0c8fdfa |
@@ -0,0 +1,160 @@
|
|||||||
|
# Session Notes — Conflict Checker
|
||||||
|
|
||||||
|
Orientation for a new coding session. Setup and Docker details live in [README.md](README.md). This file tracks what the code actually does and what tends to waste time.
|
||||||
|
|
||||||
|
**Last updated:** 2026-08-02 · `agent-mode` branch (merged `main` tip `bf508bf`)
|
||||||
|
|
||||||
|
## What this is
|
||||||
|
|
||||||
|
Cross-discipline **design contradiction / senior-architect QAQC** for construction drawing PDFs (Arch, Struct, Mech, Elec, Plumb, FP, etc.). Flags disagreements *between disciplines* before a set goes to bid/permit.
|
||||||
|
|
||||||
|
**Not** IronBid’s scope-ownership conflict checker.
|
||||||
|
|
||||||
|
Source of truth: Scout IT Gitea — `gitea.scoutitsystems.com/woogi/Conflict_Checker`.
|
||||||
|
|
||||||
|
## Stack
|
||||||
|
|
||||||
|
| Layer | Detail |
|
||||||
|
|-------|--------|
|
||||||
|
| API | Python 3.12, FastAPI, Uvicorn ([backend/main.py](backend/main.py)) |
|
||||||
|
| UI | Single static file [frontend/index.html](frontend/index.html), served by FastAPI |
|
||||||
|
| Pipelines | **Classic:** [backend/pipeline/runner.py](backend/pipeline/runner.py) (shared by web + CLI). **Agent:** [backend/agents/runner.py](backend/agents/runner.py) — experimental scoped specialist agents, selected per job (`pipeline_mode`) |
|
||||||
|
| Review gate | Agent jobs stop at `needs_review` for human decisions before the report emails ([backend/review/](backend/review/)) |
|
||||||
|
| LLM | OpenRouter via `openai` SDK; default `google/gemini-2.5-pro`. Vision always OpenRouter; text stages can use local vLLM (classic/hybrid only) |
|
||||||
|
| PDF | `pdf2image` + system `poppler-utils` → JPEG page images |
|
||||||
|
| Jobs | In-memory threads ([backend/jobs.py](backend/jobs.py)) — no Redis/DB |
|
||||||
|
| Tests | `pytest tests/` (~82 tests; see Quick start) |
|
||||||
|
| Deploy | Docker Compose; app on port **8099**; public URL `https://conchecker.scoutitsystems.com` |
|
||||||
|
|
||||||
|
## Live pipeline (authoritative)
|
||||||
|
|
||||||
|
README still describes an older 5-stage extract-then-compare loop. **Trust `runner.py`.** Actual flow:
|
||||||
|
|
||||||
|
```
|
||||||
|
PDF → images → extract → sheet index → jurisdiction
|
||||||
|
→ normalize → project intelligence (GOIDs)
|
||||||
|
→ cluster → conflict reason
|
||||||
|
→ QAQC / code / constructability
|
||||||
|
→ dedup-validate → risk → RFIs → report
|
||||||
|
```
|
||||||
|
|
||||||
|
| Runner stage | Module | Notes |
|
||||||
|
|--------------|--------|--------|
|
||||||
|
| PDF → images | `pdf_processor` | Rasterize |
|
||||||
|
| Extract assertions | `extractor` | Vision, per sheet |
|
||||||
|
| Classify sheet index | `sheet_index` | LLM |
|
||||||
|
| Jurisdiction profile | `jurisdiction` | After cover meta |
|
||||||
|
| Normalize | `normalizer` | LLM batches |
|
||||||
|
| Project intelligence | `normalizer.build_project_intelligence` | GOIDs + relationships |
|
||||||
|
| Cluster | `llm_clusterer` or `clusterer` | Default `CLUSTERER=llm` |
|
||||||
|
| Conflicts | `conflict_checker` | Per-cluster vision reason |
|
||||||
|
| QAQC / code / constructability | `qaqc_review`, `code_review`, `constructability` | Full-set / batched |
|
||||||
|
| Validate & dedup | `validator` | Merges conflict + QAQC + code + construct issues |
|
||||||
|
| Risk / RFIs | `risk`, `rfi` | Text-only |
|
||||||
|
| Report | `report` | `conflicts.json` + `report.md` (+ stage JSON dumps when `out_dir` set) |
|
||||||
|
|
||||||
|
Design notes for Stage 2/3 engines also live under `Changes/*.docx`.
|
||||||
|
|
||||||
|
**Agent pipeline** (`pipeline_mode=agent`): scoped specialist agents in [backend/agents/](backend/agents/) (extractors, linker, brain, critics, RFI writer) run through `Orchestrator` + `ProjectMemory`; artifacts under `outputs/<job_id>/agent/`. Agent mode is OpenRouter-only (no hybrid) and, when `AGENT_REQUIRE_REVIEW=true`, stops at `needs_review` until a human saves decisions and finalizes via the review endpoints. Design docs: `docs/superpowers/`.
|
||||||
|
|
||||||
|
## HTTP API (current)
|
||||||
|
|
||||||
|
| Method | Path | Purpose |
|
||||||
|
|--------|------|---------|
|
||||||
|
| GET | `/health` | Liveness + `model` / `text_model` + `version` / `build` + key/email flags |
|
||||||
|
| GET | `/models` | OpenRouter catalog split into `vision[]` / `text[]` + `defaults`, with per-1M-token pricing (cached ~1h; **502** when OpenRouter is unreachable) |
|
||||||
|
| POST | `/check` | Upload PDF; returns `{job_id}` immediately |
|
||||||
|
| GET | `/jobs/{id}` | Status poll. Running: `stage` + `log_tail`. Done/error/needs_review/finalization_error: `report` and/or `error` + full `log` |
|
||||||
|
| GET | `/jobs/{id}/log` | Full run log as `text/plain` (404 when no log file) |
|
||||||
|
| GET | `/jobs/{id}/review` | Review queue + progress + saved decisions (agent jobs) |
|
||||||
|
| POST | `/jobs/{id}/review-decisions` | Save reviewer decisions (409 outside needs_review/reviewing) |
|
||||||
|
| POST | `/jobs/{id}/finalize-review` | Background finalize + send report (409 unless review gate passed) |
|
||||||
|
| GET | `/jobs/{id}/sheet-image/{page}` | JPEG of source PDF page for the sheet viewer |
|
||||||
|
| GET | `/` | Serves `frontend/index.html` |
|
||||||
|
|
||||||
|
### `/check` form fields
|
||||||
|
|
||||||
|
- Required: `file` (PDF)
|
||||||
|
- Optional: `notification_email`, `project_name`, `address`, `occupancy`, `work_type`
|
||||||
|
- Pipeline: `pipeline_mode` (`classic` default, or `agent`)
|
||||||
|
- Compute: `text_local` (`true` = hybrid local text; forced off for agent mode)
|
||||||
|
- Models: `vision_model`, `text_model` (OpenRouter ids; blank = config defaults)
|
||||||
|
|
||||||
|
## Job logs
|
||||||
|
|
||||||
|
Pipeline `print()` is teed for the job thread ([backend/job_log.py](backend/job_log.py)):
|
||||||
|
|
||||||
|
- Live: `GET /jobs/{id}` → `log_tail` (last 80 lines)
|
||||||
|
- Done/error/needs_review/finalization_error: same payload includes full `log`
|
||||||
|
- Disk: `backend/outputs/<job_id>/job.log` (survives restart; status registry does not)
|
||||||
|
- API: `GET /jobs/{id}/log` → `text/plain`
|
||||||
|
- UI: “Run log” panel updates while running; stays visible after finish/fail/review
|
||||||
|
- Failures append the **full traceback** to the log; each run starts with a header line (job id, mode, models, start time)
|
||||||
|
- `outputs/<job_id>/job.json` (written at start) carries email/mode/models so the disk fallback can rebuild a job after restart
|
||||||
|
|
||||||
|
## Vision vs text models
|
||||||
|
|
||||||
|
Two models, not one:
|
||||||
|
|
||||||
|
| Role | Config | Stages | Backend |
|
||||||
|
|------|--------|--------|---------|
|
||||||
|
| Vision | `MODEL` | Extract, conflict reason (images) | Always OpenRouter |
|
||||||
|
| Text | `TEXT_MODEL` (falls back to `MODEL`) | Sheet index, jurisdiction, normalize, cluster(LLM), QAQC, code, construct, validate, risk, RFI | OpenRouter, or local when hybrid |
|
||||||
|
|
||||||
|
UI: two dropdowns (with per-1M pricing) filled from `GET /models` ([backend/models.py](backend/models.py)), shown only for OpenRouter compute. Per-run picks go through `set_model_overrides(vision, text)` in [backend/llm.py](backend/llm.py): classic runs pass them as `run_pipeline` kwargs (runner clears in `finally`); agent runs set them module-level around `run_agent_pipeline`. UI picks beat per-call agent `AGENT_*_MODEL` args but **never name the hybrid local model** — local stays on `LOCAL_TEXT_MODEL`; the text pick only covers the cloud fallback.
|
||||||
|
|
||||||
|
## Where to change what
|
||||||
|
|
||||||
|
| Concern | File |
|
||||||
|
|---------|------|
|
||||||
|
| Prompts, vocab, conflict taxonomy | [backend/prompts.py](backend/prompts.py) |
|
||||||
|
| Env knobs | [backend/config.py](backend/config.py), [backend/.env.example](backend/.env.example) |
|
||||||
|
| HTTP API | [backend/main.py](backend/main.py) |
|
||||||
|
| UI (upload, models, live log, results) | [frontend/index.html](frontend/index.html) |
|
||||||
|
| CLI tuning loop | [cli/run_check.py](cli/run_check.py) |
|
||||||
|
| LLM client, cache, cost, model overrides | [backend/llm.py](backend/llm.py) |
|
||||||
|
| OpenRouter catalog + pricing + vision/text split | [backend/models.py](backend/models.py) |
|
||||||
|
| Agent-mode pipeline | [backend/agents/](backend/agents/) (`runner.py` entry; agents call `llm.call_json` with per-call model args) |
|
||||||
|
| Human review gate (queue, decisions, finalize) | [backend/review/](backend/review/) |
|
||||||
|
| Job registry + stdout tee log | [backend/jobs.py](backend/jobs.py), [backend/job_log.py](backend/job_log.py) |
|
||||||
|
| Stage helpers (prompt render, issue validate) | [backend/pipeline/_stage.py](backend/pipeline/_stage.py) |
|
||||||
|
| Code text corpus (Stage 7) | [backend/code_corpus/](backend/code_corpus/) |
|
||||||
|
| Hybrid local LLM helper | [scripts/setup_vllm.sh](scripts/setup_vllm.sh) |
|
||||||
|
|
||||||
|
Older prompt snapshot: `backend/prompts.py.v1`.
|
||||||
|
|
||||||
|
## Gotchas
|
||||||
|
|
||||||
|
1. **Prompt placeholders** — Use `str.replace` via `_stage.render`, never `str.format`. Prompts contain literal `{` JSON braces.
|
||||||
|
2. **Clustering** — Default is LLM (`CLUSTERER=llm`); empty LLM result falls back to deterministic. Deterministic clusters need ≥2 disciplines (or schedule-vs-plan); single-discipline “missing” gaps are a known limit.
|
||||||
|
3. **Jobs are in-memory** — Process restart clears job status; `outputs/<job_id>/` (report + `job.log` + `job.json` + `source.pdf`) still reload via disk fallback, including `needs_review` recovery.
|
||||||
|
4. **Dependency pin** — `httpx==0.27.2` with `openai==1.51.0`. httpx ≥0.28 breaks openai’s `proxies=` kwarg.
|
||||||
|
5. **Code corpus licensing** — Only `ada_2010.txt` is shipped. Do not paste IBC/IFC/IECC without a license (see `backend/code_corpus/README.md`).
|
||||||
|
6. **Dual assertion schema** — Newer `{sheet, objects[]}` is mapped to legacy `{assertions[]}` with `attribute`/`value` for older stages.
|
||||||
|
7. **Grounding guard** — Extractor drops objects whose numeric claims are not in `source_text` (graphical-only objects allowed).
|
||||||
|
8. **Module globals** — LLM cost counters, stdout tee, and model overrides are process-global; overlapping jobs can interleave (single-user tool assumption).
|
||||||
|
9. **Samples** — `samples/*.pdf` are gitignored; drop PDFs locally for CLI runs.
|
||||||
|
10. **README drift** — Treat README for setup/CI; treat this file + `runner.py` for pipeline truth. `prompts.py` header may still say some prompts are unwired — they are wired through the runner.
|
||||||
|
11. **Git identity** — This box has no `user.name` / `user.email`; commits need `GIT_AUTHOR_*` / `GIT_COMMITTER_*` env vars (do not `git config`). Remote push to Gitea works.
|
||||||
|
12. **No local Python deps on host** — App is meant to run in Docker; bare `python3` imports may miss `dotenv` / `httpx`. Prefer `docker compose`. For tests on this box: venv + requirements, but unpin Pillow (`Pillow>=11`) — 10.4.0 doesn't build on Python 3.14 (Docker uses 3.12, where the pin is fine).
|
||||||
|
13. **Agent mode constraints** — OpenRouter-only (hybrid disabled in UI and forced off server-side); review gate statuses are `needs_review → reviewing → finalizing → done` (`finalization_error` on finalize failure); only terminal states include the full `log` in polls.
|
||||||
|
|
||||||
|
## Quick start pointers
|
||||||
|
|
||||||
|
- Full setup: [README.md](README.md) (`docker compose up -d --build` → http://localhost:8099).
|
||||||
|
- Local CLI: `python cli/run_check.py samples/your_set.pdf --out out/your_set`.
|
||||||
|
- Prompt iteration: set `LLM_CACHE=true` in `backend/.env` so unchanged stages replay for free; clear with `rm -rf backend/.llm_cache`.
|
||||||
|
- Artifacts per job: `assertions.json`, `clusters.json`, per-stage JSON, `conflicts.json`, `report.md`, `job.log`, `job.json`, `source.pdf` under `backend/outputs/<job_id>/`.
|
||||||
|
- Tests: `python -m pytest tests/` (needs the deps from `requirements.txt` + `pytest`; on this box use a venv, see gotcha #12).
|
||||||
|
|
||||||
|
## Recent work (2026-08-02, agent-mode)
|
||||||
|
|
||||||
|
Merged `main` tip (`a6b0c8f` + `bf508bf`) into `agent-mode`, reconciling with this branch's own earlier implementations:
|
||||||
|
|
||||||
|
- **Two model dropdowns** — main's vision/text split ported onto this branch's priced catalog (`models.py`); pickers stay OpenRouter-compute-only, and UI picks never override `LOCAL_TEXT_MODEL` (main's hybrid footgun avoided).
|
||||||
|
- **Better run logs** — main's timestamped line-splitting tee, `log_tail` polls, terminal-state full log, and log-only disk recovery merged with this branch's header line, `job.json` metadata, and review-gate states. Failed runs now also append the traceback to `job.log`.
|
||||||
|
- `backend/models_catalog.py` (main's unpriced catalog) intentionally dropped in favor of `models.py`.
|
||||||
|
|
||||||
|
## Conflict categories (taxonomy)
|
||||||
|
|
||||||
|
Defined in `backend/prompts.py`: `dimensional_disagreement`, `elevation_disagreement`, `location_mismatch`, `missing_element`, `schedule_vs_plan_mismatch`, `detail_vs_plan_mismatch`, `tag_or_reference_inconsistency`, `spatial_clash`, `note_or_spec_contradiction`.
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
"""
|
||||||
|
job_log.py - Capture pipeline stdout/stderr into a per-job log.
|
||||||
|
|
||||||
|
The pipeline already prints stage progress via print(). For a reviewable
|
||||||
|
post-run log we tee those lines into memory + outputs/<job_id>/job.log
|
||||||
|
without rewriting every call site.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from typing import Callable, Iterator, List, Optional, TextIO
|
||||||
|
|
||||||
|
|
||||||
|
class _LineSplitter:
|
||||||
|
"""Accumulate write() chunks and emit complete lines."""
|
||||||
|
|
||||||
|
def __init__(self, on_line: Callable[[str], None]):
|
||||||
|
self._buf = ""
|
||||||
|
self._on_line = on_line
|
||||||
|
|
||||||
|
def write(self, s: str) -> None:
|
||||||
|
if not s:
|
||||||
|
return
|
||||||
|
self._buf += s
|
||||||
|
while "\n" in self._buf:
|
||||||
|
line, self._buf = self._buf.split("\n", 1)
|
||||||
|
# Strip trailing CR from Windows-ish streams; keep content intact.
|
||||||
|
self._on_line(line.rstrip("\r"))
|
||||||
|
|
||||||
|
def flush_remainder(self) -> None:
|
||||||
|
if self._buf:
|
||||||
|
self._on_line(self._buf.rstrip("\r"))
|
||||||
|
self._buf = ""
|
||||||
|
|
||||||
|
|
||||||
|
class _Tee:
|
||||||
|
"""Mirror writes to the original stream and a line callback."""
|
||||||
|
|
||||||
|
def __init__(self, stream: TextIO, on_line: Callable[[str], None]):
|
||||||
|
self._stream = stream
|
||||||
|
self._lines = _LineSplitter(on_line)
|
||||||
|
|
||||||
|
def write(self, s: str) -> int:
|
||||||
|
n = self._stream.write(s)
|
||||||
|
self._stream.flush()
|
||||||
|
self._lines.write(s)
|
||||||
|
return n
|
||||||
|
|
||||||
|
def flush(self) -> None:
|
||||||
|
self._stream.flush()
|
||||||
|
|
||||||
|
def flush_remainder(self) -> None:
|
||||||
|
self._lines.flush_remainder()
|
||||||
|
|
||||||
|
def __getattr__(self, name: str):
|
||||||
|
return getattr(self._stream, name)
|
||||||
|
|
||||||
|
|
||||||
|
def stamp_line(line: str, t: Optional[float] = None) -> str:
|
||||||
|
"""Prefix a log line with HH:MM:SS."""
|
||||||
|
ts = time.strftime("%H:%M:%S", time.localtime(t if t is not None else time.time()))
|
||||||
|
return f"[{ts}] {line}"
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def capture_stdio(on_line: Callable[[str], None]) -> Iterator[None]:
|
||||||
|
"""
|
||||||
|
Tee sys.stdout and sys.stderr into on_line(raw_line) for the duration.
|
||||||
|
|
||||||
|
Safe for the single-job-at-a-time usage of this app; overlapping jobs
|
||||||
|
would interleave (same limitation as the LLM cost counters).
|
||||||
|
"""
|
||||||
|
old_out, old_err = sys.stdout, sys.stderr
|
||||||
|
tee_out = _Tee(old_out, on_line)
|
||||||
|
tee_err = _Tee(old_err, on_line)
|
||||||
|
sys.stdout = tee_out # type: ignore[assignment]
|
||||||
|
sys.stderr = tee_err # type: ignore[assignment]
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
tee_out.flush_remainder()
|
||||||
|
tee_err.flush_remainder()
|
||||||
|
sys.stdout = old_out
|
||||||
|
sys.stderr = old_err
|
||||||
|
|
||||||
|
|
||||||
|
def read_log_file(path: str) -> List[str]:
|
||||||
|
try:
|
||||||
|
with open(path, encoding="utf-8") as f:
|
||||||
|
return [ln.rstrip("\n") for ln in f]
|
||||||
|
except OSError:
|
||||||
|
return []
|
||||||
+143
-78
@@ -9,61 +9,33 @@ for the completion email.
|
|||||||
State is in-memory (fine for a single-user tool); the report is also persisted
|
State is in-memory (fine for a single-user tool); the report is also persisted
|
||||||
to outputs/<job_id>/ so results survive a restart even though live status does
|
to outputs/<job_id>/ so results survive a restart even though live status does
|
||||||
not. No external queue/DB.
|
not. No external queue/DB.
|
||||||
|
|
||||||
|
A teed stdout/stderr log is kept in memory and written to outputs/<job_id>/job.log
|
||||||
|
so failed or suspicious runs can be reviewed after the fact.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import contextlib
|
|
||||||
import os
|
import os
|
||||||
import sys
|
|
||||||
import time
|
import time
|
||||||
|
import traceback
|
||||||
import uuid
|
import uuid
|
||||||
import shutil
|
import shutil
|
||||||
import threading
|
import threading
|
||||||
from typing import Dict, Optional
|
from typing import Dict, List, Optional
|
||||||
|
|
||||||
from backend import config
|
from backend import config
|
||||||
from backend import llm
|
from backend import llm
|
||||||
|
from backend.job_log import capture_stdio, read_log_file, stamp_line
|
||||||
from backend.agents.runner import run_agent_pipeline
|
from backend.agents.runner import run_agent_pipeline
|
||||||
from backend.pipeline.runner import run_pipeline
|
from backend.pipeline.runner import run_pipeline
|
||||||
from backend.email_sender import send_conflict_report, send_review_required
|
from backend.email_sender import send_conflict_report, send_review_required
|
||||||
|
|
||||||
_jobs: Dict[str, Dict] = {}
|
_jobs: Dict[str, Dict] = {}
|
||||||
_lock = threading.Lock()
|
_lock = threading.Lock()
|
||||||
|
_LOG_TAIL = 80
|
||||||
PIPELINE_MODES = {"classic", "agent"}
|
PIPELINE_MODES = {"classic", "agent"}
|
||||||
|
# States where the job will produce no more log output; polls get the full log.
|
||||||
|
_TERMINAL_STATES = {"done", "error", "needs_review", "finalization_error"}
|
||||||
class _Tee:
|
|
||||||
"""Write to both the real stream and the job log file."""
|
|
||||||
|
|
||||||
def __init__(self, stream, log_file) -> None:
|
|
||||||
self._stream = stream
|
|
||||||
self._log = log_file
|
|
||||||
|
|
||||||
def write(self, data):
|
|
||||||
self._stream.write(data)
|
|
||||||
self._log.write(data)
|
|
||||||
|
|
||||||
def flush(self):
|
|
||||||
self._stream.flush()
|
|
||||||
self._log.flush()
|
|
||||||
|
|
||||||
|
|
||||||
@contextlib.contextmanager
|
|
||||||
def _tee_log(log_path: str, header: str):
|
|
||||||
"""Mirror stdout/stderr into a per-job log file for the duration of a run.
|
|
||||||
|
|
||||||
sys.stdout is process-global, so two concurrent jobs would interleave in
|
|
||||||
each other's logs - acceptable for this single-user tool (same tradeoff as
|
|
||||||
the LLM cost globals in llm.py).
|
|
||||||
"""
|
|
||||||
with open(log_path, "a", encoding="utf-8") as log_file:
|
|
||||||
log_file.write(header + "\n")
|
|
||||||
real_out, real_err = sys.stdout, sys.stderr
|
|
||||||
sys.stdout, sys.stderr = _Tee(real_out, log_file), _Tee(real_err, log_file)
|
|
||||||
try:
|
|
||||||
yield
|
|
||||||
finally:
|
|
||||||
sys.stdout, sys.stderr = real_out, real_err
|
|
||||||
|
|
||||||
|
|
||||||
def _set(job_id: str, **fields) -> None:
|
def _set(job_id: str, **fields) -> None:
|
||||||
@@ -71,16 +43,39 @@ def _set(job_id: str, **fields) -> None:
|
|||||||
_jobs[job_id].update(fields)
|
_jobs[job_id].update(fields)
|
||||||
|
|
||||||
|
|
||||||
def create_job(pdf_path: str, source_filename: str, email: Optional[str] = None,
|
def _append_log(job_id: str, raw_line: str, log_path: str) -> None:
|
||||||
project_input: Optional[Dict] = None, text_local: bool = False,
|
"""Stamp, store, and append one captured stdout/stderr line."""
|
||||||
pipeline_mode: str = "classic", model: Optional[str] = None) -> str:
|
entry = stamp_line(raw_line)
|
||||||
|
with _lock:
|
||||||
|
job = _jobs.get(job_id)
|
||||||
|
if job is not None:
|
||||||
|
job.setdefault("log", []).append(entry)
|
||||||
|
try:
|
||||||
|
os.makedirs(os.path.dirname(log_path), exist_ok=True)
|
||||||
|
with open(log_path, "a", encoding="utf-8") as f:
|
||||||
|
f.write(entry + "\n")
|
||||||
|
except OSError:
|
||||||
|
# Don't fail the job over log I/O; avoid print() here — it would
|
||||||
|
# re-enter the stdio tee while a job is capturing.
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def create_job(
|
||||||
|
pdf_path: str,
|
||||||
|
source_filename: str,
|
||||||
|
email: Optional[str] = None,
|
||||||
|
project_input: Optional[Dict] = None,
|
||||||
|
text_local: bool = False,
|
||||||
|
pipeline_mode: str = "classic",
|
||||||
|
vision_model: Optional[str] = None,
|
||||||
|
text_model: Optional[str] = None,
|
||||||
|
) -> str:
|
||||||
"""Register a job and kick off its background thread. Returns the job_id."""
|
"""Register a job and kick off its background thread. Returns the job_id."""
|
||||||
pipeline_mode = pipeline_mode.strip().lower()
|
pipeline_mode = pipeline_mode.strip().lower()
|
||||||
if pipeline_mode not in PIPELINE_MODES:
|
if pipeline_mode not in PIPELINE_MODES:
|
||||||
raise ValueError(f"Unsupported pipeline mode: {pipeline_mode!r}")
|
raise ValueError(f"Unsupported pipeline mode: {pipeline_mode!r}")
|
||||||
# Agent mode v1 is OpenRouter-only.
|
# Agent mode v1 is OpenRouter-only.
|
||||||
text_local = bool(text_local and pipeline_mode == "classic")
|
text_local = bool(text_local and pipeline_mode == "classic")
|
||||||
model = (model or "").strip() or None
|
|
||||||
job_id = uuid.uuid4().hex[:12]
|
job_id = uuid.uuid4().hex[:12]
|
||||||
with _lock:
|
with _lock:
|
||||||
_jobs[job_id] = {
|
_jobs[job_id] = {
|
||||||
@@ -91,35 +86,61 @@ def create_job(pdf_path: str, source_filename: str, email: Optional[str] = None,
|
|||||||
"project_input": project_input or {},
|
"project_input": project_input or {},
|
||||||
"text_local": text_local,
|
"text_local": text_local,
|
||||||
"pipeline_mode": pipeline_mode,
|
"pipeline_mode": pipeline_mode,
|
||||||
"model": model,
|
"vision_model": (vision_model or "").strip() or None,
|
||||||
|
"text_model": (text_model or "").strip() or None,
|
||||||
"stage": None,
|
"stage": None,
|
||||||
"created_at": time.time(),
|
"created_at": time.time(),
|
||||||
"finished_at": None,
|
"finished_at": None,
|
||||||
"report": None,
|
"report": None,
|
||||||
"error": None,
|
"error": None,
|
||||||
|
"log": [],
|
||||||
}
|
}
|
||||||
threading.Thread(target=_run, args=(
|
threading.Thread(
|
||||||
job_id, pdf_path, project_input, text_local, pipeline_mode, model,
|
target=_run,
|
||||||
),
|
args=(job_id, pdf_path, project_input, text_local, pipeline_mode,
|
||||||
daemon=True).start()
|
vision_model, text_model),
|
||||||
|
daemon=True,
|
||||||
|
).start()
|
||||||
return job_id
|
return job_id
|
||||||
|
|
||||||
|
|
||||||
def _run(job_id: str, pdf_path: str, project_input: Optional[Dict] = None,
|
def _run(
|
||||||
text_local: bool = False, pipeline_mode: str = "classic",
|
job_id: str,
|
||||||
model: Optional[str] = None) -> None:
|
pdf_path: str,
|
||||||
|
project_input: Optional[Dict] = None,
|
||||||
|
text_local: bool = False,
|
||||||
|
pipeline_mode: str = "classic",
|
||||||
|
vision_model: Optional[str] = None,
|
||||||
|
text_model: Optional[str] = None,
|
||||||
|
) -> None:
|
||||||
out_dir = os.path.join(config.OUTPUT_DIR, job_id)
|
out_dir = os.path.join(config.OUTPUT_DIR, job_id)
|
||||||
|
log_path = os.path.join(out_dir, "job.log")
|
||||||
try:
|
try:
|
||||||
_set(job_id, status="running")
|
_set(job_id, status="running")
|
||||||
# Keep a copy of the source PDF so its sheets can be viewed later.
|
|
||||||
os.makedirs(out_dir, exist_ok=True)
|
os.makedirs(out_dir, exist_ok=True)
|
||||||
|
# Truncate any leftover log if job_id somehow collided (shouldn't).
|
||||||
|
with open(log_path, "w", encoding="utf-8"):
|
||||||
|
pass
|
||||||
header = (f"=== Job {job_id} | {pipeline_mode} | {_jobs[job_id].get('source')} | "
|
header = (f"=== Job {job_id} | {pipeline_mode} | {_jobs[job_id].get('source')} | "
|
||||||
f"model={model or 'default'} | "
|
f"vision={vision_model or 'default'} text={text_model or 'default'} | "
|
||||||
f"started {time.strftime('%Y-%m-%d %H:%M:%S %Z', time.gmtime())} UTC ===")
|
f"started {time.strftime('%Y-%m-%d %H:%M:%S %Z', time.gmtime())} UTC ===")
|
||||||
with _tee_log(os.path.join(out_dir, "job.log"), header):
|
_append_log(job_id, header, log_path)
|
||||||
|
|
||||||
|
def on_line(raw: str) -> None:
|
||||||
|
_append_log(job_id, raw, log_path)
|
||||||
|
|
||||||
|
with capture_stdio(on_line):
|
||||||
_run_pipeline(job_id, pdf_path, out_dir, project_input, text_local,
|
_run_pipeline(job_id, pdf_path, out_dir, project_input, text_local,
|
||||||
pipeline_mode, model)
|
pipeline_mode, vision_model, text_model)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
# Land the failure AND its traceback in the job log so failed runs can
|
||||||
|
# be diagnosed from the log alone (the stdio tee is already torn down).
|
||||||
|
try:
|
||||||
|
_append_log(job_id, f"[Jobs] Job {job_id} failed: {e}", log_path)
|
||||||
|
for ln in traceback.format_exc().rstrip().splitlines():
|
||||||
|
_append_log(job_id, ln, log_path)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
print(f"[Jobs] Job {job_id} failed: {e}")
|
print(f"[Jobs] Job {job_id} failed: {e}")
|
||||||
_set(job_id, status="error", error=str(e), finished_at=time.time())
|
_set(job_id, status="error", error=str(e), finished_at=time.time())
|
||||||
_notify_error(job_id)
|
_notify_error(job_id)
|
||||||
@@ -132,7 +153,8 @@ def _run(job_id: str, pdf_path: str, project_input: Optional[Dict] = None,
|
|||||||
|
|
||||||
def _run_pipeline(job_id: str, pdf_path: str, out_dir: str,
|
def _run_pipeline(job_id: str, pdf_path: str, out_dir: str,
|
||||||
project_input: Optional[Dict], text_local: bool,
|
project_input: Optional[Dict], text_local: bool,
|
||||||
pipeline_mode: str, model: Optional[str]) -> None:
|
pipeline_mode: str, vision_model: Optional[str],
|
||||||
|
text_model: Optional[str]) -> None:
|
||||||
"""The body of a job run; executes inside the job's tee'd log capture."""
|
"""The body of a job run; executes inside the job's tee'd log capture."""
|
||||||
# Persist minimal job metadata so the disk fallback in get_job can
|
# Persist minimal job metadata so the disk fallback in get_job can
|
||||||
# recover the recipient email / pipeline mode after a server restart
|
# recover the recipient email / pipeline mode after a server restart
|
||||||
@@ -143,7 +165,10 @@ def _run_pipeline(job_id: str, pdf_path: str, out_dir: str,
|
|||||||
"email": _jobs[job_id].get("email"),
|
"email": _jobs[job_id].get("email"),
|
||||||
"pipeline_mode": pipeline_mode,
|
"pipeline_mode": pipeline_mode,
|
||||||
"source": _jobs[job_id].get("source"),
|
"source": _jobs[job_id].get("source"),
|
||||||
|
"vision_model": vision_model,
|
||||||
|
"text_model": text_model,
|
||||||
}, f, indent=2)
|
}, f, indent=2)
|
||||||
|
# Keep a copy of the source PDF so its sheets can be viewed later.
|
||||||
shutil.copy2(pdf_path, os.path.join(out_dir, "source.pdf"))
|
shutil.copy2(pdf_path, os.path.join(out_dir, "source.pdf"))
|
||||||
runner = run_agent_pipeline if pipeline_mode == "agent" else run_pipeline
|
runner = run_agent_pipeline if pipeline_mode == "agent" else run_pipeline
|
||||||
runner_kwargs = {
|
runner_kwargs = {
|
||||||
@@ -153,17 +178,22 @@ def _run_pipeline(job_id: str, pdf_path: str, out_dir: str,
|
|||||||
"source_name": _jobs[job_id].get("source"),
|
"source_name": _jobs[job_id].get("source"),
|
||||||
}
|
}
|
||||||
if pipeline_mode == "classic":
|
if pipeline_mode == "classic":
|
||||||
|
# run_pipeline takes the picks as params and clears them in finally.
|
||||||
runner_kwargs["text_local"] = text_local
|
runner_kwargs["text_local"] = text_local
|
||||||
|
runner_kwargs["vision_model"] = vision_model
|
||||||
|
runner_kwargs["text_model"] = text_model
|
||||||
else:
|
else:
|
||||||
runner_kwargs["require_review"] = config.AGENT_REQUIRE_REVIEW
|
runner_kwargs["require_review"] = config.AGENT_REQUIRE_REVIEW
|
||||||
if model:
|
# The agent runner has no override params; set them module-level.
|
||||||
print(f"[Jobs] Model override for this run: {model}")
|
if vision_model or text_model:
|
||||||
llm.set_model_override(model)
|
print(f"[Jobs] Model overrides for this run: "
|
||||||
|
f"vision={vision_model or '(default)'} text={text_model or '(default)'}")
|
||||||
|
llm.set_model_overrides(vision_model, text_model)
|
||||||
try:
|
try:
|
||||||
report = runner(pdf_path, **runner_kwargs)
|
report = runner(pdf_path, **runner_kwargs)
|
||||||
finally:
|
finally:
|
||||||
if model:
|
if pipeline_mode == "agent":
|
||||||
llm.set_model_override(None)
|
llm.set_model_overrides(None, None)
|
||||||
report.setdefault("summary", {})["pipeline_mode"] = pipeline_mode
|
report.setdefault("summary", {})["pipeline_mode"] = pipeline_mode
|
||||||
if report["summary"].get("agent_status") == "needs_review":
|
if report["summary"].get("agent_status") == "needs_review":
|
||||||
# Human-review gate: hold the job, don't email the unreviewed report.
|
# Human-review gate: hold the job, don't email the unreviewed report.
|
||||||
@@ -197,11 +227,6 @@ def _notify_error(job_id: str) -> None:
|
|||||||
email = job.get("email")
|
email = job.get("email")
|
||||||
if not email:
|
if not email:
|
||||||
return
|
return
|
||||||
# Reuse the report mailer with a minimal error-shaped payload.
|
|
||||||
err_report = {
|
|
||||||
"source": job.get("source", ""),
|
|
||||||
"summary": {"conflicts_found": 0, "by_severity": {}, "disciplines": []},
|
|
||||||
}
|
|
||||||
try:
|
try:
|
||||||
from backend.email_sender import _smtp_ready, _send
|
from backend.email_sender import _smtp_ready, _send
|
||||||
from email.message import EmailMessage
|
from email.message import EmailMessage
|
||||||
@@ -216,6 +241,7 @@ def _notify_error(job_id: str) -> None:
|
|||||||
"Your conflict check did not complete.\n\n"
|
"Your conflict check did not complete.\n\n"
|
||||||
f"Drawing set: {job.get('source','')}\n"
|
f"Drawing set: {job.get('source','')}\n"
|
||||||
f"Error: {job.get('error','unknown')}\n\n"
|
f"Error: {job.get('error','unknown')}\n\n"
|
||||||
|
f"Review the run log at: {config.APP_BASE_URL.rstrip('/')}/?job={job_id}\n\n"
|
||||||
"Generated by Conflict Checker"
|
"Generated by Conflict Checker"
|
||||||
)
|
)
|
||||||
_send(msg)
|
_send(msg)
|
||||||
@@ -223,28 +249,63 @@ def _notify_error(job_id: str) -> None:
|
|||||||
print(f"[Email] Failed to send error notice: {e}")
|
print(f"[Email] Failed to send error notice: {e}")
|
||||||
|
|
||||||
|
|
||||||
|
def _log_from_disk(job_id: str) -> List[str]:
|
||||||
|
return read_log_file(os.path.join(config.OUTPUT_DIR, job_id, "job.log"))
|
||||||
|
|
||||||
|
|
||||||
|
def get_job_log(job_id: str) -> Optional[List[str]]:
|
||||||
|
"""Full job log lines, from memory or disk. None if job unknown."""
|
||||||
|
with _lock:
|
||||||
|
job = _jobs.get(job_id)
|
||||||
|
if job is not None:
|
||||||
|
return list(job.get("log") or [])
|
||||||
|
|
||||||
|
log = _log_from_disk(job_id)
|
||||||
|
# Job exists on disk if we have a log or a report artifact.
|
||||||
|
report_path = os.path.join(config.OUTPUT_DIR, job_id, "conflicts.json")
|
||||||
|
if log or os.path.isfile(report_path):
|
||||||
|
return log
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def get_job(job_id: str) -> Optional[Dict]:
|
def get_job(job_id: str) -> Optional[Dict]:
|
||||||
"""Public job view. Includes the full report only when done.
|
"""Public job view. Includes the full report only when done.
|
||||||
|
|
||||||
Falls back to the on-disk conflicts.json when the job isn't in the
|
Falls back to the on-disk artifacts (conflicts.json / job.log) when the
|
||||||
in-memory registry (e.g. after a server restart).
|
job isn't in the in-memory registry (e.g. after a server restart).
|
||||||
"""
|
"""
|
||||||
with _lock:
|
with _lock:
|
||||||
job = _jobs.get(job_id)
|
job = _jobs.get(job_id)
|
||||||
if job:
|
if job:
|
||||||
return dict(job)
|
out = dict(job)
|
||||||
|
log = list(job.get("log") or [])
|
||||||
|
out["log_tail"] = log[-_LOG_TAIL:]
|
||||||
|
# Full log on terminal states so the UI can show it without a
|
||||||
|
# second fetch; keep polls light while running.
|
||||||
|
if out.get("status") in _TERMINAL_STATES:
|
||||||
|
out["log"] = log
|
||||||
|
else:
|
||||||
|
out.pop("log", None)
|
||||||
|
return out
|
||||||
|
|
||||||
# Try loading from disk
|
# Try loading from disk
|
||||||
report_path = os.path.join(config.OUTPUT_DIR, job_id, "conflicts.json")
|
report_path = os.path.join(config.OUTPUT_DIR, job_id, "conflicts.json")
|
||||||
if not os.path.isfile(report_path):
|
log = _log_from_disk(job_id)
|
||||||
|
if not os.path.isfile(report_path) and not log:
|
||||||
return None
|
return None
|
||||||
try:
|
try:
|
||||||
with open(report_path, encoding="utf-8") as f:
|
report = None
|
||||||
report = json.load(f)
|
if os.path.isfile(report_path):
|
||||||
summary = report.get("summary", {})
|
with open(report_path, encoding="utf-8") as f:
|
||||||
# Recover the job's real state: a job that stopped at the review gate
|
report = json.load(f)
|
||||||
# must come back as needs_review (not done) or it can never finalize.
|
summary = (report or {}).get("summary", {})
|
||||||
status = "needs_review" if summary.get("agent_status") == "needs_review" else "done"
|
if report is None:
|
||||||
|
# Crashed before writing a report; the log is the only artifact.
|
||||||
|
status = "error"
|
||||||
|
else:
|
||||||
|
# Recover the job's real state: a job that stopped at the review gate
|
||||||
|
# must come back as needs_review (not done) or it can never finalize.
|
||||||
|
status = "needs_review" if summary.get("agent_status") == "needs_review" else "done"
|
||||||
# job.json (written at job start) carries the recipient email and
|
# job.json (written at job start) carries the recipient email and
|
||||||
# pipeline mode so the final notification still fires after a restart.
|
# pipeline mode so the final notification still fires after a restart.
|
||||||
# Missing/corrupt job.json degrades to the previous derivations.
|
# Missing/corrupt job.json degrades to the previous derivations.
|
||||||
@@ -261,16 +322,20 @@ def get_job(job_id: str) -> Optional[Dict]:
|
|||||||
job = {
|
job = {
|
||||||
"job_id": job_id,
|
"job_id": job_id,
|
||||||
"status": status,
|
"status": status,
|
||||||
"source": meta.get("source") or report.get("source", os.path.basename(report_path)),
|
"source": meta.get("source") or (report or {}).get("source", os.path.basename(report_path)),
|
||||||
"email": meta.get("email"),
|
"email": meta.get("email"),
|
||||||
"project_input": report.get("project_input", {}),
|
"project_input": (report or {}).get("project_input", {}),
|
||||||
"text_local": summary.get("text_backend") == "local",
|
"text_local": summary.get("text_backend") == "local",
|
||||||
"pipeline_mode": meta.get("pipeline_mode") or summary.get("pipeline_mode", "classic"),
|
"pipeline_mode": meta.get("pipeline_mode") or summary.get("pipeline_mode", "classic"),
|
||||||
|
"vision_model": meta.get("vision_model"),
|
||||||
|
"text_model": meta.get("text_model"),
|
||||||
"stage": None,
|
"stage": None,
|
||||||
"created_at": os.path.getmtime(source_pdf) if os.path.isfile(source_pdf) else None,
|
"created_at": os.path.getmtime(source_pdf) if os.path.isfile(source_pdf) else None,
|
||||||
"finished_at": os.path.getmtime(report_path),
|
"finished_at": os.path.getmtime(report_path) if os.path.isfile(report_path) else None,
|
||||||
"report": report,
|
"report": report,
|
||||||
"error": None,
|
"error": None if report is not None else "Report missing; see job log",
|
||||||
|
"log": log,
|
||||||
|
"log_tail": log[-_LOG_TAIL:],
|
||||||
}
|
}
|
||||||
# Hydrate the in-memory registry so _set(...) transitions (reviewing,
|
# Hydrate the in-memory registry so _set(...) transitions (reviewing,
|
||||||
# finalizing, done) work for restart-recovered jobs.
|
# finalizing, done) work for restart-recovered jobs.
|
||||||
|
|||||||
+22
-12
@@ -23,16 +23,13 @@ _clients: Dict[str, OpenAI] = {}
|
|||||||
# call_json when routing a no-image (text) call. Module-global mirrors the
|
# call_json when routing a no-image (text) call. Module-global mirrors the
|
||||||
# set_stage/cost pattern (single-user tool).
|
# set_stage/cost pattern (single-user tool).
|
||||||
_text_local = False
|
_text_local = False
|
||||||
|
# Per-job model overrides (user picked models in the UI). Same module-global
|
||||||
# Per-job model override (user picked a model in the UI). Same module-global
|
|
||||||
# pattern: set by the job runner before the pipeline starts, cleared after.
|
# pattern: set by the job runner before the pipeline starts, cleared after.
|
||||||
_model_override: Optional[str] = None
|
# Vision applies to image calls, text to no-image calls on OpenRouter (and to
|
||||||
|
# the local->cloud fallback). The LOCAL endpoint's model name is never taken
|
||||||
|
# from these overrides - hybrid local keeps LOCAL_TEXT_MODEL.
|
||||||
def set_model_override(model: Optional[str]) -> None:
|
_vision_model_override: Optional[str] = None
|
||||||
"""Override the model for all OpenRouter calls (vision + text), or None to clear."""
|
_text_model_override: Optional[str] = None
|
||||||
global _model_override
|
|
||||||
_model_override = (model or "").strip() or None
|
|
||||||
|
|
||||||
|
|
||||||
def set_text_backend(local: bool) -> None:
|
def set_text_backend(local: bool) -> None:
|
||||||
@@ -40,6 +37,13 @@ def set_text_backend(local: bool) -> None:
|
|||||||
global _text_local
|
global _text_local
|
||||||
_text_local = bool(local)
|
_text_local = bool(local)
|
||||||
|
|
||||||
|
|
||||||
|
def set_model_overrides(vision: Optional[str] = None, text: Optional[str] = None) -> None:
|
||||||
|
"""Per-run OpenRouter vision/text model picks. None/blank clears to defaults."""
|
||||||
|
global _vision_model_override, _text_model_override
|
||||||
|
_vision_model_override = (vision or "").strip() or None
|
||||||
|
_text_model_override = (text or "").strip() or None
|
||||||
|
|
||||||
# --- per-job cost accounting -------------------------------------------------
|
# --- per-job cost accounting -------------------------------------------------
|
||||||
# OpenRouter returns the real USD cost of each call when we request usage
|
# OpenRouter returns the real USD cost of each call when we request usage
|
||||||
# accounting. We accumulate it in a module-level counter; the runner resets it
|
# accounting. We accumulate it in a module-level counter; the runner resets it
|
||||||
@@ -177,17 +181,22 @@ def _resolve_backend(has_images: bool, model_override: Optional[str]) -> Dict[st
|
|||||||
return {
|
return {
|
||||||
"base_url": config.LOCAL_BASE_URL,
|
"base_url": config.LOCAL_BASE_URL,
|
||||||
"api_key": config.LOCAL_API_KEY,
|
"api_key": config.LOCAL_API_KEY,
|
||||||
|
# Local model name comes from per-call args or LOCAL_TEXT_MODEL —
|
||||||
|
# never the UI's OpenRouter picks, which a local server won't serve.
|
||||||
"model": model_override or config.LOCAL_TEXT_MODEL or config.TEXT_MODEL,
|
"model": model_override or config.LOCAL_TEXT_MODEL or config.TEXT_MODEL,
|
||||||
"usage": False, # local has no OpenRouter usage accounting
|
"usage": False, # local has no OpenRouter usage accounting
|
||||||
"local": True,
|
"local": True,
|
||||||
}
|
}
|
||||||
# Vision, or text-on-OpenRouter (default / fallback). A per-job override
|
# Vision, or text-on-OpenRouter (default / fallback). A per-job override
|
||||||
# (user's UI model pick) wins over per-call and env defaults.
|
# (user's UI model pick) wins over per-call and env defaults.
|
||||||
default_model = config.MODEL if has_images else config.TEXT_MODEL
|
if has_images:
|
||||||
|
model = _vision_model_override or model_override or config.MODEL
|
||||||
|
else:
|
||||||
|
model = _text_model_override or model_override or config.TEXT_MODEL
|
||||||
return {
|
return {
|
||||||
"base_url": config.AI_BASE_URL,
|
"base_url": config.AI_BASE_URL,
|
||||||
"api_key": config.AI_API_KEY,
|
"api_key": config.AI_API_KEY,
|
||||||
"model": _model_override or model_override or default_model,
|
"model": model,
|
||||||
"usage": True,
|
"usage": True,
|
||||||
"local": False,
|
"local": False,
|
||||||
}
|
}
|
||||||
@@ -362,7 +371,8 @@ def call_json(
|
|||||||
_models["text_local"].add(be["model"])
|
_models["text_local"].add(be["model"])
|
||||||
_models["fallback_count"] += 1
|
_models["fallback_count"] += 1
|
||||||
be = {"base_url": config.AI_BASE_URL, "api_key": config.AI_API_KEY,
|
be = {"base_url": config.AI_BASE_URL, "api_key": config.AI_API_KEY,
|
||||||
"model": config.TEXT_MODEL, "usage": True, "local": False}
|
"model": _text_model_override or config.TEXT_MODEL,
|
||||||
|
"usage": True, "local": False}
|
||||||
cache_key = None # don't cache fallback under the local-model key
|
cache_key = None # don't cache fallback under the local-model key
|
||||||
fell_back = True
|
fell_back = True
|
||||||
continue
|
continue
|
||||||
|
|||||||
+15
-5
@@ -35,6 +35,7 @@ _FRONTEND_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "
|
|||||||
@app.get("/health")
|
@app.get("/health")
|
||||||
def health():
|
def health():
|
||||||
return {"status": "ok", "model": config.MODEL,
|
return {"status": "ok", "model": config.MODEL,
|
||||||
|
"text_model": config.TEXT_MODEL,
|
||||||
"version": config.APP_VERSION,
|
"version": config.APP_VERSION,
|
||||||
"build": config.APP_BUILD,
|
"build": config.APP_BUILD,
|
||||||
"key_configured": bool(config.AI_API_KEY),
|
"key_configured": bool(config.AI_API_KEY),
|
||||||
@@ -43,13 +44,15 @@ def health():
|
|||||||
|
|
||||||
@app.get("/models")
|
@app.get("/models")
|
||||||
def list_models():
|
def list_models():
|
||||||
"""Available OpenRouter models with per-1M-token pricing for the UI picker."""
|
"""Vision/text OpenRouter model lists with pricing for the UI dropdowns."""
|
||||||
from backend.models import fetch_models
|
from backend.models import fetch_models, split_vision_text
|
||||||
models = fetch_models()
|
models = fetch_models()
|
||||||
if models is None:
|
if models is None:
|
||||||
raise HTTPException(status_code=502,
|
raise HTTPException(status_code=502,
|
||||||
detail="Could not fetch the model list from OpenRouter")
|
detail="Could not fetch the model list from OpenRouter")
|
||||||
return {"models": models, "default": config.MODEL, "default_text": config.TEXT_MODEL}
|
vision, text = split_vision_text(models)
|
||||||
|
return {"vision": vision, "text": text,
|
||||||
|
"defaults": {"vision": config.MODEL, "text": config.TEXT_MODEL}}
|
||||||
|
|
||||||
|
|
||||||
@app.get("/jobs/{job_id}/log")
|
@app.get("/jobs/{job_id}/log")
|
||||||
@@ -72,7 +75,8 @@ async def check(
|
|||||||
work_type: Optional[str] = Form(None),
|
work_type: Optional[str] = Form(None),
|
||||||
text_local: bool = Form(False),
|
text_local: bool = Form(False),
|
||||||
pipeline_mode: str = Form("classic"),
|
pipeline_mode: str = Form("classic"),
|
||||||
model: Optional[str] = Form(None),
|
vision_model: Optional[str] = Form(None),
|
||||||
|
text_model: Optional[str] = Form(None),
|
||||||
):
|
):
|
||||||
"""
|
"""
|
||||||
Accept a PDF, start a background conflict check, and return a job_id
|
Accept a PDF, start a background conflict check, and return a job_id
|
||||||
@@ -81,6 +85,9 @@ async def check(
|
|||||||
Optional intake fields (project_name/address/occupancy/work_type) feed the
|
Optional intake fields (project_name/address/occupancy/work_type) feed the
|
||||||
Stage 0 jurisdiction profile; anything left blank is derived from the cover
|
Stage 0 jurisdiction profile; anything left blank is derived from the cover
|
||||||
sheet.
|
sheet.
|
||||||
|
|
||||||
|
vision_model / text_model override the configured defaults for this run
|
||||||
|
(vision always OpenRouter; text follows the OpenRouter vs hybrid choice).
|
||||||
"""
|
"""
|
||||||
if not file.filename.lower().endswith(".pdf"):
|
if not file.filename.lower().endswith(".pdf"):
|
||||||
raise HTTPException(status_code=400, detail="Please upload a PDF.")
|
raise HTTPException(status_code=400, detail="Please upload a PDF.")
|
||||||
@@ -106,9 +113,12 @@ async def check(
|
|||||||
}.items()
|
}.items()
|
||||||
if v and v.strip()
|
if v and v.strip()
|
||||||
}
|
}
|
||||||
|
v_model = (vision_model or "").strip() or None
|
||||||
|
t_model = (text_model or "").strip() or None
|
||||||
job_id = create_job(tmp_path, source_filename=file.filename, email=email,
|
job_id = create_job(tmp_path, source_filename=file.filename, email=email,
|
||||||
project_input=project_input, text_local=text_local,
|
project_input=project_input, text_local=text_local,
|
||||||
pipeline_mode=pipeline_mode, model=model)
|
pipeline_mode=pipeline_mode, vision_model=v_model,
|
||||||
|
text_model=t_model)
|
||||||
return JSONResponse({
|
return JSONResponse({
|
||||||
"job_id": job_id,
|
"job_id": job_id,
|
||||||
"status": "queued",
|
"status": "queued",
|
||||||
|
|||||||
+27
-2
@@ -3,11 +3,13 @@ models.py - Fetch the available OpenRouter model list with pricing (cached).
|
|||||||
|
|
||||||
The /models endpoint is public (no API key needed). Results are normalized to
|
The /models endpoint is public (no API key needed). Results are normalized to
|
||||||
per-1M-token USD costs for display and cached in memory for an hour; callers
|
per-1M-token USD costs for display and cached in memory for an hour; callers
|
||||||
degrade gracefully when OpenRouter is unreachable.
|
degrade gracefully when OpenRouter is unreachable. Each entry also carries a
|
||||||
|
vision flag (accepts image input) so the UI can offer separate vision/text
|
||||||
|
model dropdowns.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import time
|
import time
|
||||||
from typing import List, Optional
|
from typing import List, Optional, Tuple
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
@@ -25,6 +27,18 @@ def _per_mtok(rate) -> float:
|
|||||||
return 0.0
|
return 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def _is_vision(item: dict) -> bool:
|
||||||
|
"""True when the model accepts image input and produces text output."""
|
||||||
|
arch = item.get("architecture") or {}
|
||||||
|
inputs = arch.get("input_modalities") or []
|
||||||
|
outputs = arch.get("output_modalities") or []
|
||||||
|
# Legacy string form: "text+image->text"
|
||||||
|
modality = (arch.get("modality") or "").lower()
|
||||||
|
has_image_in = ("image" in inputs) or ("image" in modality.split("->")[0])
|
||||||
|
has_text_out = ("text" in outputs) or ("->text" in modality) or (not outputs and not modality)
|
||||||
|
return has_image_in and has_text_out
|
||||||
|
|
||||||
|
|
||||||
def _fetch_openrouter_models() -> Optional[List[dict]]:
|
def _fetch_openrouter_models() -> Optional[List[dict]]:
|
||||||
"""Raw GET of the OpenRouter model list; None on any failure."""
|
"""Raw GET of the OpenRouter model list; None on any failure."""
|
||||||
try:
|
try:
|
||||||
@@ -55,6 +69,7 @@ def fetch_models(force: bool = False) -> Optional[List[dict]]:
|
|||||||
"prompt_usd_per_mtok": _per_mtok((item.get("pricing") or {}).get("prompt")),
|
"prompt_usd_per_mtok": _per_mtok((item.get("pricing") or {}).get("prompt")),
|
||||||
"completion_usd_per_mtok": _per_mtok((item.get("pricing") or {}).get("completion")),
|
"completion_usd_per_mtok": _per_mtok((item.get("pricing") or {}).get("completion")),
|
||||||
"context_length": item.get("context_length"),
|
"context_length": item.get("context_length"),
|
||||||
|
"vision": _is_vision(item),
|
||||||
}
|
}
|
||||||
for item in data
|
for item in data
|
||||||
if item.get("id")
|
if item.get("id")
|
||||||
@@ -63,3 +78,13 @@ def fetch_models(force: bool = False) -> Optional[List[dict]]:
|
|||||||
_cache["models"] = models
|
_cache["models"] = models
|
||||||
_cache["at"] = time.time()
|
_cache["at"] = time.time()
|
||||||
return models
|
return models
|
||||||
|
|
||||||
|
|
||||||
|
def split_vision_text(models: List[dict]) -> Tuple[List[dict], List[dict]]:
|
||||||
|
"""Partition the normalized catalog into (vision, text) lists for the UI.
|
||||||
|
|
||||||
|
Every catalog model takes text in/out, so vision models appear in both
|
||||||
|
lists (same dicts, pricing included).
|
||||||
|
"""
|
||||||
|
vision = [m for m in models if m.get("vision")]
|
||||||
|
return vision, list(models)
|
||||||
|
|||||||
@@ -42,7 +42,9 @@ from backend.pipeline.risk import score_and_prioritize
|
|||||||
from backend.pipeline.rfi import generate_rfis
|
from backend.pipeline.rfi import generate_rfis
|
||||||
from backend.pipeline.report import build_report, to_markdown
|
from backend.pipeline.report import build_report, to_markdown
|
||||||
from backend.pipeline._stage import validate_issue
|
from backend.pipeline._stage import validate_issue
|
||||||
from backend.llm import reset_cost, get_cost, set_stage, set_text_backend
|
from backend.llm import (
|
||||||
|
reset_cost, get_cost, set_stage, set_text_backend, set_model_overrides,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def run_pipeline(
|
def run_pipeline(
|
||||||
@@ -52,6 +54,8 @@ def run_pipeline(
|
|||||||
project_input: Optional[Dict] = None,
|
project_input: Optional[Dict] = None,
|
||||||
source_name: Optional[str] = None,
|
source_name: Optional[str] = None,
|
||||||
text_local: bool = False,
|
text_local: bool = False,
|
||||||
|
vision_model: Optional[str] = None,
|
||||||
|
text_model: Optional[str] = None,
|
||||||
) -> Dict:
|
) -> Dict:
|
||||||
"""
|
"""
|
||||||
Run the full QAQC pipeline on one PDF and return the report dict.
|
Run the full QAQC pipeline on one PDF and return the report dict.
|
||||||
@@ -59,6 +63,9 @@ def run_pipeline(
|
|||||||
project_input: optional intake fields (project_name, address, occupancy,
|
project_input: optional intake fields (project_name, address, occupancy,
|
||||||
work_type). Cover-sheet-derived values fill any gaps; intake fields win.
|
work_type). Cover-sheet-derived values fill any gaps; intake fields win.
|
||||||
|
|
||||||
|
vision_model / text_model: optional per-run OpenRouter (or local text)
|
||||||
|
model overrides from the UI. Blank/None keeps config defaults.
|
||||||
|
|
||||||
If out_dir is given, writes conflicts.json, report.md, and the intermediate
|
If out_dir is given, writes conflicts.json, report.md, and the intermediate
|
||||||
artifacts (assertions.json, clusters.json, and one json per QAQC stage).
|
artifacts (assertions.json, clusters.json, and one json per QAQC stage).
|
||||||
"""
|
"""
|
||||||
@@ -70,7 +77,29 @@ def run_pipeline(
|
|||||||
|
|
||||||
reset_cost()
|
reset_cost()
|
||||||
set_text_backend(text_local)
|
set_text_backend(text_local)
|
||||||
|
set_model_overrides(vision_model, text_model)
|
||||||
|
if vision_model or text_model:
|
||||||
|
print(f"[Runner] model overrides: vision={vision_model or '(default)'} "
|
||||||
|
f"text={text_model or '(default)'}")
|
||||||
|
|
||||||
|
try:
|
||||||
|
return _run_stages(
|
||||||
|
pdf_path, out_dir, stage, project_input, source_name, text_local,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
# Don't leak per-run picks into a later overlapping/CLI call.
|
||||||
|
set_model_overrides(None, None)
|
||||||
|
set_text_backend(False)
|
||||||
|
|
||||||
|
|
||||||
|
def _run_stages(
|
||||||
|
pdf_path: str,
|
||||||
|
out_dir: Optional[str],
|
||||||
|
stage: Callable[[str], None],
|
||||||
|
project_input: Optional[Dict],
|
||||||
|
source_name: Optional[str],
|
||||||
|
text_local: bool,
|
||||||
|
) -> Dict:
|
||||||
stage("PDF -> images")
|
stage("PDF -> images")
|
||||||
pages = convert_pdf_to_images(pdf_path)
|
pages = convert_pdf_to_images(pdf_path)
|
||||||
|
|
||||||
|
|||||||
+104
-27
@@ -60,7 +60,15 @@
|
|||||||
border:1px solid var(--line); background:#0c0e13; color:var(--text); font-size:16px; outline:none; }
|
border:1px solid var(--line); background:#0c0e13; color:var(--text); font-size:16px; outline:none; }
|
||||||
.email-card input[type=email]::placeholder { color:#6b7280; }
|
.email-card input[type=email]::placeholder { color:#6b7280; }
|
||||||
.email-card input[type=email]:focus { border-color:var(--accent); box-shadow:0 0 0 3px rgba(91,140,255,.18); }
|
.email-card input[type=email]:focus { border-color:var(--accent); box-shadow:0 0 0 3px rgba(91,140,255,.18); }
|
||||||
|
.email-card select { width:100%; padding:10px 12px; border-radius:8px; margin-top:6px;
|
||||||
|
border:1px solid var(--line); background:#0c0e13; color:var(--text); font-size:14px; }
|
||||||
|
.email-card .field { margin-top:12px; }
|
||||||
|
.email-card .field > span { display:block; font-size:13px; color:var(--muted); margin-bottom:2px; }
|
||||||
.btn.full { width:100%; padding:14px; font-size:15px; margin-top:0; }
|
.btn.full { width:100%; padding:14px; font-size:15px; margin-top:0; }
|
||||||
|
.logbox { background:#0c0e13; border:1px solid var(--line); border-radius:8px; padding:12px 14px;
|
||||||
|
margin-top:10px; max-height:320px; overflow:auto; font:12px/1.45 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;
|
||||||
|
color:#c6cdd8; white-space:pre-wrap; word-break:break-word; }
|
||||||
|
.logbox .empty-log { color:var(--muted); }
|
||||||
.note { background:var(--panel); border:1px solid var(--line); border-radius:10px;
|
.note { background:var(--panel); border:1px solid var(--line); border-radius:10px;
|
||||||
padding:16px 18px; margin:18px 0; }
|
padding:16px 18px; margin:18px 0; }
|
||||||
.note b { color:var(--text); }
|
.note b { color:var(--text); }
|
||||||
@@ -121,12 +129,22 @@
|
|||||||
<label style="display:block;font-weight:400;margin-top:6px">
|
<label style="display:block;font-weight:400;margin-top:6px">
|
||||||
<input type="radio" name="compute" value="local"> Hybrid — text stages on local LLM (cheaper, slower)</label>
|
<input type="radio" name="compute" value="local"> Hybrid — text stages on local LLM (cheaper, slower)</label>
|
||||||
<div id="modelPick" style="margin-top:10px">
|
<div id="modelPick" style="margin-top:10px">
|
||||||
<label for="model" style="font-weight:400">Model <span class="opt" id="modelNote">loading...</span></label>
|
<div class="field">
|
||||||
<select id="model" style="width:100%;margin-top:6px;padding:10px;border-radius:8px;border:1px solid var(--line);background:#0c0e13;color:var(--text)"></select>
|
<span>Vision model <span class="opt">(image stages)</span> <span class="opt" id="modelNote">loading...</span></span>
|
||||||
|
<select id="vision_model" disabled><option value="">Loading models…</option></select>
|
||||||
|
</div>
|
||||||
|
<div class="field">
|
||||||
|
<span>Text model <span class="opt">(non-image stages)</span></span>
|
||||||
|
<select id="text_model" disabled><option value="">Loading models…</option></select>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<button class="btn full" id="run" disabled>Run conflict check</button>
|
<button class="btn full" id="run" disabled>Run conflict check</button>
|
||||||
<div class="status" id="status"></div>
|
<div class="status" id="status"></div>
|
||||||
|
<div id="liveLog" style="display:none" class="note">
|
||||||
|
<b>Run log</b> <span class="opt" id="logHint">(updates live)</span>
|
||||||
|
<pre class="logbox" id="logBox"><span class="empty-log">Waiting for output…</span></pre>
|
||||||
|
</div>
|
||||||
<div id="results"></div>
|
<div id="results"></div>
|
||||||
</main>
|
</main>
|
||||||
<div id="viewer">
|
<div id="viewer">
|
||||||
@@ -144,9 +162,71 @@
|
|||||||
const drop=document.getElementById('drop'), fileInput=document.getElementById('file'),
|
const drop=document.getElementById('drop'), fileInput=document.getElementById('file'),
|
||||||
runBtn=document.getElementById('run'), statusEl=document.getElementById('status'),
|
runBtn=document.getElementById('run'), statusEl=document.getElementById('status'),
|
||||||
results=document.getElementById('results'), dropLabel=document.getElementById('dropLabel'),
|
results=document.getElementById('results'), dropLabel=document.getElementById('dropLabel'),
|
||||||
emailEl=document.getElementById('email');
|
emailEl=document.getElementById('email'),
|
||||||
|
visionSel=document.getElementById('vision_model'),
|
||||||
|
textSel=document.getElementById('text_model'),
|
||||||
|
liveLog=document.getElementById('liveLog'),
|
||||||
|
logBox=document.getElementById('logBox'),
|
||||||
|
logHint=document.getElementById('logHint');
|
||||||
let chosen=null, polling=null, currentJobId=null, sheetPage={}, viewerZoom=1, reviewDirty=false;
|
let chosen=null, polling=null, currentJobId=null, sheetPage={}, viewerZoom=1, reviewDirty=false;
|
||||||
|
|
||||||
|
function modelLabel(m){
|
||||||
|
// Include per-1M-token pricing when the catalog provides it.
|
||||||
|
let s=m.name||m.id;
|
||||||
|
if(m.prompt_usd_per_mtok!=null)
|
||||||
|
s+=' — $'+m.prompt_usd_per_mtok+' / $'+m.completion_usd_per_mtok+' per 1M tok';
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
function fillSelect(sel, items, preferred){
|
||||||
|
sel.innerHTML='';
|
||||||
|
(items||[]).forEach(m=>{
|
||||||
|
const opt=document.createElement('option');
|
||||||
|
opt.value=m.id; opt.textContent=modelLabel(m);
|
||||||
|
if(m.id===preferred) opt.selected=true;
|
||||||
|
sel.appendChild(opt);
|
||||||
|
});
|
||||||
|
if(!sel.options.length){
|
||||||
|
const opt=document.createElement('option');
|
||||||
|
opt.value=preferred||''; opt.textContent=preferred||'(no models)';
|
||||||
|
sel.appendChild(opt);
|
||||||
|
}
|
||||||
|
sel.disabled=false;
|
||||||
|
}
|
||||||
|
|
||||||
|
let modelsLoaded=false;
|
||||||
|
async function loadModels(){
|
||||||
|
const note=document.getElementById('modelNote');
|
||||||
|
try{
|
||||||
|
const res=await fetch('/models');
|
||||||
|
if(!res.ok) throw new Error('models HTTP '+res.status);
|
||||||
|
const data=await res.json();
|
||||||
|
const defs=data.defaults||{};
|
||||||
|
fillSelect(visionSel, data.vision, defs.vision);
|
||||||
|
fillSelect(textSel, data.text, defs.text);
|
||||||
|
modelsLoaded=true;
|
||||||
|
note.textContent='('+(data.text||[]).length+' text / '+(data.vision||[]).length+' vision available)';
|
||||||
|
}catch(err){
|
||||||
|
visionSel.innerHTML='<option value="">(default)</option>';
|
||||||
|
textSel.innerHTML='<option value="">(default)</option>';
|
||||||
|
visionSel.disabled=false; textSel.disabled=false;
|
||||||
|
note.textContent='using configured defaults (list unavailable)';
|
||||||
|
console.warn('Could not load models:', err);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function showLog(lines, live){
|
||||||
|
liveLog.style.display='block';
|
||||||
|
logHint.textContent=live?'(updates live)':'(saved with this job)';
|
||||||
|
const arr=lines||[];
|
||||||
|
if(!arr.length){
|
||||||
|
logBox.innerHTML='<span class="empty-log">No log lines yet…</span>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
logBox.textContent=arr.join('\n');
|
||||||
|
logBox.scrollTop=logBox.scrollHeight;
|
||||||
|
}
|
||||||
|
|
||||||
function setFile(f){ chosen=f; dropLabel.textContent=f?('Selected: '+f.name):'Drop a PDF drawing set here, or click to choose';
|
function setFile(f){ chosen=f; dropLabel.textContent=f?('Selected: '+f.name):'Drop a PDF drawing set here, or click to choose';
|
||||||
runBtn.disabled=!f; }
|
runBtn.disabled=!f; }
|
||||||
dropLabel.addEventListener('click',()=>fileInput.click());
|
dropLabel.addEventListener('click',()=>fileInput.click());
|
||||||
@@ -160,6 +240,7 @@ runBtn.addEventListener('click',async e=>{
|
|||||||
if(!chosen) return;
|
if(!chosen) return;
|
||||||
runBtn.disabled=true; results.innerHTML='';
|
runBtn.disabled=true; results.innerHTML='';
|
||||||
statusEl.innerHTML='<span class="spinner"></span>Uploading...';
|
statusEl.innerHTML='<span class="spinner"></span>Uploading...';
|
||||||
|
showLog([], true);
|
||||||
const fd=new FormData(); fd.append('file',chosen);
|
const fd=new FormData(); fd.append('file',chosen);
|
||||||
const email=(emailEl.value||'').trim(); if(email) fd.append('notification_email',email);
|
const email=(emailEl.value||'').trim(); if(email) fd.append('notification_email',email);
|
||||||
['project_name','address','occupancy','work_type'].forEach(id=>{
|
['project_name','address','occupancy','work_type'].forEach(id=>{
|
||||||
@@ -168,8 +249,9 @@ runBtn.addEventListener('click',async e=>{
|
|||||||
const compute=(document.querySelector('input[name="compute"]:checked')||{}).value;
|
const compute=(document.querySelector('input[name="compute"]:checked')||{}).value;
|
||||||
fd.append('text_local', compute==='local' ? 'true' : 'false');
|
fd.append('text_local', compute==='local' ? 'true' : 'false');
|
||||||
if(compute==='openrouter'){
|
if(compute==='openrouter'){
|
||||||
const modelSel=document.getElementById('model');
|
// Model picks only apply to OpenRouter compute; hybrid keeps its local model.
|
||||||
if(modelSel.value) fd.append('model', modelSel.value);
|
if(visionSel.value) fd.append('vision_model', visionSel.value);
|
||||||
|
if(textSel.value) fd.append('text_model', textSel.value);
|
||||||
}
|
}
|
||||||
const pipelineMode=(document.querySelector('input[name="pipeline_mode"]:checked')||{}).value||'classic';
|
const pipelineMode=(document.querySelector('input[name="pipeline_mode"]:checked')||{}).value||'classic';
|
||||||
fd.append('pipeline_mode',pipelineMode);
|
fd.append('pipeline_mode',pipelineMode);
|
||||||
@@ -196,21 +278,30 @@ function poll(jobId){
|
|||||||
const res=await fetch('/jobs/'+jobId);
|
const res=await fetch('/jobs/'+jobId);
|
||||||
if(!res.ok) throw new Error('job not found');
|
if(!res.ok) throw new Error('job not found');
|
||||||
const job=await res.json();
|
const job=await res.json();
|
||||||
|
const live=['running','queued','finalizing'].includes(job.status);
|
||||||
|
if(job.log_tail && job.log_tail.length) showLog(job.log_tail, live);
|
||||||
if(job.status==='running'||job.status==='queued'){
|
if(job.status==='running'||job.status==='queued'){
|
||||||
statusEl.innerHTML='<span class="spinner"></span>'+esc(job.stage||'Working...')+
|
statusEl.innerHTML='<span class="spinner"></span>'+esc(job.stage||'Working...')+
|
||||||
' · you can leave this page';
|
' · you can leave this page';
|
||||||
} else if(job.status==='done'){
|
} else if(job.status==='done'){
|
||||||
clearInterval(polling); polling=null; runBtn.disabled=false; render(job.report);
|
clearInterval(polling); polling=null; runBtn.disabled=false;
|
||||||
|
if(job.log && job.log.length) showLog(job.log, false);
|
||||||
|
render(job.report);
|
||||||
} else if(job.status==='needs_review'||job.status==='reviewing'){
|
} else if(job.status==='needs_review'||job.status==='reviewing'){
|
||||||
clearInterval(polling); polling=null; runBtn.disabled=false; renderReview(job);
|
clearInterval(polling); polling=null; runBtn.disabled=false;
|
||||||
|
if(job.log && job.log.length) showLog(job.log, false);
|
||||||
|
renderReview(job);
|
||||||
} else if(job.status==='finalizing'){
|
} else if(job.status==='finalizing'){
|
||||||
statusEl.innerHTML='<span class="spinner"></span>Finalizing reviewed report...';
|
statusEl.innerHTML='<span class="spinner"></span>Finalizing reviewed report...';
|
||||||
} else if(job.status==='finalization_error'){
|
} else if(job.status==='finalization_error'){
|
||||||
clearInterval(polling); polling=null; runBtn.disabled=false;
|
clearInterval(polling); polling=null; runBtn.disabled=false;
|
||||||
statusEl.textContent='Finalization failed: '+(job.error||'unknown error');
|
statusEl.textContent='Finalization failed: '+(job.error||'unknown error');
|
||||||
|
if(job.log && job.log.length) showLog(job.log, false);
|
||||||
} else if(job.status==='error'){
|
} else if(job.status==='error'){
|
||||||
clearInterval(polling); polling=null; runBtn.disabled=false;
|
clearInterval(polling); polling=null; runBtn.disabled=false;
|
||||||
statusEl.textContent='Run failed: '+(job.error||'unknown error');
|
statusEl.textContent='Run failed: '+(job.error||'unknown error');
|
||||||
|
if(job.log && job.log.length) showLog(job.log, false);
|
||||||
|
else if(job.log_tail && job.log_tail.length) showLog(job.log_tail, false);
|
||||||
}
|
}
|
||||||
}catch(err){ clearInterval(polling); polling=null; runBtn.disabled=false;
|
}catch(err){ clearInterval(polling); polling=null; runBtn.disabled=false;
|
||||||
statusEl.textContent='Error: '+err.message; }
|
statusEl.textContent='Error: '+err.message; }
|
||||||
@@ -230,29 +321,11 @@ function syncPipelineOptions(){
|
|||||||
document.querySelectorAll('input[name="pipeline_mode"]').forEach(el=>el.addEventListener('change',syncPipelineOptions));
|
document.querySelectorAll('input[name="pipeline_mode"]').forEach(el=>el.addEventListener('change',syncPipelineOptions));
|
||||||
syncPipelineOptions();
|
syncPipelineOptions();
|
||||||
|
|
||||||
// --- model picker (OpenRouter compute only) ---
|
// --- model pickers (OpenRouter compute only) ---
|
||||||
let modelList=null;
|
|
||||||
async function loadModels(){
|
|
||||||
const note=document.getElementById('modelNote'), sel=document.getElementById('model');
|
|
||||||
try{
|
|
||||||
const res=await fetch('/models');
|
|
||||||
if(!res.ok) throw new Error('list unavailable');
|
|
||||||
const data=await res.json();
|
|
||||||
modelList=data.models||[];
|
|
||||||
sel.innerHTML=modelList.map(m=>
|
|
||||||
'<option value="'+escAttr(m.id)+'"'+(m.id===data.default?' selected':'')+'>'+
|
|
||||||
esc(m.name||m.id)+' — $'+esc(m.prompt_usd_per_mtok)+' / $'+esc(m.completion_usd_per_mtok)+
|
|
||||||
' per 1M tok</option>').join('');
|
|
||||||
note.textContent='('+modelList.length+' available)';
|
|
||||||
}catch(e){
|
|
||||||
sel.innerHTML='';
|
|
||||||
note.textContent='using configured default (list unavailable)';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
function syncCompute(){
|
function syncCompute(){
|
||||||
const openrouter=(document.querySelector('input[name="compute"]:checked')||{}).value==='openrouter';
|
const openrouter=(document.querySelector('input[name="compute"]:checked')||{}).value==='openrouter';
|
||||||
document.getElementById('modelPick').style.display=openrouter?'block':'none';
|
document.getElementById('modelPick').style.display=openrouter?'block':'none';
|
||||||
if(openrouter&&!modelList) loadModels();
|
if(openrouter&&!modelsLoaded) loadModels();
|
||||||
}
|
}
|
||||||
document.querySelectorAll('input[name="compute"]').forEach(el=>el.addEventListener('change',syncCompute));
|
document.querySelectorAll('input[name="compute"]').forEach(el=>el.addEventListener('change',syncCompute));
|
||||||
syncCompute();
|
syncCompute();
|
||||||
@@ -394,6 +467,10 @@ function render(rep){
|
|||||||
html+='</details>';
|
html+='</details>';
|
||||||
}
|
}
|
||||||
|
|
||||||
|
html+='<div class="meta" style="margin-top:14px">Full run log: <a href="/jobs/'+
|
||||||
|
esc(currentJobId)+'/log?plain=1" target="_blank" rel="noopener">/jobs/'+
|
||||||
|
esc(currentJobId)+'/log</a> (also saved as job.log on the server)</div>';
|
||||||
|
|
||||||
results.innerHTML=html;
|
results.innerHTML=html;
|
||||||
}
|
}
|
||||||
function stat(v,l){ return '<div class="stat"><b>'+esc(v)+'</b><span>'+esc(l)+'</span></div>'; }
|
function stat(v,l){ return '<div class="stat"><b>'+esc(v)+'</b><span>'+esc(l)+'</span></div>'; }
|
||||||
|
|||||||
@@ -60,18 +60,56 @@ def test_job_log_endpoint_serves_log_and_404s(job_env, monkeypatch):
|
|||||||
assert client.get("/jobs/nope/log").status_code == 404
|
assert client.get("/jobs/nope/log").status_code == 404
|
||||||
|
|
||||||
|
|
||||||
def test_model_override_set_and_cleared_around_run(job_env, monkeypatch):
|
def test_model_overrides_passed_to_classic_runner(job_env, monkeypatch):
|
||||||
from backend import llm
|
"""Classic mode: per-run picks travel as run_pipeline kwargs (the runner
|
||||||
|
sets and clears llm.set_model_overrides itself)."""
|
||||||
seen = {}
|
seen = {}
|
||||||
|
|
||||||
def fake_runner(pdf_path, **kwargs):
|
def fake_runner(pdf_path, **kwargs):
|
||||||
seen["override"] = llm._model_override
|
seen.update(kwargs)
|
||||||
return {"source": "set.pdf", "summary": {}}
|
return {"source": "set.pdf", "summary": {}}
|
||||||
|
|
||||||
monkeypatch.setattr("backend.jobs.run_pipeline", fake_runner)
|
monkeypatch.setattr("backend.jobs.run_pipeline", fake_runner)
|
||||||
jobs.create_job(str(job_env / "set.pdf"), "set.pdf",
|
jobs.create_job(str(job_env / "set.pdf"), "set.pdf",
|
||||||
pipeline_mode="classic", model="openai/gpt-4o")
|
pipeline_mode="classic",
|
||||||
|
vision_model="openai/gpt-4o", text_model="openai/gpt-4o-mini")
|
||||||
|
|
||||||
assert seen["override"] == "openai/gpt-4o"
|
assert seen["vision_model"] == "openai/gpt-4o"
|
||||||
assert llm._model_override is None # cleared after the run
|
assert seen["text_model"] == "openai/gpt-4o-mini"
|
||||||
|
|
||||||
|
|
||||||
|
def test_model_overrides_set_and_cleared_around_agent_run(job_env, monkeypatch):
|
||||||
|
"""Agent mode: the agent runner has no override params, so jobs.py sets
|
||||||
|
them module-level for the duration of the run."""
|
||||||
|
from backend import llm
|
||||||
|
|
||||||
|
seen = {}
|
||||||
|
|
||||||
|
def fake_agent_runner(pdf_path, **kwargs):
|
||||||
|
seen["vision"] = llm._vision_model_override
|
||||||
|
seen["text"] = llm._text_model_override
|
||||||
|
return {"source": "set.pdf", "summary": {}}
|
||||||
|
|
||||||
|
monkeypatch.setattr("backend.jobs.run_agent_pipeline", fake_agent_runner)
|
||||||
|
jobs.create_job(str(job_env / "set.pdf"), "set.pdf",
|
||||||
|
pipeline_mode="agent",
|
||||||
|
vision_model="openai/gpt-4o", text_model="openai/gpt-4o-mini")
|
||||||
|
|
||||||
|
assert seen["vision"] == "openai/gpt-4o"
|
||||||
|
assert seen["text"] == "openai/gpt-4o-mini"
|
||||||
|
assert llm._vision_model_override is None # cleared after the run
|
||||||
|
assert llm._text_model_override is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_failed_run_logs_traceback(job_env, monkeypatch):
|
||||||
|
"""A crashed job must leave the traceback in job.log, not just str(e)."""
|
||||||
|
def boom(pdf_path, **kwargs):
|
||||||
|
raise RuntimeError("kaboom-stage-failure")
|
||||||
|
|
||||||
|
monkeypatch.setattr("backend.jobs.run_pipeline", boom)
|
||||||
|
job_id = jobs.create_job(str(job_env / "set.pdf"), "set.pdf", pipeline_mode="classic")
|
||||||
|
|
||||||
|
assert jobs._jobs[job_id]["status"] == "error"
|
||||||
|
content = (job_env / job_id / "job.log").read_text()
|
||||||
|
assert "Traceback (most recent call last)" in content
|
||||||
|
assert "RuntimeError: kaboom-stage-failure" in content
|
||||||
|
|||||||
@@ -11,12 +11,23 @@ _PAYLOAD = {
|
|||||||
"name": "GPT-4o",
|
"name": "GPT-4o",
|
||||||
"pricing": {"prompt": "0.0000025", "completion": "0.00001"},
|
"pricing": {"prompt": "0.0000025", "completion": "0.00001"},
|
||||||
"context_length": 128000,
|
"context_length": 128000,
|
||||||
|
"architecture": {"input_modalities": ["text", "image"],
|
||||||
|
"output_modalities": ["text"]},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"id": "google/gemini-2.5-pro",
|
"id": "google/gemini-2.5-pro",
|
||||||
"name": "Gemini 2.5 Pro",
|
"name": "Gemini 2.5 Pro",
|
||||||
"pricing": {"prompt": "0.00000125", "completion": "0.00001"},
|
"pricing": {"prompt": "0.00000125", "completion": "0.00001"},
|
||||||
"context_length": 1000000,
|
"context_length": 1000000,
|
||||||
|
"architecture": {"modality": "text+image->text"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "meta-llama/llama-3.1-70b-instruct",
|
||||||
|
"name": "Llama 3.1 70B Instruct",
|
||||||
|
"pricing": {"prompt": "0.0000005", "completion": "0.0000008"},
|
||||||
|
"context_length": 131072,
|
||||||
|
"architecture": {"input_modalities": ["text"],
|
||||||
|
"output_modalities": ["text"]},
|
||||||
},
|
},
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
@@ -34,14 +45,27 @@ def test_models_endpoint_normalizes_pricing(monkeypatch):
|
|||||||
response = client.get("/models")
|
response = client.get("/models")
|
||||||
assert response.status_code == 200
|
assert response.status_code == 200
|
||||||
body = response.json()
|
body = response.json()
|
||||||
assert body["default"] == config.MODEL
|
assert body["defaults"] == {"vision": config.MODEL, "text": config.TEXT_MODEL}
|
||||||
assert body["default_text"] == config.TEXT_MODEL
|
by_id = {m["id"]: m for m in body["text"]}
|
||||||
by_id = {m["id"]: m for m in body["models"]}
|
|
||||||
assert by_id["openai/gpt-4o"]["prompt_usd_per_mtok"] == 2.5
|
assert by_id["openai/gpt-4o"]["prompt_usd_per_mtok"] == 2.5
|
||||||
assert by_id["openai/gpt-4o"]["completion_usd_per_mtok"] == 10.0
|
assert by_id["openai/gpt-4o"]["completion_usd_per_mtok"] == 10.0
|
||||||
assert by_id["openai/gpt-4o"]["context_length"] == 128000
|
assert by_id["openai/gpt-4o"]["context_length"] == 128000
|
||||||
|
|
||||||
|
|
||||||
|
def test_models_endpoint_splits_vision_and_text(monkeypatch):
|
||||||
|
_reset_cache()
|
||||||
|
monkeypatch.setattr(models, "_fetch_openrouter_models", lambda: _PAYLOAD["data"])
|
||||||
|
client = TestClient(app)
|
||||||
|
body = client.get("/models").json()
|
||||||
|
vision_ids = {m["id"] for m in body["vision"]}
|
||||||
|
text_ids = {m["id"] for m in body["text"]}
|
||||||
|
# Both modality shapes (structured and legacy string) are recognized.
|
||||||
|
assert vision_ids == {"openai/gpt-4o", "google/gemini-2.5-pro"}
|
||||||
|
# Text list is the full catalog; vision models appear in both.
|
||||||
|
assert text_ids == {"openai/gpt-4o", "google/gemini-2.5-pro",
|
||||||
|
"meta-llama/llama-3.1-70b-instruct"}
|
||||||
|
|
||||||
|
|
||||||
def test_models_endpoint_caches(monkeypatch):
|
def test_models_endpoint_caches(monkeypatch):
|
||||||
_reset_cache()
|
_reset_cache()
|
||||||
calls = []
|
calls = []
|
||||||
|
|||||||
@@ -1,26 +1,44 @@
|
|||||||
from backend import config
|
from backend import config
|
||||||
from backend.llm import _resolve_backend, set_model_override
|
from backend.llm import _resolve_backend, set_model_overrides, set_text_backend
|
||||||
|
|
||||||
|
|
||||||
def test_override_wins_for_vision_and_text():
|
def teardown_function():
|
||||||
set_model_override("openai/gpt-4o")
|
set_model_overrides(None, None)
|
||||||
try:
|
set_text_backend(False)
|
||||||
assert _resolve_backend(has_images=True, model_override=None)["model"] == "openai/gpt-4o"
|
|
||||||
assert _resolve_backend(has_images=False, model_override=None)["model"] == "openai/gpt-4o"
|
|
||||||
finally:
|
def test_vision_override_wins_for_vision_only():
|
||||||
set_model_override(None)
|
set_model_overrides(vision="openai/gpt-4o", text=None)
|
||||||
|
assert _resolve_backend(has_images=True, model_override=None)["model"] == "openai/gpt-4o"
|
||||||
|
assert _resolve_backend(has_images=False, model_override=None)["model"] == config.TEXT_MODEL
|
||||||
|
|
||||||
|
|
||||||
|
def test_text_override_wins_for_text_only():
|
||||||
|
set_model_overrides(vision=None, text="anthropic/claude-sonnet-4")
|
||||||
|
assert _resolve_backend(has_images=False, model_override=None)["model"] == "anthropic/claude-sonnet-4"
|
||||||
|
assert _resolve_backend(has_images=True, model_override=None)["model"] == config.MODEL
|
||||||
|
|
||||||
|
|
||||||
def test_override_beats_per_call_model_arg():
|
def test_override_beats_per_call_model_arg():
|
||||||
set_model_override("openai/gpt-4o")
|
set_model_overrides(vision="openai/gpt-4o", text="openai/gpt-4o-mini")
|
||||||
try:
|
# Agents pass their AGENT_*_MODEL per call; the user's job pick wins.
|
||||||
# Agents pass their AGENT_*_MODEL per call; the user's job pick wins.
|
assert _resolve_backend(has_images=True, model_override="other/model")["model"] == "openai/gpt-4o"
|
||||||
assert _resolve_backend(has_images=False, model_override="other/model")["model"] == "openai/gpt-4o"
|
assert _resolve_backend(has_images=False, model_override="other/model")["model"] == "openai/gpt-4o-mini"
|
||||||
finally:
|
|
||||||
set_model_override(None)
|
|
||||||
|
|
||||||
|
|
||||||
def test_no_override_keeps_defaults():
|
def test_no_override_keeps_defaults():
|
||||||
set_model_override(None)
|
set_model_overrides(None, None)
|
||||||
assert _resolve_backend(has_images=True, model_override=None)["model"] == config.MODEL
|
assert _resolve_backend(has_images=True, model_override=None)["model"] == config.MODEL
|
||||||
assert _resolve_backend(has_images=False, model_override=None)["model"] == config.TEXT_MODEL
|
assert _resolve_backend(has_images=False, model_override=None)["model"] == config.TEXT_MODEL
|
||||||
|
|
||||||
|
|
||||||
|
def test_ui_picks_never_name_the_local_model(monkeypatch):
|
||||||
|
"""Hybrid runs keep LOCAL_TEXT_MODEL; OpenRouter picks must not leak into
|
||||||
|
the local endpoint (a vLLM server won't serve OpenRouter model ids)."""
|
||||||
|
monkeypatch.setattr(config, "LOCAL_BASE_URL", "http://localhost:8000/v1")
|
||||||
|
monkeypatch.setattr(config, "LOCAL_TEXT_MODEL", "qwen/local-instruct")
|
||||||
|
set_text_backend(True)
|
||||||
|
set_model_overrides(vision="openai/gpt-4o", text="anthropic/claude-sonnet-4")
|
||||||
|
be = _resolve_backend(has_images=False, model_override=None)
|
||||||
|
assert be["local"] is True
|
||||||
|
assert be["model"] == "qwen/local-instruct"
|
||||||
|
|||||||
Reference in New Issue
Block a user