Compare commits
2
Commits
afa1089311
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bf508bfdf6 | ||
|
|
a6b0c8fdfa |
@@ -0,0 +1,147 @@
|
|||||||
|
# Session Notes — Conflict Checker
|
||||||
|
|
||||||
|
Orientation for a new coding session. Setup and Docker details live in [README.md](README.md). This file tracks what the code actually does and what tends to waste time.
|
||||||
|
|
||||||
|
**Last updated:** 2026-07-31 · tip `a6b0c8f` on Gitea `main`
|
||||||
|
|
||||||
|
## What this is
|
||||||
|
|
||||||
|
Cross-discipline **design contradiction / senior-architect QAQC** for construction drawing PDFs (Arch, Struct, Mech, Elec, Plumb, FP, etc.). Flags disagreements *between disciplines* before a set goes to bid/permit.
|
||||||
|
|
||||||
|
**Not** IronBid’s scope-ownership conflict checker.
|
||||||
|
|
||||||
|
Source of truth: Scout IT Gitea — `gitea.scoutitsystems.com/woogi/Conflict_Checker`.
|
||||||
|
|
||||||
|
## Stack
|
||||||
|
|
||||||
|
| Layer | Detail |
|
||||||
|
|-------|--------|
|
||||||
|
| API | Python 3.12, FastAPI, Uvicorn ([backend/main.py](backend/main.py)) |
|
||||||
|
| UI | Single static file [frontend/index.html](frontend/index.html), served by FastAPI |
|
||||||
|
| Pipeline | Shared by web + CLI: [backend/pipeline/runner.py](backend/pipeline/runner.py) |
|
||||||
|
| LLM | OpenRouter via `openai` SDK; default `google/gemini-2.5-pro`. Vision always OpenRouter; text stages can use local vLLM |
|
||||||
|
| PDF | `pdf2image` + system `poppler-utils` → JPEG page images |
|
||||||
|
| Jobs | In-memory threads ([backend/jobs.py](backend/jobs.py)) — no Redis/DB |
|
||||||
|
| Deploy | Docker Compose; app on port **8099** |
|
||||||
|
|
||||||
|
## Live pipeline (authoritative)
|
||||||
|
|
||||||
|
README still describes an older 5-stage extract-then-compare loop. **Trust `runner.py`.** Actual flow:
|
||||||
|
|
||||||
|
```
|
||||||
|
PDF → images → extract → sheet index → jurisdiction
|
||||||
|
→ normalize → project intelligence (GOIDs)
|
||||||
|
→ cluster → conflict reason
|
||||||
|
→ QAQC / code / constructability
|
||||||
|
→ dedup-validate → risk → RFIs → report
|
||||||
|
```
|
||||||
|
|
||||||
|
| Runner stage | Module | Notes |
|
||||||
|
|--------------|--------|--------|
|
||||||
|
| PDF → images | `pdf_processor` | Rasterize |
|
||||||
|
| Extract assertions | `extractor` | Vision, per sheet |
|
||||||
|
| Classify sheet index | `sheet_index` | LLM |
|
||||||
|
| Jurisdiction profile | `jurisdiction` | After cover meta |
|
||||||
|
| Normalize | `normalizer` | LLM batches |
|
||||||
|
| Project intelligence | `normalizer.build_project_intelligence` | GOIDs + relationships |
|
||||||
|
| Cluster | `llm_clusterer` or `clusterer` | Default `CLUSTERER=llm` |
|
||||||
|
| Conflicts | `conflict_checker` | Per-cluster vision reason |
|
||||||
|
| QAQC / code / constructability | `qaqc_review`, `code_review`, `constructability` | Full-set / batched |
|
||||||
|
| Validate & dedup | `validator` | Merges conflict + QAQC + code + construct issues |
|
||||||
|
| Risk / RFIs | `risk`, `rfi` | Text-only |
|
||||||
|
| Report | `report` | `conflicts.json` + `report.md` (+ stage JSON dumps when `out_dir` set) |
|
||||||
|
|
||||||
|
Design notes for Stage 2/3 engines also live under `Changes/*.docx`.
|
||||||
|
|
||||||
|
## HTTP API (current)
|
||||||
|
|
||||||
|
| Method | Path | Purpose |
|
||||||
|
|--------|------|---------|
|
||||||
|
| GET | `/health` | Liveness + default `model` / `text_model` + key/email flags |
|
||||||
|
| GET | `/models` | OpenRouter catalog split into `vision[]` / `text[]` + `defaults` (cached ~1h) |
|
||||||
|
| POST | `/check` | Upload PDF; returns `{job_id}` immediately |
|
||||||
|
| GET | `/jobs/{id}` | Status poll. Running: `stage` + `log_tail`. Done/error: `report` and/or `error` + full `log` |
|
||||||
|
| GET | `/jobs/{id}/log` | Full run log JSON (`lines`, `text`); `?plain=1` for text/plain |
|
||||||
|
| GET | `/jobs/{id}/sheet-image/{page}` | JPEG of source PDF page for the sheet viewer |
|
||||||
|
| GET | `/` | Serves `frontend/index.html` |
|
||||||
|
|
||||||
|
### `/check` form fields
|
||||||
|
|
||||||
|
- Required: `file` (PDF)
|
||||||
|
- Optional: `notification_email`, `project_name`, `address`, `occupancy`, `work_type`
|
||||||
|
- Compute: `text_local` (`true` = hybrid local text)
|
||||||
|
- Models: `vision_model`, `text_model` (OpenRouter ids; blank = config defaults)
|
||||||
|
|
||||||
|
## Job logs
|
||||||
|
|
||||||
|
Pipeline `print()` is teed for the job thread ([backend/job_log.py](backend/job_log.py)):
|
||||||
|
|
||||||
|
- Live: `GET /jobs/{id}` → `log_tail` (last 80 lines)
|
||||||
|
- Done/error: same payload includes full `log`
|
||||||
|
- Disk: `backend/outputs/<job_id>/job.log` (survives restart; status registry does not)
|
||||||
|
- API: `GET /jobs/{id}/log` or `?plain=1`
|
||||||
|
- UI: “Run log” panel updates while running; stays visible after finish/fail
|
||||||
|
|
||||||
|
## Vision vs text models
|
||||||
|
|
||||||
|
Two models, not one:
|
||||||
|
|
||||||
|
| Role | Config | Stages | Backend |
|
||||||
|
|------|--------|--------|---------|
|
||||||
|
| Vision | `MODEL` | Extract, conflict reason (images) | Always OpenRouter |
|
||||||
|
| Text | `TEXT_MODEL` (falls back to `MODEL`) | Sheet index, jurisdiction, normalize, cluster(LLM), QAQC, code, construct, validate, risk, RFI | OpenRouter, or local when hybrid |
|
||||||
|
|
||||||
|
UI: two dropdowns filled from `GET /models` ([backend/models_catalog.py](backend/models_catalog.py)). Per-run picks go through `set_model_overrides()` in [backend/llm.py](backend/llm.py); runner clears them in `finally`. Hybrid: text dropdown also names the local model override and OpenRouter fallback.
|
||||||
|
|
||||||
|
## Where to change what
|
||||||
|
|
||||||
|
| Concern | File |
|
||||||
|
|---------|------|
|
||||||
|
| Prompts, vocab, conflict taxonomy | [backend/prompts.py](backend/prompts.py) |
|
||||||
|
| Env knobs | [backend/config.py](backend/config.py), [backend/.env.example](backend/.env.example) |
|
||||||
|
| HTTP API | [backend/main.py](backend/main.py) |
|
||||||
|
| UI (upload, models, live log, results) | [frontend/index.html](frontend/index.html) |
|
||||||
|
| CLI tuning loop | [cli/run_check.py](cli/run_check.py) |
|
||||||
|
| LLM client, cache, cost, model overrides | [backend/llm.py](backend/llm.py) |
|
||||||
|
| OpenRouter vision/text model lists | [backend/models_catalog.py](backend/models_catalog.py) |
|
||||||
|
| Job registry + stdout tee log | [backend/jobs.py](backend/jobs.py), [backend/job_log.py](backend/job_log.py) |
|
||||||
|
| Stage helpers (prompt render, issue validate) | [backend/pipeline/_stage.py](backend/pipeline/_stage.py) |
|
||||||
|
| Code text corpus (Stage 7) | [backend/code_corpus/](backend/code_corpus/) |
|
||||||
|
| Hybrid local LLM helper | [scripts/setup_vllm.sh](scripts/setup_vllm.sh) |
|
||||||
|
|
||||||
|
Older prompt snapshot: `backend/prompts.py.v1`.
|
||||||
|
|
||||||
|
## Gotchas
|
||||||
|
|
||||||
|
1. **Prompt placeholders** — Use `str.replace` via `_stage.render`, never `str.format`. Prompts contain literal `{` JSON braces.
|
||||||
|
2. **Clustering** — Default is LLM (`CLUSTERER=llm`); empty LLM result falls back to deterministic. Deterministic clusters need ≥2 disciplines (or schedule-vs-plan); single-discipline “missing” gaps are a known limit.
|
||||||
|
3. **Jobs are in-memory** — Process restart clears job status; `outputs/<job_id>/` (report + `job.log` + `source.pdf`) still reload via disk fallback.
|
||||||
|
4. **Dependency pin** — `httpx==0.27.2` with `openai==1.51.0`. httpx ≥0.28 breaks openai’s `proxies=` kwarg.
|
||||||
|
5. **Code corpus licensing** — Only `ada_2010.txt` is shipped. Do not paste IBC/IFC/IECC without a license (see `backend/code_corpus/README.md`).
|
||||||
|
6. **Dual assertion schema** — Newer `{sheet, objects[]}` is mapped to legacy `{assertions[]}` with `attribute`/`value` for older stages.
|
||||||
|
7. **Grounding guard** — Extractor drops objects whose numeric claims are not in `source_text` (graphical-only objects allowed).
|
||||||
|
8. **Module globals** — LLM cost counters, stdout tee, and model overrides are process-global; overlapping jobs can interleave (single-user tool assumption).
|
||||||
|
9. **Samples** — `samples/*.pdf` are gitignored; drop PDFs locally for CLI runs.
|
||||||
|
10. **README drift** — Treat README for setup/CI; treat this file + `runner.py` for pipeline truth. `prompts.py` header may still say some prompts are unwired — they are wired through the runner.
|
||||||
|
11. **Git identity** — This box has no `user.name` / `user.email`; commits need `GIT_AUTHOR_*` / `GIT_COMMITTER_*` env vars (do not `git config`). Remote push to Gitea works.
|
||||||
|
12. **No local Python deps on host** — App is meant to run in Docker; bare `python3` imports may miss `dotenv` / `httpx`. Prefer `docker compose`.
|
||||||
|
|
||||||
|
## Quick start pointers
|
||||||
|
|
||||||
|
- Full setup: [README.md](README.md) (`docker compose up -d --build` → http://localhost:8099).
|
||||||
|
- Local CLI: `python cli/run_check.py samples/your_set.pdf --out out/your_set`.
|
||||||
|
- Prompt iteration: set `LLM_CACHE=true` in `backend/.env` so unchanged stages replay for free; clear with `rm -rf backend/.llm_cache`.
|
||||||
|
- Artifacts per job: `assertions.json`, `clusters.json`, per-stage JSON, `conflicts.json`, `report.md`, `job.log`, `source.pdf` under `backend/outputs/<job_id>/`.
|
||||||
|
- No automated test suite; validate via CLI dumps and golden-set diffs (described in README).
|
||||||
|
|
||||||
|
## Recent work (2026-07-31)
|
||||||
|
|
||||||
|
Shipped on `main` as `a6b0c8f`:
|
||||||
|
|
||||||
|
- Per-job run log (tee + disk + API + UI)
|
||||||
|
- Separate vision/text model dropdowns backed by OpenRouter `/models`
|
||||||
|
- Session notes file (this doc)
|
||||||
|
|
||||||
|
## Conflict categories (taxonomy)
|
||||||
|
|
||||||
|
Defined in `backend/prompts.py`: `dimensional_disagreement`, `elevation_disagreement`, `location_mismatch`, `missing_element`, `schedule_vs_plan_mismatch`, `detail_vs_plan_mismatch`, `tag_or_reference_inconsistency`, `spatial_clash`, `note_or_spec_contradiction`.
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
"""
|
||||||
|
job_log.py - Capture pipeline stdout/stderr into a per-job log.
|
||||||
|
|
||||||
|
The pipeline already prints stage progress via print(). For a reviewable
|
||||||
|
post-run log we tee those lines into memory + outputs/<job_id>/job.log
|
||||||
|
without rewriting every call site.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from contextlib import contextmanager
|
||||||
|
from typing import Callable, Iterator, List, Optional, TextIO
|
||||||
|
|
||||||
|
|
||||||
|
class _LineSplitter:
|
||||||
|
"""Accumulate write() chunks and emit complete lines."""
|
||||||
|
|
||||||
|
def __init__(self, on_line: Callable[[str], None]):
|
||||||
|
self._buf = ""
|
||||||
|
self._on_line = on_line
|
||||||
|
|
||||||
|
def write(self, s: str) -> None:
|
||||||
|
if not s:
|
||||||
|
return
|
||||||
|
self._buf += s
|
||||||
|
while "\n" in self._buf:
|
||||||
|
line, self._buf = self._buf.split("\n", 1)
|
||||||
|
# Strip trailing CR from Windows-ish streams; keep content intact.
|
||||||
|
self._on_line(line.rstrip("\r"))
|
||||||
|
|
||||||
|
def flush_remainder(self) -> None:
|
||||||
|
if self._buf:
|
||||||
|
self._on_line(self._buf.rstrip("\r"))
|
||||||
|
self._buf = ""
|
||||||
|
|
||||||
|
|
||||||
|
class _Tee:
|
||||||
|
"""Mirror writes to the original stream and a line callback."""
|
||||||
|
|
||||||
|
def __init__(self, stream: TextIO, on_line: Callable[[str], None]):
|
||||||
|
self._stream = stream
|
||||||
|
self._lines = _LineSplitter(on_line)
|
||||||
|
|
||||||
|
def write(self, s: str) -> int:
|
||||||
|
n = self._stream.write(s)
|
||||||
|
self._stream.flush()
|
||||||
|
self._lines.write(s)
|
||||||
|
return n
|
||||||
|
|
||||||
|
def flush(self) -> None:
|
||||||
|
self._stream.flush()
|
||||||
|
|
||||||
|
def flush_remainder(self) -> None:
|
||||||
|
self._lines.flush_remainder()
|
||||||
|
|
||||||
|
def __getattr__(self, name: str):
|
||||||
|
return getattr(self._stream, name)
|
||||||
|
|
||||||
|
|
||||||
|
def stamp_line(line: str, t: Optional[float] = None) -> str:
|
||||||
|
"""Prefix a log line with HH:MM:SS."""
|
||||||
|
ts = time.strftime("%H:%M:%S", time.localtime(t if t is not None else time.time()))
|
||||||
|
return f"[{ts}] {line}"
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def capture_stdio(on_line: Callable[[str], None]) -> Iterator[None]:
|
||||||
|
"""
|
||||||
|
Tee sys.stdout and sys.stderr into on_line(raw_line) for the duration.
|
||||||
|
|
||||||
|
Safe for the single-job-at-a-time usage of this app; overlapping jobs
|
||||||
|
would interleave (same limitation as the LLM cost counters).
|
||||||
|
"""
|
||||||
|
old_out, old_err = sys.stdout, sys.stderr
|
||||||
|
tee_out = _Tee(old_out, on_line)
|
||||||
|
tee_err = _Tee(old_err, on_line)
|
||||||
|
sys.stdout = tee_out # type: ignore[assignment]
|
||||||
|
sys.stderr = tee_err # type: ignore[assignment]
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
tee_out.flush_remainder()
|
||||||
|
tee_err.flush_remainder()
|
||||||
|
sys.stdout = old_out
|
||||||
|
sys.stderr = old_err
|
||||||
|
|
||||||
|
|
||||||
|
def read_log_file(path: str) -> List[str]:
|
||||||
|
try:
|
||||||
|
with open(path, encoding="utf-8") as f:
|
||||||
|
return [ln.rstrip("\n") for ln in f]
|
||||||
|
except OSError:
|
||||||
|
return []
|
||||||
+109
-19
@@ -9,6 +9,9 @@ for the completion email.
|
|||||||
State is in-memory (fine for a single-user tool); the report is also persisted
|
State is in-memory (fine for a single-user tool); the report is also persisted
|
||||||
to outputs/<job_id>/ so results survive a restart even though live status does
|
to outputs/<job_id>/ so results survive a restart even though live status does
|
||||||
not. No external queue/DB.
|
not. No external queue/DB.
|
||||||
|
|
||||||
|
A teed stdout/stderr log is kept in memory and written to outputs/<job_id>/job.log
|
||||||
|
so failed or suspicious runs can be reviewed after the fact.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
@@ -16,14 +19,16 @@ import time
|
|||||||
import uuid
|
import uuid
|
||||||
import shutil
|
import shutil
|
||||||
import threading
|
import threading
|
||||||
from typing import Dict, Optional
|
from typing import Dict, List, Optional
|
||||||
|
|
||||||
from backend import config
|
from backend import config
|
||||||
|
from backend.job_log import capture_stdio, read_log_file, stamp_line
|
||||||
from backend.pipeline.runner import run_pipeline
|
from backend.pipeline.runner import run_pipeline
|
||||||
from backend.email_sender import send_conflict_report
|
from backend.email_sender import send_conflict_report
|
||||||
|
|
||||||
_jobs: Dict[str, Dict] = {}
|
_jobs: Dict[str, Dict] = {}
|
||||||
_lock = threading.Lock()
|
_lock = threading.Lock()
|
||||||
|
_LOG_TAIL = 80
|
||||||
|
|
||||||
|
|
||||||
def _set(job_id: str, **fields) -> None:
|
def _set(job_id: str, **fields) -> None:
|
||||||
@@ -31,8 +36,32 @@ def _set(job_id: str, **fields) -> None:
|
|||||||
_jobs[job_id].update(fields)
|
_jobs[job_id].update(fields)
|
||||||
|
|
||||||
|
|
||||||
def create_job(pdf_path: str, source_filename: str, email: Optional[str] = None,
|
def _append_log(job_id: str, raw_line: str, log_path: str) -> None:
|
||||||
project_input: Optional[Dict] = None, text_local: bool = False) -> str:
|
"""Stamp, store, and append one captured stdout/stderr line."""
|
||||||
|
entry = stamp_line(raw_line)
|
||||||
|
with _lock:
|
||||||
|
job = _jobs.get(job_id)
|
||||||
|
if job is not None:
|
||||||
|
job.setdefault("log", []).append(entry)
|
||||||
|
try:
|
||||||
|
os.makedirs(os.path.dirname(log_path), exist_ok=True)
|
||||||
|
with open(log_path, "a", encoding="utf-8") as f:
|
||||||
|
f.write(entry + "\n")
|
||||||
|
except OSError:
|
||||||
|
# Don't fail the job over log I/O; avoid print() here — it would
|
||||||
|
# re-enter the stdio tee while a job is capturing.
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def create_job(
|
||||||
|
pdf_path: str,
|
||||||
|
source_filename: str,
|
||||||
|
email: Optional[str] = None,
|
||||||
|
project_input: Optional[Dict] = None,
|
||||||
|
text_local: bool = False,
|
||||||
|
vision_model: Optional[str] = None,
|
||||||
|
text_model: Optional[str] = None,
|
||||||
|
) -> str:
|
||||||
"""Register a job and kick off its background thread. Returns the job_id."""
|
"""Register a job and kick off its background thread. Returns the job_id."""
|
||||||
job_id = uuid.uuid4().hex[:12]
|
job_id = uuid.uuid4().hex[:12]
|
||||||
with _lock:
|
with _lock:
|
||||||
@@ -43,25 +72,46 @@ def create_job(pdf_path: str, source_filename: str, email: Optional[str] = None,
|
|||||||
"email": email or None,
|
"email": email or None,
|
||||||
"project_input": project_input or {},
|
"project_input": project_input or {},
|
||||||
"text_local": text_local,
|
"text_local": text_local,
|
||||||
|
"vision_model": (vision_model or "").strip() or None,
|
||||||
|
"text_model": (text_model or "").strip() or None,
|
||||||
"stage": None,
|
"stage": None,
|
||||||
"created_at": time.time(),
|
"created_at": time.time(),
|
||||||
"finished_at": None,
|
"finished_at": None,
|
||||||
"report": None,
|
"report": None,
|
||||||
"error": None,
|
"error": None,
|
||||||
|
"log": [],
|
||||||
}
|
}
|
||||||
threading.Thread(target=_run, args=(job_id, pdf_path, project_input, text_local),
|
threading.Thread(
|
||||||
daemon=True).start()
|
target=_run,
|
||||||
|
args=(job_id, pdf_path, project_input, text_local, vision_model, text_model),
|
||||||
|
daemon=True,
|
||||||
|
).start()
|
||||||
return job_id
|
return job_id
|
||||||
|
|
||||||
|
|
||||||
def _run(job_id: str, pdf_path: str, project_input: Optional[Dict] = None,
|
def _run(
|
||||||
text_local: bool = False) -> None:
|
job_id: str,
|
||||||
|
pdf_path: str,
|
||||||
|
project_input: Optional[Dict] = None,
|
||||||
|
text_local: bool = False,
|
||||||
|
vision_model: Optional[str] = None,
|
||||||
|
text_model: Optional[str] = None,
|
||||||
|
) -> None:
|
||||||
out_dir = os.path.join(config.OUTPUT_DIR, job_id)
|
out_dir = os.path.join(config.OUTPUT_DIR, job_id)
|
||||||
|
log_path = os.path.join(out_dir, "job.log")
|
||||||
try:
|
try:
|
||||||
_set(job_id, status="running")
|
_set(job_id, status="running")
|
||||||
# Keep a copy of the source PDF so its sheets can be viewed later.
|
# Keep a copy of the source PDF so its sheets can be viewed later.
|
||||||
os.makedirs(out_dir, exist_ok=True)
|
os.makedirs(out_dir, exist_ok=True)
|
||||||
|
# Truncate any leftover log if job_id somehow collided (shouldn't).
|
||||||
|
with open(log_path, "w", encoding="utf-8"):
|
||||||
|
pass
|
||||||
shutil.copy2(pdf_path, os.path.join(out_dir, "source.pdf"))
|
shutil.copy2(pdf_path, os.path.join(out_dir, "source.pdf"))
|
||||||
|
|
||||||
|
def on_line(raw: str) -> None:
|
||||||
|
_append_log(job_id, raw, log_path)
|
||||||
|
|
||||||
|
with capture_stdio(on_line):
|
||||||
report = run_pipeline(
|
report = run_pipeline(
|
||||||
pdf_path,
|
pdf_path,
|
||||||
out_dir=out_dir,
|
out_dir=out_dir,
|
||||||
@@ -69,10 +119,17 @@ def _run(job_id: str, pdf_path: str, project_input: Optional[Dict] = None,
|
|||||||
project_input=project_input,
|
project_input=project_input,
|
||||||
source_name=_jobs[job_id].get("source"),
|
source_name=_jobs[job_id].get("source"),
|
||||||
text_local=text_local,
|
text_local=text_local,
|
||||||
|
vision_model=vision_model,
|
||||||
|
text_model=text_model,
|
||||||
)
|
)
|
||||||
_set(job_id, status="done", report=report, finished_at=time.time(), stage=None)
|
_set(job_id, status="done", report=report, finished_at=time.time(), stage=None)
|
||||||
_notify(job_id, report, out_dir)
|
_notify(job_id, report, out_dir)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
# Also land in the job log via print under the tee when possible.
|
||||||
|
try:
|
||||||
|
_append_log(job_id, f"[Jobs] Job {job_id} failed: {e}", log_path)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
print(f"[Jobs] Job {job_id} failed: {e}")
|
print(f"[Jobs] Job {job_id} failed: {e}")
|
||||||
_set(job_id, status="error", error=str(e), finished_at=time.time())
|
_set(job_id, status="error", error=str(e), finished_at=time.time())
|
||||||
_notify_error(job_id)
|
_notify_error(job_id)
|
||||||
@@ -103,10 +160,6 @@ def _notify_error(job_id: str) -> None:
|
|||||||
if not email:
|
if not email:
|
||||||
return
|
return
|
||||||
# Reuse the report mailer with a minimal error-shaped payload.
|
# Reuse the report mailer with a minimal error-shaped payload.
|
||||||
err_report = {
|
|
||||||
"source": job.get("source", ""),
|
|
||||||
"summary": {"conflicts_found": 0, "by_severity": {}, "disciplines": []},
|
|
||||||
}
|
|
||||||
try:
|
try:
|
||||||
from backend.email_sender import _smtp_ready, _send
|
from backend.email_sender import _smtp_ready, _send
|
||||||
from email.message import EmailMessage
|
from email.message import EmailMessage
|
||||||
@@ -121,6 +174,7 @@ def _notify_error(job_id: str) -> None:
|
|||||||
"Your conflict check did not complete.\n\n"
|
"Your conflict check did not complete.\n\n"
|
||||||
f"Drawing set: {job.get('source','')}\n"
|
f"Drawing set: {job.get('source','')}\n"
|
||||||
f"Error: {job.get('error','unknown')}\n\n"
|
f"Error: {job.get('error','unknown')}\n\n"
|
||||||
|
f"Review the run log at: {config.APP_BASE_URL.rstrip('/')}/?job={job_id}\n\n"
|
||||||
"Generated by Conflict Checker"
|
"Generated by Conflict Checker"
|
||||||
)
|
)
|
||||||
_send(msg)
|
_send(msg)
|
||||||
@@ -128,6 +182,25 @@ def _notify_error(job_id: str) -> None:
|
|||||||
print(f"[Email] Failed to send error notice: {e}")
|
print(f"[Email] Failed to send error notice: {e}")
|
||||||
|
|
||||||
|
|
||||||
|
def _log_from_disk(job_id: str) -> List[str]:
|
||||||
|
return read_log_file(os.path.join(config.OUTPUT_DIR, job_id, "job.log"))
|
||||||
|
|
||||||
|
|
||||||
|
def get_job_log(job_id: str) -> Optional[List[str]]:
|
||||||
|
"""Full job log lines, from memory or disk. None if job unknown."""
|
||||||
|
with _lock:
|
||||||
|
job = _jobs.get(job_id)
|
||||||
|
if job is not None:
|
||||||
|
return list(job.get("log") or [])
|
||||||
|
|
||||||
|
log = _log_from_disk(job_id)
|
||||||
|
# Job exists on disk if we have a log or a report artifact.
|
||||||
|
report_path = os.path.join(config.OUTPUT_DIR, job_id, "conflicts.json")
|
||||||
|
if log or os.path.isfile(report_path):
|
||||||
|
return log
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def get_job(job_id: str) -> Optional[Dict]:
|
def get_job(job_id: str) -> Optional[Dict]:
|
||||||
"""Public job view. Includes the full report only when done.
|
"""Public job view. Includes the full report only when done.
|
||||||
|
|
||||||
@@ -137,29 +210,46 @@ def get_job(job_id: str) -> Optional[Dict]:
|
|||||||
with _lock:
|
with _lock:
|
||||||
job = _jobs.get(job_id)
|
job = _jobs.get(job_id)
|
||||||
if job:
|
if job:
|
||||||
return dict(job)
|
out = dict(job)
|
||||||
|
log = list(job.get("log") or [])
|
||||||
|
out["log_tail"] = log[-_LOG_TAIL:]
|
||||||
|
# Full log on terminal states so the UI can show it without a
|
||||||
|
# second fetch; keep polls light while running.
|
||||||
|
if out.get("status") in ("done", "error"):
|
||||||
|
out["log"] = log
|
||||||
|
else:
|
||||||
|
out.pop("log", None)
|
||||||
|
return out
|
||||||
|
|
||||||
# Try loading from disk
|
# Try loading from disk
|
||||||
report_path = os.path.join(config.OUTPUT_DIR, job_id, "conflicts.json")
|
report_path = os.path.join(config.OUTPUT_DIR, job_id, "conflicts.json")
|
||||||
if not os.path.isfile(report_path):
|
log = _log_from_disk(job_id)
|
||||||
|
if not os.path.isfile(report_path) and not log:
|
||||||
return None
|
return None
|
||||||
try:
|
try:
|
||||||
import json
|
import json
|
||||||
|
report = None
|
||||||
|
if os.path.isfile(report_path):
|
||||||
with open(report_path, encoding="utf-8") as f:
|
with open(report_path, encoding="utf-8") as f:
|
||||||
report = json.load(f)
|
report = json.load(f)
|
||||||
source_pdf = os.path.join(config.OUTPUT_DIR, job_id, "source.pdf")
|
source_pdf = os.path.join(config.OUTPUT_DIR, job_id, "source.pdf")
|
||||||
|
status = "done" if report is not None else "error"
|
||||||
return {
|
return {
|
||||||
"job_id": job_id,
|
"job_id": job_id,
|
||||||
"status": "done",
|
"status": status,
|
||||||
"source": report.get("source", os.path.basename(report_path)),
|
"source": (report or {}).get("source", os.path.basename(report_path)),
|
||||||
"email": None,
|
"email": None,
|
||||||
"project_input": report.get("project_input", {}),
|
"project_input": (report or {}).get("project_input", {}),
|
||||||
"text_local": report.get("summary", {}).get("text_backend") == "local",
|
"text_local": (report or {}).get("summary", {}).get("text_backend") == "local",
|
||||||
|
"vision_model": None,
|
||||||
|
"text_model": None,
|
||||||
"stage": None,
|
"stage": None,
|
||||||
"created_at": os.path.getmtime(source_pdf) if os.path.isfile(source_pdf) else None,
|
"created_at": os.path.getmtime(source_pdf) if os.path.isfile(source_pdf) else None,
|
||||||
"finished_at": os.path.getmtime(report_path),
|
"finished_at": os.path.getmtime(report_path) if os.path.isfile(report_path) else None,
|
||||||
"report": report,
|
"report": report,
|
||||||
"error": None,
|
"error": None if report is not None else "Report missing; see job log",
|
||||||
|
"log": log,
|
||||||
|
"log_tail": log[-_LOG_TAIL:],
|
||||||
}
|
}
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f"[Jobs] Failed to load job {job_id} from disk: {e}")
|
print(f"[Jobs] Failed to load job {job_id} from disk: {e}")
|
||||||
|
|||||||
+18
-3
@@ -23,6 +23,9 @@ _clients: Dict[str, OpenAI] = {}
|
|||||||
# call_json when routing a no-image (text) call. Module-global mirrors the
|
# call_json when routing a no-image (text) call. Module-global mirrors the
|
||||||
# set_stage/cost pattern (single-user tool).
|
# set_stage/cost pattern (single-user tool).
|
||||||
_text_local = False
|
_text_local = False
|
||||||
|
# Optional per-run model overrides from the UI (empty = use config defaults).
|
||||||
|
_vision_model_override: Optional[str] = None
|
||||||
|
_text_model_override: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
def set_text_backend(local: bool) -> None:
|
def set_text_backend(local: bool) -> None:
|
||||||
@@ -30,6 +33,13 @@ def set_text_backend(local: bool) -> None:
|
|||||||
global _text_local
|
global _text_local
|
||||||
_text_local = bool(local)
|
_text_local = bool(local)
|
||||||
|
|
||||||
|
|
||||||
|
def set_model_overrides(vision: Optional[str] = None, text: Optional[str] = None) -> None:
|
||||||
|
"""Per-run OpenRouter model picks. None/blank clears back to config defaults."""
|
||||||
|
global _vision_model_override, _text_model_override
|
||||||
|
_vision_model_override = (vision or "").strip() or None
|
||||||
|
_text_model_override = (text or "").strip() or None
|
||||||
|
|
||||||
# --- per-job cost accounting -------------------------------------------------
|
# --- per-job cost accounting -------------------------------------------------
|
||||||
# OpenRouter returns the real USD cost of each call when we request usage
|
# OpenRouter returns the real USD cost of each call when we request usage
|
||||||
# accounting. We accumulate it in a module-level counter; the runner resets it
|
# accounting. We accumulate it in a module-level counter; the runner resets it
|
||||||
@@ -167,12 +177,16 @@ def _resolve_backend(has_images: bool, model_override: Optional[str]) -> Dict[st
|
|||||||
return {
|
return {
|
||||||
"base_url": config.LOCAL_BASE_URL,
|
"base_url": config.LOCAL_BASE_URL,
|
||||||
"api_key": config.LOCAL_API_KEY,
|
"api_key": config.LOCAL_API_KEY,
|
||||||
"model": model_override or config.LOCAL_TEXT_MODEL or config.TEXT_MODEL,
|
"model": (model_override or _text_model_override
|
||||||
|
or config.LOCAL_TEXT_MODEL or config.TEXT_MODEL),
|
||||||
"usage": False, # local has no OpenRouter usage accounting
|
"usage": False, # local has no OpenRouter usage accounting
|
||||||
"local": True,
|
"local": True,
|
||||||
}
|
}
|
||||||
# Vision, or text-on-OpenRouter (default / fallback).
|
# Vision, or text-on-OpenRouter (default / fallback).
|
||||||
default_model = config.MODEL if has_images else config.TEXT_MODEL
|
if has_images:
|
||||||
|
default_model = _vision_model_override or config.MODEL
|
||||||
|
else:
|
||||||
|
default_model = _text_model_override or config.TEXT_MODEL
|
||||||
return {
|
return {
|
||||||
"base_url": config.AI_BASE_URL,
|
"base_url": config.AI_BASE_URL,
|
||||||
"api_key": config.AI_API_KEY,
|
"api_key": config.AI_API_KEY,
|
||||||
@@ -340,7 +354,8 @@ def call_json(
|
|||||||
_models["text_local"].add(be["model"])
|
_models["text_local"].add(be["model"])
|
||||||
_models["fallback_count"] += 1
|
_models["fallback_count"] += 1
|
||||||
be = {"base_url": config.AI_BASE_URL, "api_key": config.AI_API_KEY,
|
be = {"base_url": config.AI_BASE_URL, "api_key": config.AI_API_KEY,
|
||||||
"model": config.TEXT_MODEL, "usage": True, "local": False}
|
"model": _text_model_override or config.TEXT_MODEL,
|
||||||
|
"usage": True, "local": False}
|
||||||
cache_key = None # don't cache fallback under the local-model key
|
cache_key = None # don't cache fallback under the local-model key
|
||||||
fell_back = True
|
fell_back = True
|
||||||
continue
|
continue
|
||||||
|
|||||||
+37
-4
@@ -14,11 +14,12 @@ import tempfile
|
|||||||
from typing import Optional
|
from typing import Optional
|
||||||
|
|
||||||
from fastapi import FastAPI, UploadFile, File, Form, HTTPException
|
from fastapi import FastAPI, UploadFile, File, Form, HTTPException
|
||||||
from fastapi.responses import HTMLResponse, JSONResponse, Response
|
from fastapi.responses import HTMLResponse, JSONResponse, Response, PlainTextResponse
|
||||||
from fastapi.staticfiles import StaticFiles
|
from fastapi.staticfiles import StaticFiles
|
||||||
|
|
||||||
from backend import config
|
from backend import config
|
||||||
from backend.jobs import create_job, get_job
|
from backend.jobs import create_job, get_job, get_job_log
|
||||||
|
from backend.models_catalog import list_models
|
||||||
from backend.pipeline.pdf_processor import render_page_jpeg
|
from backend.pipeline.pdf_processor import render_page_jpeg
|
||||||
|
|
||||||
app = FastAPI(title=config.APP_TITLE, version=config.APP_VERSION)
|
app = FastAPI(title=config.APP_TITLE, version=config.APP_VERSION)
|
||||||
@@ -29,10 +30,17 @@ _FRONTEND_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "
|
|||||||
@app.get("/health")
|
@app.get("/health")
|
||||||
def health():
|
def health():
|
||||||
return {"status": "ok", "model": config.MODEL,
|
return {"status": "ok", "model": config.MODEL,
|
||||||
|
"text_model": config.TEXT_MODEL,
|
||||||
"key_configured": bool(config.AI_API_KEY),
|
"key_configured": bool(config.AI_API_KEY),
|
||||||
"email_configured": bool(config.SMTP_HOST and config.SMTP_USER and config.SMTP_PASSWORD)}
|
"email_configured": bool(config.SMTP_HOST and config.SMTP_USER and config.SMTP_PASSWORD)}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/models")
|
||||||
|
def models():
|
||||||
|
"""Vision vs text OpenRouter model lists for the UI dropdowns."""
|
||||||
|
return JSONResponse(list_models())
|
||||||
|
|
||||||
|
|
||||||
@app.post("/check")
|
@app.post("/check")
|
||||||
async def check(
|
async def check(
|
||||||
file: UploadFile = File(...),
|
file: UploadFile = File(...),
|
||||||
@@ -42,6 +50,8 @@ async def check(
|
|||||||
occupancy: Optional[str] = Form(None),
|
occupancy: Optional[str] = Form(None),
|
||||||
work_type: Optional[str] = Form(None),
|
work_type: Optional[str] = Form(None),
|
||||||
text_local: bool = Form(False),
|
text_local: bool = Form(False),
|
||||||
|
vision_model: Optional[str] = Form(None),
|
||||||
|
text_model: Optional[str] = Form(None),
|
||||||
):
|
):
|
||||||
"""
|
"""
|
||||||
Accept a PDF, start a background conflict check, and return a job_id
|
Accept a PDF, start a background conflict check, and return a job_id
|
||||||
@@ -50,6 +60,9 @@ async def check(
|
|||||||
Optional intake fields (project_name/address/occupancy/work_type) feed the
|
Optional intake fields (project_name/address/occupancy/work_type) feed the
|
||||||
Stage 0 jurisdiction profile; anything left blank is derived from the cover
|
Stage 0 jurisdiction profile; anything left blank is derived from the cover
|
||||||
sheet.
|
sheet.
|
||||||
|
|
||||||
|
vision_model / text_model override the configured defaults for this run
|
||||||
|
(vision always OpenRouter; text follows the OpenRouter vs hybrid choice).
|
||||||
"""
|
"""
|
||||||
if not file.filename.lower().endswith(".pdf"):
|
if not file.filename.lower().endswith(".pdf"):
|
||||||
raise HTTPException(status_code=400, detail="Please upload a PDF.")
|
raise HTTPException(status_code=400, detail="Please upload a PDF.")
|
||||||
@@ -69,8 +82,17 @@ async def check(
|
|||||||
}.items()
|
}.items()
|
||||||
if v and v.strip()
|
if v and v.strip()
|
||||||
}
|
}
|
||||||
job_id = create_job(tmp_path, source_filename=file.filename, email=email,
|
v_model = (vision_model or "").strip() or None
|
||||||
project_input=project_input, text_local=text_local)
|
t_model = (text_model or "").strip() or None
|
||||||
|
job_id = create_job(
|
||||||
|
tmp_path,
|
||||||
|
source_filename=file.filename,
|
||||||
|
email=email,
|
||||||
|
project_input=project_input,
|
||||||
|
text_local=text_local,
|
||||||
|
vision_model=v_model,
|
||||||
|
text_model=t_model,
|
||||||
|
)
|
||||||
return JSONResponse({"job_id": job_id, "status": "queued", "email": email})
|
return JSONResponse({"job_id": job_id, "status": "queued", "email": email})
|
||||||
|
|
||||||
|
|
||||||
@@ -82,6 +104,17 @@ def job_status(job_id: str):
|
|||||||
return JSONResponse(job)
|
return JSONResponse(job)
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/jobs/{job_id}/log")
|
||||||
|
def job_log(job_id: str, plain: bool = False):
|
||||||
|
"""Full captured run log (also on disk as outputs/<job_id>/job.log)."""
|
||||||
|
lines = get_job_log(job_id)
|
||||||
|
if lines is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Job not found")
|
||||||
|
if plain:
|
||||||
|
return PlainTextResponse("\n".join(lines) + ("\n" if lines else ""))
|
||||||
|
return JSONResponse({"job_id": job_id, "lines": lines, "text": "\n".join(lines)})
|
||||||
|
|
||||||
|
|
||||||
@app.get("/jobs/{job_id}/sheet-image/{page}")
|
@app.get("/jobs/{job_id}/sheet-image/{page}")
|
||||||
def sheet_image(job_id: str, page: int):
|
def sheet_image(job_id: str, page: int):
|
||||||
"""Render one page of a completed job's source PDF as JPEG (sheet viewer)."""
|
"""Render one page of a completed job's source PDF as JPEG (sheet viewer)."""
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
"""
|
||||||
|
models_catalog.py - OpenRouter model list for the UI dropdowns.
|
||||||
|
|
||||||
|
Fetches https://openrouter.ai/api/v1/models (cached ~1h) and splits into:
|
||||||
|
- vision: accepts image input and returns text
|
||||||
|
- text: chat models that return text (may also be multimodal)
|
||||||
|
"""
|
||||||
|
|
||||||
|
import time
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
|
||||||
|
from backend import config
|
||||||
|
|
||||||
|
_TTL_SEC = 3600
|
||||||
|
_cache: Dict[str, Any] = {"at": 0.0, "payload": None}
|
||||||
|
|
||||||
|
|
||||||
|
def _entry(m: Dict[str, Any]) -> Dict[str, str]:
|
||||||
|
return {
|
||||||
|
"id": m.get("id") or "",
|
||||||
|
"name": m.get("name") or m.get("id") or "",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_default(items: List[Dict[str, str]], model_id: str) -> List[Dict[str, str]]:
|
||||||
|
"""Prepend the configured default if OpenRouter didn't return it."""
|
||||||
|
if not model_id:
|
||||||
|
return items
|
||||||
|
if any(x["id"] == model_id for x in items):
|
||||||
|
return items
|
||||||
|
return [{"id": model_id, "name": model_id}] + items
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch_raw() -> List[Dict[str, Any]]:
|
||||||
|
import httpx # local import so the app can start without httpx in odd envs
|
||||||
|
headers = {"Accept": "application/json"}
|
||||||
|
if config.AI_API_KEY:
|
||||||
|
headers["Authorization"] = f"Bearer {config.AI_API_KEY}"
|
||||||
|
url = f"{config.AI_BASE_URL.rstrip('/')}/models"
|
||||||
|
# Ask for text-output chat models (includes multimodal). "all" is huge.
|
||||||
|
with httpx.Client(timeout=30.0) as client:
|
||||||
|
r = client.get(url, headers=headers, params={"output_modalities": "text"})
|
||||||
|
r.raise_for_status()
|
||||||
|
data = r.json()
|
||||||
|
return data.get("data") or []
|
||||||
|
|
||||||
|
|
||||||
|
def list_models() -> Dict[str, Any]:
|
||||||
|
"""Return {vision, text, defaults} for the frontend selects."""
|
||||||
|
now = time.time()
|
||||||
|
if _cache["payload"] and (now - _cache["at"]) < _TTL_SEC:
|
||||||
|
return _cache["payload"]
|
||||||
|
|
||||||
|
try:
|
||||||
|
raw = _fetch_raw()
|
||||||
|
except Exception as e:
|
||||||
|
# Degrade to configured defaults so the UI still works offline.
|
||||||
|
print(f"[Models] OpenRouter catalog fetch failed: {e}")
|
||||||
|
vision = _ensure_default([], config.MODEL)
|
||||||
|
text = _ensure_default([], config.TEXT_MODEL)
|
||||||
|
payload = {
|
||||||
|
"vision": vision,
|
||||||
|
"text": text,
|
||||||
|
"defaults": {"vision": config.MODEL, "text": config.TEXT_MODEL},
|
||||||
|
"error": str(e),
|
||||||
|
}
|
||||||
|
return payload
|
||||||
|
|
||||||
|
vision: List[Dict[str, str]] = []
|
||||||
|
text: List[Dict[str, str]] = []
|
||||||
|
for m in raw:
|
||||||
|
mid = m.get("id") or ""
|
||||||
|
if not mid:
|
||||||
|
continue
|
||||||
|
arch = m.get("architecture") or {}
|
||||||
|
inputs = arch.get("input_modalities") or []
|
||||||
|
outputs = arch.get("output_modalities") or []
|
||||||
|
# Legacy string form: "text+image->text"
|
||||||
|
modality = (arch.get("modality") or "").lower()
|
||||||
|
has_image_in = ("image" in inputs) or ("image" in modality.split("->")[0])
|
||||||
|
has_text_out = ("text" in outputs) or ("->text" in modality) or (not outputs and not modality)
|
||||||
|
has_text_in = ("text" in inputs) or ("text" in modality) or not inputs
|
||||||
|
|
||||||
|
if has_image_in and has_text_out:
|
||||||
|
vision.append(_entry(m))
|
||||||
|
if has_text_in and has_text_out:
|
||||||
|
text.append(_entry(m))
|
||||||
|
|
||||||
|
vision.sort(key=lambda x: x["name"].lower())
|
||||||
|
text.sort(key=lambda x: x["name"].lower())
|
||||||
|
vision = _ensure_default(vision, config.MODEL)
|
||||||
|
text = _ensure_default(text, config.TEXT_MODEL)
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"vision": vision,
|
||||||
|
"text": text,
|
||||||
|
"defaults": {"vision": config.MODEL, "text": config.TEXT_MODEL},
|
||||||
|
}
|
||||||
|
_cache["at"] = now
|
||||||
|
_cache["payload"] = payload
|
||||||
|
return payload
|
||||||
@@ -42,7 +42,9 @@ from backend.pipeline.risk import score_and_prioritize
|
|||||||
from backend.pipeline.rfi import generate_rfis
|
from backend.pipeline.rfi import generate_rfis
|
||||||
from backend.pipeline.report import build_report, to_markdown
|
from backend.pipeline.report import build_report, to_markdown
|
||||||
from backend.pipeline._stage import validate_issue
|
from backend.pipeline._stage import validate_issue
|
||||||
from backend.llm import reset_cost, get_cost, set_stage, set_text_backend
|
from backend.llm import (
|
||||||
|
reset_cost, get_cost, set_stage, set_text_backend, set_model_overrides,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def run_pipeline(
|
def run_pipeline(
|
||||||
@@ -52,6 +54,8 @@ def run_pipeline(
|
|||||||
project_input: Optional[Dict] = None,
|
project_input: Optional[Dict] = None,
|
||||||
source_name: Optional[str] = None,
|
source_name: Optional[str] = None,
|
||||||
text_local: bool = False,
|
text_local: bool = False,
|
||||||
|
vision_model: Optional[str] = None,
|
||||||
|
text_model: Optional[str] = None,
|
||||||
) -> Dict:
|
) -> Dict:
|
||||||
"""
|
"""
|
||||||
Run the full QAQC pipeline on one PDF and return the report dict.
|
Run the full QAQC pipeline on one PDF and return the report dict.
|
||||||
@@ -59,6 +63,9 @@ def run_pipeline(
|
|||||||
project_input: optional intake fields (project_name, address, occupancy,
|
project_input: optional intake fields (project_name, address, occupancy,
|
||||||
work_type). Cover-sheet-derived values fill any gaps; intake fields win.
|
work_type). Cover-sheet-derived values fill any gaps; intake fields win.
|
||||||
|
|
||||||
|
vision_model / text_model: optional per-run OpenRouter (or local text)
|
||||||
|
model overrides from the UI. Blank/None keeps config defaults.
|
||||||
|
|
||||||
If out_dir is given, writes conflicts.json, report.md, and the intermediate
|
If out_dir is given, writes conflicts.json, report.md, and the intermediate
|
||||||
artifacts (assertions.json, clusters.json, and one json per QAQC stage).
|
artifacts (assertions.json, clusters.json, and one json per QAQC stage).
|
||||||
"""
|
"""
|
||||||
@@ -70,7 +77,29 @@ def run_pipeline(
|
|||||||
|
|
||||||
reset_cost()
|
reset_cost()
|
||||||
set_text_backend(text_local)
|
set_text_backend(text_local)
|
||||||
|
set_model_overrides(vision_model, text_model)
|
||||||
|
if vision_model or text_model:
|
||||||
|
print(f"[Runner] model overrides: vision={vision_model or '(default)'} "
|
||||||
|
f"text={text_model or '(default)'}")
|
||||||
|
|
||||||
|
try:
|
||||||
|
return _run_stages(
|
||||||
|
pdf_path, out_dir, stage, project_input, source_name, text_local,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
# Don't leak per-run picks into a later overlapping/CLI call.
|
||||||
|
set_model_overrides(None, None)
|
||||||
|
set_text_backend(False)
|
||||||
|
|
||||||
|
|
||||||
|
def _run_stages(
|
||||||
|
pdf_path: str,
|
||||||
|
out_dir: Optional[str],
|
||||||
|
stage: Callable[[str], None],
|
||||||
|
project_input: Optional[Dict],
|
||||||
|
source_name: Optional[str],
|
||||||
|
text_local: bool,
|
||||||
|
) -> Dict:
|
||||||
stage("PDF -> images")
|
stage("PDF -> images")
|
||||||
pages = convert_pdf_to_images(pdf_path)
|
pages = convert_pdf_to_images(pdf_path)
|
||||||
|
|
||||||
|
|||||||
+87
-2
@@ -60,7 +60,15 @@
|
|||||||
border:1px solid var(--line); background:#0c0e13; color:var(--text); font-size:16px; outline:none; }
|
border:1px solid var(--line); background:#0c0e13; color:var(--text); font-size:16px; outline:none; }
|
||||||
.email-card input[type=email]::placeholder { color:#6b7280; }
|
.email-card input[type=email]::placeholder { color:#6b7280; }
|
||||||
.email-card input[type=email]:focus { border-color:var(--accent); box-shadow:0 0 0 3px rgba(91,140,255,.18); }
|
.email-card input[type=email]:focus { border-color:var(--accent); box-shadow:0 0 0 3px rgba(91,140,255,.18); }
|
||||||
|
.email-card select { width:100%; padding:10px 12px; border-radius:8px; margin-top:6px;
|
||||||
|
border:1px solid var(--line); background:#0c0e13; color:var(--text); font-size:14px; }
|
||||||
|
.email-card .field { margin-top:12px; }
|
||||||
|
.email-card .field > span { display:block; font-size:13px; color:var(--muted); margin-bottom:2px; }
|
||||||
.btn.full { width:100%; padding:14px; font-size:15px; margin-top:0; }
|
.btn.full { width:100%; padding:14px; font-size:15px; margin-top:0; }
|
||||||
|
.logbox { background:#0c0e13; border:1px solid var(--line); border-radius:8px; padding:12px 14px;
|
||||||
|
margin-top:10px; max-height:320px; overflow:auto; font:12px/1.45 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;
|
||||||
|
color:#c6cdd8; white-space:pre-wrap; word-break:break-word; }
|
||||||
|
.logbox .empty-log { color:var(--muted); }
|
||||||
.note { background:var(--panel); border:1px solid var(--line); border-radius:10px;
|
.note { background:var(--panel); border:1px solid var(--line); border-radius:10px;
|
||||||
padding:16px 18px; margin:18px 0; }
|
padding:16px 18px; margin:18px 0; }
|
||||||
.note b { color:var(--text); }
|
.note b { color:var(--text); }
|
||||||
@@ -103,9 +111,21 @@
|
|||||||
<input type="radio" name="compute" value="openrouter" checked> OpenRouter — all stages (fastest, paid)</label>
|
<input type="radio" name="compute" value="openrouter" checked> OpenRouter — all stages (fastest, paid)</label>
|
||||||
<label style="display:block;font-weight:400;margin-top:6px">
|
<label style="display:block;font-weight:400;margin-top:6px">
|
||||||
<input type="radio" name="compute" value="local"> Hybrid — text stages on local LLM (cheaper, slower)</label>
|
<input type="radio" name="compute" value="local"> Hybrid — text stages on local LLM (cheaper, slower)</label>
|
||||||
|
<div class="field">
|
||||||
|
<span>Vision model <span class="opt">(image stages)</span></span>
|
||||||
|
<select id="vision_model" disabled><option value="">Loading models…</option></select>
|
||||||
|
</div>
|
||||||
|
<div class="field">
|
||||||
|
<span>Text model <span class="opt">(non-image stages / hybrid fallback)</span></span>
|
||||||
|
<select id="text_model" disabled><option value="">Loading models…</option></select>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<button class="btn full" id="run" disabled>Run conflict check</button>
|
<button class="btn full" id="run" disabled>Run conflict check</button>
|
||||||
<div class="status" id="status"></div>
|
<div class="status" id="status"></div>
|
||||||
|
<div id="liveLog" style="display:none" class="note">
|
||||||
|
<b>Run log</b> <span class="opt" id="logHint">(updates live)</span>
|
||||||
|
<pre class="logbox" id="logBox"><span class="empty-log">Waiting for output…</span></pre>
|
||||||
|
</div>
|
||||||
<div id="results"></div>
|
<div id="results"></div>
|
||||||
</main>
|
</main>
|
||||||
<div id="viewer">
|
<div id="viewer">
|
||||||
@@ -123,9 +143,61 @@
|
|||||||
const drop=document.getElementById('drop'), fileInput=document.getElementById('file'),
|
const drop=document.getElementById('drop'), fileInput=document.getElementById('file'),
|
||||||
runBtn=document.getElementById('run'), statusEl=document.getElementById('status'),
|
runBtn=document.getElementById('run'), statusEl=document.getElementById('status'),
|
||||||
results=document.getElementById('results'), dropLabel=document.getElementById('dropLabel'),
|
results=document.getElementById('results'), dropLabel=document.getElementById('dropLabel'),
|
||||||
emailEl=document.getElementById('email');
|
emailEl=document.getElementById('email'),
|
||||||
|
visionSel=document.getElementById('vision_model'),
|
||||||
|
textSel=document.getElementById('text_model'),
|
||||||
|
liveLog=document.getElementById('liveLog'),
|
||||||
|
logBox=document.getElementById('logBox'),
|
||||||
|
logHint=document.getElementById('logHint');
|
||||||
let chosen=null, polling=null, currentJobId=null, sheetPage={}, viewerZoom=1;
|
let chosen=null, polling=null, currentJobId=null, sheetPage={}, viewerZoom=1;
|
||||||
|
|
||||||
|
function fillSelect(sel, items, preferred){
|
||||||
|
sel.innerHTML='';
|
||||||
|
(items||[]).forEach(m=>{
|
||||||
|
const opt=document.createElement('option');
|
||||||
|
opt.value=m.id; opt.textContent=m.name||m.id;
|
||||||
|
if(m.id===preferred) opt.selected=true;
|
||||||
|
sel.appendChild(opt);
|
||||||
|
});
|
||||||
|
if(!sel.options.length){
|
||||||
|
const opt=document.createElement('option');
|
||||||
|
opt.value=preferred||''; opt.textContent=preferred||'(no models)';
|
||||||
|
sel.appendChild(opt);
|
||||||
|
}
|
||||||
|
sel.disabled=false;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadModels(){
|
||||||
|
try{
|
||||||
|
const res=await fetch('/models');
|
||||||
|
if(!res.ok) throw new Error('models HTTP '+res.status);
|
||||||
|
const data=await res.json();
|
||||||
|
const defs=data.defaults||{};
|
||||||
|
fillSelect(visionSel, data.vision, defs.vision);
|
||||||
|
fillSelect(textSel, data.text, defs.text);
|
||||||
|
if(data.error){
|
||||||
|
console.warn('Model catalog degraded:', data.error);
|
||||||
|
}
|
||||||
|
}catch(err){
|
||||||
|
visionSel.innerHTML='<option value="">(default)</option>';
|
||||||
|
textSel.innerHTML='<option value="">(default)</option>';
|
||||||
|
visionSel.disabled=false; textSel.disabled=false;
|
||||||
|
console.warn('Could not load models:', err);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function showLog(lines, live){
|
||||||
|
liveLog.style.display='block';
|
||||||
|
logHint.textContent=live?'(updates live)':'(saved with this job)';
|
||||||
|
const arr=lines||[];
|
||||||
|
if(!arr.length){
|
||||||
|
logBox.innerHTML='<span class="empty-log">No log lines yet…</span>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
logBox.textContent=arr.join('\n');
|
||||||
|
logBox.scrollTop=logBox.scrollHeight;
|
||||||
|
}
|
||||||
|
|
||||||
function setFile(f){ chosen=f; dropLabel.textContent=f?('Selected: '+f.name):'Drop a PDF drawing set here, or click to choose';
|
function setFile(f){ chosen=f; dropLabel.textContent=f?('Selected: '+f.name):'Drop a PDF drawing set here, or click to choose';
|
||||||
runBtn.disabled=!f; }
|
runBtn.disabled=!f; }
|
||||||
dropLabel.addEventListener('click',()=>fileInput.click());
|
dropLabel.addEventListener('click',()=>fileInput.click());
|
||||||
@@ -139,6 +211,7 @@ runBtn.addEventListener('click',async e=>{
|
|||||||
if(!chosen) return;
|
if(!chosen) return;
|
||||||
runBtn.disabled=true; results.innerHTML='';
|
runBtn.disabled=true; results.innerHTML='';
|
||||||
statusEl.innerHTML='<span class="spinner"></span>Uploading...';
|
statusEl.innerHTML='<span class="spinner"></span>Uploading...';
|
||||||
|
showLog([], true);
|
||||||
const fd=new FormData(); fd.append('file',chosen);
|
const fd=new FormData(); fd.append('file',chosen);
|
||||||
const email=(emailEl.value||'').trim(); if(email) fd.append('notification_email',email);
|
const email=(emailEl.value||'').trim(); if(email) fd.append('notification_email',email);
|
||||||
['project_name','address','occupancy','work_type'].forEach(id=>{
|
['project_name','address','occupancy','work_type'].forEach(id=>{
|
||||||
@@ -146,6 +219,8 @@ runBtn.addEventListener('click',async e=>{
|
|||||||
});
|
});
|
||||||
const compute=(document.querySelector('input[name="compute"]:checked')||{}).value;
|
const compute=(document.querySelector('input[name="compute"]:checked')||{}).value;
|
||||||
fd.append('text_local', compute==='local' ? 'true' : 'false');
|
fd.append('text_local', compute==='local' ? 'true' : 'false');
|
||||||
|
if(visionSel.value) fd.append('vision_model', visionSel.value);
|
||||||
|
if(textSel.value) fd.append('text_model', textSel.value);
|
||||||
try{
|
try{
|
||||||
const res=await fetch('/check',{method:'POST',body:fd});
|
const res=await fetch('/check',{method:'POST',body:fd});
|
||||||
if(!res.ok){ const err=await res.json().catch(()=>({detail:res.statusText}));
|
if(!res.ok){ const err=await res.json().catch(()=>({detail:res.statusText}));
|
||||||
@@ -169,14 +244,19 @@ function poll(jobId){
|
|||||||
const res=await fetch('/jobs/'+jobId);
|
const res=await fetch('/jobs/'+jobId);
|
||||||
if(!res.ok) throw new Error('job not found');
|
if(!res.ok) throw new Error('job not found');
|
||||||
const job=await res.json();
|
const job=await res.json();
|
||||||
|
if(job.log_tail && job.log_tail.length) showLog(job.log_tail, job.status==='running'||job.status==='queued');
|
||||||
if(job.status==='running'||job.status==='queued'){
|
if(job.status==='running'||job.status==='queued'){
|
||||||
statusEl.innerHTML='<span class="spinner"></span>'+esc(job.stage||'Working...')+
|
statusEl.innerHTML='<span class="spinner"></span>'+esc(job.stage||'Working...')+
|
||||||
' · you can leave this page';
|
' · you can leave this page';
|
||||||
} else if(job.status==='done'){
|
} else if(job.status==='done'){
|
||||||
clearInterval(polling); polling=null; runBtn.disabled=false; render(job.report);
|
clearInterval(polling); polling=null; runBtn.disabled=false;
|
||||||
|
if(job.log && job.log.length) showLog(job.log, false);
|
||||||
|
render(job.report);
|
||||||
} else if(job.status==='error'){
|
} else if(job.status==='error'){
|
||||||
clearInterval(polling); polling=null; runBtn.disabled=false;
|
clearInterval(polling); polling=null; runBtn.disabled=false;
|
||||||
statusEl.textContent='Run failed: '+(job.error||'unknown error');
|
statusEl.textContent='Run failed: '+(job.error||'unknown error');
|
||||||
|
if(job.log && job.log.length) showLog(job.log, false);
|
||||||
|
else if(job.log_tail && job.log_tail.length) showLog(job.log_tail, false);
|
||||||
}
|
}
|
||||||
}catch(err){ clearInterval(polling); polling=null; runBtn.disabled=false;
|
}catch(err){ clearInterval(polling); polling=null; runBtn.disabled=false;
|
||||||
statusEl.textContent='Error: '+err.message; }
|
statusEl.textContent='Error: '+err.message; }
|
||||||
@@ -286,12 +366,17 @@ function render(rep){
|
|||||||
html+='</details>';
|
html+='</details>';
|
||||||
}
|
}
|
||||||
|
|
||||||
|
html+='<div class="meta" style="margin-top:14px">Full run log: <a href="/jobs/'+
|
||||||
|
esc(currentJobId)+'/log?plain=1" target="_blank" rel="noopener">/jobs/'+
|
||||||
|
esc(currentJobId)+'/log</a> (also saved as job.log on the server)</div>';
|
||||||
|
|
||||||
results.innerHTML=html;
|
results.innerHTML=html;
|
||||||
}
|
}
|
||||||
function stat(v,l){ return '<div class="stat"><b>'+esc(v)+'</b><span>'+esc(l)+'</span></div>'; }
|
function stat(v,l){ return '<div class="stat"><b>'+esc(v)+'</b><span>'+esc(l)+'</span></div>'; }
|
||||||
|
|
||||||
// If opened from an email link (/?job=<id>), load that job's results directly.
|
// If opened from an email link (/?job=<id>), load that job's results directly.
|
||||||
(function init(){
|
(function init(){
|
||||||
|
loadModels();
|
||||||
const jobId=new URLSearchParams(location.search).get('job');
|
const jobId=new URLSearchParams(location.search).get('job');
|
||||||
if(jobId){ statusEl.innerHTML='<span class="spinner"></span>Loading job '+esc(jobId)+'...'; poll(jobId); }
|
if(jobId){ statusEl.innerHTML='<span class="spinner"></span>Loading job '+esc(jobId)+'...'; poll(jobId); }
|
||||||
})();
|
})();
|
||||||
|
|||||||
Reference in New Issue
Block a user