Conflict_Checker/NOTES.md

# Session Notes — Conflict Checker

Orientation for a new coding session. Setup and Docker details live in [README.md](README.md). This file tracks what the code actually does and what tends to waste time.

**Last updated:** 2026-08-02 · `agent-mode` branch (merged `main` tip `bf508bf`)

## What this is

Cross-discipline **design contradiction / senior-architect QAQC** for construction drawing PDFs (Arch, Struct, Mech, Elec, Plumb, FP, etc.). Flags disagreements *between disciplines* before a set goes to bid/permit.

**Not** IronBid’s scope-ownership conflict checker.

Source of truth: Scout IT Gitea — `gitea.scoutitsystems.com/woogi/Conflict_Checker`.

## Stack

| Layer | Detail |
|-------|--------|
| API | Python 3.12, FastAPI, Uvicorn ([backend/main.py](backend/main.py)) |
| UI | Single static file [frontend/index.html](frontend/index.html), served by FastAPI |
| Pipelines | **Classic:** [backend/pipeline/runner.py](backend/pipeline/runner.py) (shared by web + CLI). **Agent:** [backend/agents/runner.py](backend/agents/runner.py) — experimental scoped specialist agents, selected per job (`pipeline_mode`) |
| Review gate | Agent jobs stop at `needs_review` for human decisions before the report emails ([backend/review/](backend/review/)) |
| LLM | OpenRouter via `openai` SDK; default `google/gemini-2.5-pro`. Vision always OpenRouter; text stages can use local vLLM (classic/hybrid only) |
| PDF | `pdf2image` + system `poppler-utils` → JPEG page images |
| Jobs | In-memory threads ([backend/jobs.py](backend/jobs.py)) — no Redis/DB |
| Tests | `pytest tests/` (~82 tests; see Quick start) |
| Deploy | Docker Compose; app on port **8099**; public URL `https://conchecker.scoutitsystems.com` |

## Live pipeline (authoritative)

README still describes an older 5-stage extract-then-compare loop. **Trust `runner.py`.** Actual flow:

```
PDF → images → extract → sheet index → jurisdiction
    → normalize → project intelligence (GOIDs)
    → cluster → conflict reason
    → QAQC / code / constructability
    → dedup-validate → risk → RFIs → report
```

| Runner stage | Module | Notes |
|--------------|--------|--------|
| PDF → images | `pdf_processor` | Rasterize |
| Extract assertions | `extractor` | Vision, per sheet |
| Classify sheet index | `sheet_index` | LLM |
| Jurisdiction profile | `jurisdiction` | After cover meta |
| Normalize | `normalizer` | LLM batches |
| Project intelligence | `normalizer.build_project_intelligence` | GOIDs + relationships |
| Cluster | `llm_clusterer` or `clusterer` | Default `CLUSTERER=llm` |
| Conflicts | `conflict_checker` | Per-cluster vision reason |
| QAQC / code / constructability | `qaqc_review`, `code_review`, `constructability` | Full-set / batched |
| Validate & dedup | `validator` | Merges conflict + QAQC + code + construct issues |
| Risk / RFIs | `risk`, `rfi` | Text-only |
| Report | `report` | `conflicts.json` + `report.md` (+ stage JSON dumps when `out_dir` set) |

Design notes for Stage 2/3 engines also live under `Changes/*.docx`.

**Agent pipeline** (`pipeline_mode=agent`): scoped specialist agents in [backend/agents/](backend/agents/) (extractors, linker, brain, critics, RFI writer) run through `Orchestrator` + `ProjectMemory`; artifacts under `outputs/<job_id>/agent/`. Agent mode is OpenRouter-only (no hybrid) and, when `AGENT_REQUIRE_REVIEW=true`, stops at `needs_review` until a human saves decisions and finalizes via the review endpoints. Design docs: `docs/superpowers/`.

## HTTP API (current)

| Method | Path | Purpose |
|--------|------|---------|
| GET | `/health` | Liveness + `model` / `text_model` + `version` / `build` + key/email flags |
| GET | `/models` | OpenRouter catalog split into `vision[]` / `text[]` + `defaults`, with per-1M-token pricing (cached ~1h; **502** when OpenRouter is unreachable) |
| POST | `/check` | Upload PDF; returns `{job_id}` immediately |
| GET | `/jobs/{id}` | Status poll. Running: `stage` + `log_tail`. Done/error/needs_review/finalization_error: `report` and/or `error` + full `log` |
| GET | `/jobs/{id}/log` | Full run log as `text/plain` (404 when no log file) |
| GET | `/jobs/{id}/review` | Review queue + progress + saved decisions (agent jobs) |
| POST | `/jobs/{id}/review-decisions` | Save reviewer decisions (409 outside needs_review/reviewing) |
| POST | `/jobs/{id}/finalize-review` | Background finalize + send report (409 unless review gate passed) |
| GET | `/jobs/{id}/sheet-image/{page}` | JPEG of source PDF page for the sheet viewer |
| GET | `/` | Serves `frontend/index.html` |

### `/check` form fields

- Required: `file` (PDF)
- Optional: `notification_email`, `project_name`, `address`, `occupancy`, `work_type`
- Pipeline: `pipeline_mode` (`classic` default, or `agent`)
- Compute: `text_local` (`true` = hybrid local text; forced off for agent mode)
- Models: `vision_model`, `text_model` (OpenRouter ids; blank = config defaults)

## Job logs

Pipeline `print()` is teed for the job thread ([backend/job_log.py](backend/job_log.py)):

- Live: `GET /jobs/{id}` → `log_tail` (last 80 lines)
- Done/error/needs_review/finalization_error: same payload includes full `log`
- Disk: `backend/outputs/<job_id>/job.log` (survives restart; status registry does not)
- API: `GET /jobs/{id}/log` → `text/plain`
- UI: “Run log” panel updates while running; stays visible after finish/fail/review
- Failures append the **full traceback** to the log; each run starts with a header line (job id, mode, models, start time)
- `outputs/<job_id>/job.json` (written at start) carries email/mode/models so the disk fallback can rebuild a job after restart
- **Verbose LLM observability** (`LLM_VERBOSE=true`, default on): one `[LLM] #NNNN stage | model (backend) | in/out sizes | cost | parsed-item counts` line per call in `job.log` — list-valued keys show counts (`conflicts[3]`) so empty (miss) or invented (hallucination) results stand out
- **Raw dumps** (`LLM_RAW_DUMP=true`, default on): full prompt + raw response per call in `outputs/<job_id>/llm_raw/NNNN_stage_model.json` (base64 images excluded, `n_images` recorded). This is the source for tracing *why* the model missed/invented an item
- **End-of-log cost**: every run ends with an `=== Estimated LLM cost (this run) ===` block (total, per-stage, models); failed runs append a cost-so-far line. Local/hybrid calls have no $ accounting — total covers OpenRouter only

## Vision vs text models

Two models, not one:

| Role | Config | Stages | Backend |
|------|--------|--------|---------|
| Vision | `MODEL` | Extract, conflict reason (images) | Always OpenRouter |
| Text | `TEXT_MODEL` (falls back to `MODEL`) | Sheet index, jurisdiction, normalize, cluster(LLM), QAQC, code, construct, validate, risk, RFI | OpenRouter, or local when hybrid |

UI: two dropdowns (with per-1M pricing) filled from `GET /models` ([backend/models.py](backend/models.py)), shown only for OpenRouter compute. Per-run picks go through `set_model_overrides(vision, text)` in [backend/llm.py](backend/llm.py): classic runs pass them as `run_pipeline` kwargs (runner clears in `finally`); agent runs set them module-level around `run_agent_pipeline`. UI picks beat per-call agent `AGENT_*_MODEL` args but **never name the hybrid local model** — local stays on `LOCAL_TEXT_MODEL`; the text pick only covers the cloud fallback.

## Where to change what

| Concern | File |
|---------|------|
| Prompts, vocab, conflict taxonomy | [backend/prompts.py](backend/prompts.py) |
| Env knobs | [backend/config.py](backend/config.py), [backend/.env.example](backend/.env.example) |
| HTTP API | [backend/main.py](backend/main.py) |
| UI (upload, models, live log, results) | [frontend/index.html](frontend/index.html) |
| CLI tuning loop | [cli/run_check.py](cli/run_check.py) |
| LLM client, cache, cost, model overrides | [backend/llm.py](backend/llm.py) |
| OpenRouter catalog + pricing + vision/text split | [backend/models.py](backend/models.py) |
| Agent-mode pipeline | [backend/agents/](backend/agents/) (`runner.py` entry; agents call `llm.call_json` with per-call model args) |
| Human review gate (queue, decisions, finalize) | [backend/review/](backend/review/) |
| Job registry + stdout tee log | [backend/jobs.py](backend/jobs.py), [backend/job_log.py](backend/job_log.py) |
| Stage helpers (prompt render, issue validate) | [backend/pipeline/_stage.py](backend/pipeline/_stage.py) |
| Code text corpus (Stage 7) | [backend/code_corpus/](backend/code_corpus/) |
| Hybrid local LLM helper | [scripts/setup_vllm.sh](scripts/setup_vllm.sh) |

Older prompt snapshot: `backend/prompts.py.v1`.

## Gotchas

1. **Prompt placeholders** — Use `str.replace` via `_stage.render`, never `str.format`. Prompts contain literal `{` JSON braces.
2. **Clustering** — Default is LLM (`CLUSTERER=llm`); empty LLM result falls back to deterministic. Deterministic clusters need ≥2 disciplines (or schedule-vs-plan); single-discipline “missing” gaps are a known limit.
3. **Jobs are in-memory** — Process restart clears job status; `outputs/<job_id>/` (report + `job.log` + `job.json` + `source.pdf`) still reload via disk fallback, including `needs_review` recovery.
4. **Dependency pin** — `httpx==0.27.2` with `openai==1.51.0`. httpx ≥0.28 breaks openai’s `proxies=` kwarg.
5. **Code corpus licensing** — Only `ada_2010.txt` is shipped. Do not paste IBC/IFC/IECC without a license (see `backend/code_corpus/README.md`).
6. **Dual assertion schema** — Newer `{sheet, objects[]}` is mapped to legacy `{assertions[]}` with `attribute`/`value` for older stages.
7. **Grounding guard** — Extractor drops objects whose numeric claims are not in `source_text` (graphical-only objects allowed).
8. **Module globals** — LLM cost counters, stdout tee, and model overrides are process-global; overlapping jobs can interleave (single-user tool assumption).
9. **Samples** — `samples/*.pdf` are gitignored; drop PDFs locally for CLI runs.
10. **README drift** — Treat README for setup/CI; treat this file + `runner.py` for pipeline truth. `prompts.py` header may still say some prompts are unwired — they are wired through the runner.
11. **Git identity** — This box has no `user.name` / `user.email`; commits need `GIT_AUTHOR_*` / `GIT_COMMITTER_*` env vars (do not `git config`). Remote push to Gitea works.
12. **No local Python deps on host** — App is meant to run in Docker; bare `python3` imports may miss `dotenv` / `httpx`. Prefer `docker compose`. For tests on this box: venv + requirements, but unpin Pillow (`Pillow>=11`) — 10.4.0 doesn't build on Python 3.14 (Docker uses 3.12, where the pin is fine).
13. **Agent mode constraints** — OpenRouter-only (hybrid disabled in UI and forced off server-side); review gate statuses are `needs_review → reviewing → finalizing → done` (`finalization_error` on finalize failure); only terminal states include the full `log` in polls.

## Quick start pointers

- Full setup: [README.md](README.md) (`docker compose up -d --build` → http://localhost:8099).
- Local CLI: `python cli/run_check.py samples/your_set.pdf --out out/your_set`.
- Prompt iteration: set `LLM_CACHE=true` in `backend/.env` so unchanged stages replay for free; clear with `rm -rf backend/.llm_cache`.
- Artifacts per job: `assertions.json`, `clusters.json`, per-stage JSON, `conflicts.json`, `report.md`, `job.log`, `job.json`, `source.pdf` under `backend/outputs/<job_id>/`.
- Tests: `python -m pytest tests/` (needs the deps from `requirements.txt` + `pytest`; on this box use a venv, see gotcha #12).

## Recent work (2026-08-02, agent-mode)

Merged `main` tip (`a6b0c8f` + `bf508bf`) into `agent-mode`, reconciling with this branch's own earlier implementations:

- **Two model dropdowns** — main's vision/text split ported onto this branch's priced catalog (`models.py`); pickers stay OpenRouter-compute-only, and UI picks never override `LOCAL_TEXT_MODEL` (main's hybrid footgun avoided).
- **Better run logs** — main's timestamped line-splitting tee, `log_tail` polls, terminal-state full log, and log-only disk recovery merged with this branch's header line, `job.json` metadata, and review-gate states. Failed runs now also append the traceback to `job.log`.
- `backend/models_catalog.py` (main's unpriced catalog) intentionally dropped in favor of `models.py`.

## Conflict categories (taxonomy)

Defined in `backend/prompts.py`: `dimensional_disagreement`, `elevation_disagreement`, `location_mismatch`, `missing_element`, `schedule_vs_plan_mismatch`, `detail_vs_plan_mismatch`, `tag_or_reference_inconsistency`, `spatial_clash`, `note_or_spec_contradiction`.