Conflict_Checker/NOTES.md

Session Notes — Conflict Checker

Orientation for a new coding session. Setup and Docker details live in README.md. This file tracks what the code actually does and what tends to waste time.

Last updated: 2026-08-02 · agent-mode branch (merged main tip bf508bf)

What this is

Cross-discipline design contradiction / senior-architect QAQC for construction drawing PDFs (Arch, Struct, Mech, Elec, Plumb, FP, etc.). Flags disagreements between disciplines before a set goes to bid/permit.

Not IronBid’s scope-ownership conflict checker.

Source of truth: Scout IT Gitea — gitea.scoutitsystems.com/woogi/Conflict_Checker.

Stack

Layer	Detail
API	Python 3.12, FastAPI, Uvicorn (backend/main.py)
UI	Single static file frontend/index.html, served by FastAPI
Pipelines	Classic: backend/pipeline/runner.py (shared by web + CLI). Agent: backend/agents/runner.py — experimental scoped specialist agents, selected per job (pipeline_mode)
Review gate	Agent jobs stop at needs_review for human decisions before the report emails (backend/review/)
LLM	OpenRouter via openai SDK; default google/gemini-2.5-pro. Vision always OpenRouter; text stages can use local vLLM (classic/hybrid only)
PDF	pdf2image + system poppler-utils → JPEG page images
Jobs	In-memory threads (backend/jobs.py) — no Redis/DB
Tests	pytest tests/ (~82 tests; see Quick start)
Deploy	Docker Compose; app on port 8099; public URL https://conchecker.scoutitsystems.com

Live pipeline (authoritative)

README still describes an older 5-stage extract-then-compare loop. Trust runner.py. Actual flow:

PDF → images → extract → sheet index → jurisdiction
    → normalize → project intelligence (GOIDs)
    → cluster → conflict reason
    → QAQC / code / constructability
    → dedup-validate → risk → RFIs → report

Runner stage	Module	Notes
PDF → images	pdf_processor	Rasterize
Extract assertions	extractor	Vision, per sheet
Classify sheet index	sheet_index	LLM
Jurisdiction profile	jurisdiction	After cover meta
Normalize	normalizer	LLM batches
Project intelligence	normalizer.build_project_intelligence	GOIDs + relationships
Cluster	llm_clusterer or clusterer	Default CLUSTERER=llm
Conflicts	conflict_checker	Per-cluster vision reason
QAQC / code / constructability	qaqc_review, code_review, constructability	Full-set / batched
Validate & dedup	validator	Merges conflict + QAQC + code + construct issues
Risk / RFIs	risk, rfi	Text-only
Report	report	conflicts.json + report.md (+ stage JSON dumps when out_dir set)

Design notes for Stage 2/3 engines also live under Changes/*.docx.

Agent pipeline (pipeline_mode=agent): scoped specialist agents in backend/agents/ (extractors, linker, brain, critics, RFI writer) run through Orchestrator + ProjectMemory; artifacts under outputs/<job_id>/agent/. Agent mode is OpenRouter-only (no hybrid) and, when AGENT_REQUIRE_REVIEW=true, stops at needs_review until a human saves decisions and finalizes via the review endpoints. Design docs: docs/superpowers/.

HTTP API (current)

Method	Path	Purpose
GET	/health	Liveness + model / text_model + version / build + key/email flags
GET	/models	OpenRouter catalog split into vision[] / text[] + defaults, with per-1M-token pricing (cached ~1h; 502 when OpenRouter is unreachable)
POST	/check	Upload PDF; returns {job_id} immediately
GET	/jobs/{id}	Status poll. Running: stage + log_tail. Done/error/needs_review/finalization_error: report and/or error + full log
GET	/jobs/{id}/log	Full run log as text/plain (404 when no log file)
GET	/jobs/{id}/review	Review queue + progress + saved decisions (agent jobs)
POST	/jobs/{id}/review-decisions	Save reviewer decisions (409 outside needs_review/reviewing)
POST	/jobs/{id}/finalize-review	Background finalize + send report (409 unless review gate passed)
GET	/jobs/{id}/sheet-image/{page}	JPEG of source PDF page for the sheet viewer
GET	/	Serves frontend/index.html

/check form fields

• Required: file (PDF)
• Optional: notification_email, project_name, address, occupancy, work_type
• Pipeline: pipeline_mode (classic default, or agent)
• Compute: text_local (true = hybrid local text; forced off for agent mode)
• Models: vision_model, text_model (OpenRouter ids; blank = config defaults)

Job logs

Pipeline print() is teed for the job thread (backend/job_log.py):

• Live: GET /jobs/{id} → log_tail (last 80 lines)
• Done/error/needs_review/finalization_error: same payload includes full log
• Disk: backend/outputs/<job_id>/job.log (survives restart; status registry does not)
• API: GET /jobs/{id}/log → text/plain
• UI: “Run log” panel updates while running; stays visible after finish/fail/review
• Failures append the full traceback to the log; each run starts with a header line (job id, mode, models, start time)
• outputs/<job_id>/job.json (written at start) carries email/mode/models so the disk fallback can rebuild a job after restart
• Verbose LLM observability (LLM_VERBOSE=true, default on): one [LLM] #NNNN stage | model (backend) | in/out sizes | cost | parsed-item counts line per call in job.log — list-valued keys show counts (conflicts[3]) so empty (miss) or invented (hallucination) results stand out
• Raw dumps (LLM_RAW_DUMP=true, default on): full prompt + raw response per call in outputs/<job_id>/llm_raw/NNNN_stage_model.json (base64 images excluded, n_images recorded). This is the source for tracing why the model missed/invented an item
• End-of-log cost: every run ends with an === Estimated LLM cost (this run) === block (total, per-stage, models); failed runs append a cost-so-far line. Local/hybrid calls have no $ accounting — total covers OpenRouter only

Vision vs text models

Two models, not one:

Role	Config	Stages	Backend
Vision	MODEL	Extract, conflict reason (images)	Always OpenRouter
Text	TEXT_MODEL (falls back to MODEL)	Sheet index, jurisdiction, normalize, cluster(LLM), QAQC, code, construct, validate, risk, RFI	OpenRouter, or local when hybrid

UI: two dropdowns (with per-1M pricing) filled from GET /models (backend/models.py), shown only for OpenRouter compute. Per-run picks go through set_model_overrides(vision, text) in backend/llm.py: classic runs pass them as run_pipeline kwargs (runner clears in finally); agent runs set them module-level around run_agent_pipeline. UI picks beat per-call agent AGENT_*_MODEL args but never name the hybrid local model — local stays on LOCAL_TEXT_MODEL; the text pick only covers the cloud fallback.

Where to change what

Concern	File
Prompts, vocab, conflict taxonomy	backend/prompts.py
Env knobs	backend/config.py, backend/.env.example
HTTP API	backend/main.py
UI (upload, models, live log, results)	frontend/index.html
CLI tuning loop	cli/run_check.py
LLM client, cache, cost, model overrides	backend/llm.py
OpenRouter catalog + pricing + vision/text split	backend/models.py
Agent-mode pipeline	backend/agents/ (runner.py entry; agents call llm.call_json with per-call model args)
Human review gate (queue, decisions, finalize)	backend/review/
Job registry + stdout tee log	backend/jobs.py, backend/job_log.py
Stage helpers (prompt render, issue validate)	backend/pipeline/_stage.py
Code text corpus (Stage 7)	backend/code_corpus/
Hybrid local LLM helper	scripts/setup_vllm.sh

Older prompt snapshot: backend/prompts.py.v1.

Gotchas

1. Prompt placeholders — Use str.replace via _stage.render, never str.format. Prompts contain literal { JSON braces.
2. Clustering — Default is LLM (CLUSTERER=llm); empty LLM result falls back to deterministic. Deterministic clusters need ≥2 disciplines (or schedule-vs-plan); single-discipline “missing” gaps are a known limit.
3. Jobs are in-memory — Process restart clears job status; outputs/<job_id>/ (report + job.log + job.json + source.pdf) still reload via disk fallback, including needs_review recovery.
4. Dependency pin — httpx==0.27.2 with openai==1.51.0. httpx ≥0.28 breaks openai’s proxies= kwarg.
5. Code corpus licensing — Only ada_2010.txt is shipped. Do not paste IBC/IFC/IECC without a license (see backend/code_corpus/README.md).
6. Dual assertion schema — Newer {sheet, objects[]} is mapped to legacy {assertions[]} with attribute/value for older stages.
7. Grounding guard — Extractor drops objects whose numeric claims are not in source_text (graphical-only objects allowed).
8. Module globals — LLM cost counters, stdout tee, and model overrides are process-global; overlapping jobs can interleave (single-user tool assumption).
9. Samples — samples/*.pdf are gitignored; drop PDFs locally for CLI runs.
10. README drift — Treat README for setup/CI; treat this file + runner.py for pipeline truth. prompts.py header may still say some prompts are unwired — they are wired through the runner.
11. Git identity — This box has no user.name / user.email; commits need GIT_AUTHOR_* / GIT_COMMITTER_* env vars (do not git config). Remote push to Gitea works.
12. No local Python deps on host — App is meant to run in Docker; bare python3 imports may miss dotenv / httpx. Prefer docker compose. For tests on this box: venv + requirements, but unpin Pillow (Pillow>=11) — 10.4.0 doesn't build on Python 3.14 (Docker uses 3.12, where the pin is fine).
13. Agent mode constraints — OpenRouter-only (hybrid disabled in UI and forced off server-side); review gate statuses are needs_review → reviewing → finalizing → done (finalization_error on finalize failure); only terminal states include the full log in polls.

Quick start pointers

• Full setup: README.md (docker compose up -d --build → http://localhost:8099).
• Local CLI: python cli/run_check.py samples/your_set.pdf --out out/your_set.
• Prompt iteration: set LLM_CACHE=true in backend/.env so unchanged stages replay for free; clear with rm -rf backend/.llm_cache.
• Artifacts per job: assertions.json, clusters.json, per-stage JSON, conflicts.json, report.md, job.log, job.json, source.pdf under backend/outputs/<job_id>/.
• Tests: python -m pytest tests/ (needs the deps from requirements.txt + pytest; on this box use a venv, see gotcha #12).

Recent work (2026-08-02, agent-mode)

Merged main tip (a6b0c8f + bf508bf) into agent-mode, reconciling with this branch's own earlier implementations:

• Two model dropdowns — main's vision/text split ported onto this branch's priced catalog (models.py); pickers stay OpenRouter-compute-only, and UI picks never override LOCAL_TEXT_MODEL (main's hybrid footgun avoided).
• Better run logs — main's timestamped line-splitting tee, log_tail polls, terminal-state full log, and log-only disk recovery merged with this branch's header line, job.json metadata, and review-gate states. Failed runs now also append the traceback to job.log.
• backend/models_catalog.py (main's unpriced catalog) intentionally dropped in favor of models.py.

Conflict categories (taxonomy)

Defined in backend/prompts.py: dimensional_disagreement, elevation_disagreement, location_mismatch, missing_element, schedule_vs_plan_mismatch, detail_vs_plan_mismatch, tag_or_reference_inconsistency, spatial_clash, note_or_spec_contradiction.