woogiandClaude Opus 5
23e19f53b2
Read-only Q&A on the review screen, per finding and per run, answered from
the job's own artifacts (evidence, cluster, extraction, verification, Brain
merge, sheet index, cover reconciliation, job.log). It never mutates findings,
decisions, or the report.
Turns are logged job-locally (review/chat_log.jsonl, transcript at
/jobs/{id}/review-chat/log) and to a cross-job feedback store
(REVIEW_FEEDBACK_DIR), which now also receives review decisions with their
category/severity corrections.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115gGtrSxXE9DKvS9XPFSoT
214 lines
12 KiB
Markdown
214 lines
12 KiB
Markdown
# Conflict Checker — How It Works (Plain Language)
|
|
|
|
**What it does:** You upload a set of construction drawings (a PDF of blueprints).
|
|
A team of AI assistants reads every page, compares everything against everything
|
|
else, and hands you a list of problems — contradictions between sheets, mistakes
|
|
within a single sheet, missing information, and things that would be hard to
|
|
build — before they cost you money in the field.
|
|
|
|
Think of it like hiring a room full of specialist consultants to review your
|
|
plans overnight. Each one has a specific job, they pass their notes down the
|
|
table, and a senior reviewer at the end sorts it all into one clean report.
|
|
|
|
---
|
|
|
|
## The Big Picture (one sentence per step)
|
|
|
|
```
|
|
YOUR PDF OF BLUEPRINTS
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 0. SCANNER |
|
|
| Turns every PDF page into a picture the AI can read |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 1. READERS (one assistant per page, all at once) |
|
|
| Reads each sheet and writes down every fact: |
|
|
| dimensions, notes, materials, room names, callouts |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 2. LIBRARIAN + LOCAL-CODE SCOUT (work side by side) |
|
|
| Librarian: builds the table of contents — which |
|
|
| sheets exist (electrical, plumbing, structural...) |
|
|
| Scout: figures out WHERE the project is, so we know |
|
|
| which building codes apply |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 3. CONNECTOR |
|
|
| Connects the dots across sheets — e.g. "the water |
|
|
| heater on the plumbing sheet is the same one on the |
|
|
| electrical sheet" — and groups related facts into |
|
|
| topic piles (clusters) |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 4. CONFLICT DETECTIVES (one per topic pile) |
|
|
| Compares sheets that should agree and looks for |
|
|
| contradictions: "Wall shown here on A-201 but not |
|
|
| on S-101", "Pipe runs through the duct". Now also |
|
|
| catches contradictions WITHIN a single sheet |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 5. THREE SPECIALISTS (work side by side) |
|
|
| * Drawing Checker — problems on a sheet BY ITSELF: |
|
|
| a callout pointing to a detail that isn't there, |
|
|
| a schedule that disagrees with its own plan, |
|
|
| dimensions that don't add up, missing scale |
|
|
| * Builder — can this actually be built as |
|
|
| drawn? (access, clearances, sequencing) |
|
|
| * Completeness Checker — is anything MISSING from |
|
|
| the set? (sheets, schedules, required details) |
|
|
| (A Code Inspector also lives here but is turned OFF |
|
|
| by default — the focus is the drawings themselves) |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 6. THE BRAIN (senior reviewer) |
|
|
| Collects EVERY finding from everyone, merges the |
|
|
| duplicates, throws out the weak ones, and ranks the |
|
|
| rest by how much trouble they'd cause |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 6.5 THE BRAIN DOUBLE-CHECKS (asks for a second look) |
|
|
| For the findings it's unsure about, the Brain sends |
|
|
| them back to a fact-checker that re-reads the actual |
|
|
| sheet (zoomed-in image + the page's real text) to |
|
|
| confirm or debunk. Debunked findings are dropped |
|
|
| before they ever reach you |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 7. HUMAN REVIEW GATE |
|
|
| The important/uncertain findings are queued for a |
|
|
| real person to Confirm / Reject / mark Unsure. |
|
|
| You can also ASK the run why it concluded any of |
|
|
| them, or why it never looked at something |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
+----------------------------------------------------------+
|
|
| 8. LETTER WRITER |
|
|
| Drafts a formal RFI (Request For Information — the |
|
|
| official "please clarify this" letter) for each |
|
|
| confirmed issue, ready to send to the design team |
|
|
+----------------------------------------------------------+
|
|
|
|
|
v
|
|
FINAL REPORT + DRAFT RFIs
|
|
```
|
|
|
|
---
|
|
|
|
## Who's Who (the "agents")
|
|
|
|
| # | Name | Analogy | What it actually does |
|
|
|---|------|---------|-----------------------|
|
|
| 0 | PDF Scanner | Photocopier | Converts each PDF page into an image the AI can "see" |
|
|
| 1 | Sheet Extractor | Speed-reader | Reads one page, writes structured notes (every page gets its own reader, in parallel) |
|
|
| 2 | Sheet Indexer | Librarian | Builds the table of contents of the drawing set |
|
|
| 2 | Jurisdiction Scout | Local guide | Identifies the project's location so the right building codes are used |
|
|
| 3 | Linker | Connector | Groups related facts from different sheets into topic clusters |
|
|
| 4 | Conflict Critic | Detective | Examines each cluster for contradictions — between disciplines, across a discipline's own sheets, or within one sheet |
|
|
| 5 | Drawing Integrity Agent | Proofreader | Checks each sheet on its own: dangling callouts, a schedule vs its own plan, dimensions that don't sum, missing scale/north/title-block |
|
|
| 5 | Constructability Agent | Veteran builder | Flags things that are drawn fine but can't be built practically |
|
|
| 5 | Completeness Agent | Checklist keeper | Flags missing sheets, missing details, gaps in the set |
|
|
| 5 | Code Agent *(off by default)* | Code inspector | Building-code/ADA checks — kept in the codebase but disabled so the review focuses on the drawings; one flag turns it back on |
|
|
| 6 | Brain | Chief estimator | Deduplicates, judges, and prioritizes all findings |
|
|
| 6.5 | Brain (clarification) | Second opinion | For findings it distrusts, sends them back to the fact-checker to re-read the sheet; debunked findings are dropped |
|
|
| 7 | Review Gate | Your desk | Presents the findings a human should approve before anything goes out |
|
|
| 7 | Review Chat | The analyst you can question | Answers "why did it decide that?" and "why didn't it check that?" from the run's own records — it explains, it never changes anything |
|
|
| 8 | RFI Writer | Secretary | Writes the formal clarification letters for confirmed issues |
|
|
|
|
Everything the assistants learn is kept in a shared notebook (the "project
|
|
memory"), so each step builds on the last. If one reader fails on one page, the
|
|
rest of the team keeps going — that page is noted as a gap instead of crashing
|
|
the whole review.
|
|
|
|
---
|
|
|
|
## Where Improvements Could Be Made
|
|
|
|
*(Several items from earlier versions have since shipped — noted below.)*
|
|
|
|
### 1. Coverage — "make sure every page actually got read" ✅ *largely shipped*
|
|
- Failed pages used to quietly disappear, and the Completeness Checker would
|
|
then report a sheet as "missing" when it was really just unread.
|
|
- **Done:** a retry ladder now re-reads a page (text-only pass, then a
|
|
deterministic text-layer fallback) so no text-bearing page goes dark, and the
|
|
report separates "sheet doesn't exist" from "sheet couldn't be read."
|
|
- **Still open:** try a different backup model on the hardest pages.
|
|
|
|
### 2. Speed — "the team waits in line more than it needs to"
|
|
- The steps run strictly one after another, but some could start earlier. The
|
|
Jurisdiction Scout only needs the cover page — it could run while the other
|
|
Readers are still working. The Letter Writer could start on high-confidence
|
|
findings instead of waiting for all human review.
|
|
- **Improvement:** overlap independent steps; start drafting letters for
|
|
confirmed/high-confidence findings sooner.
|
|
|
|
### 3. Cost — "smarter reading, fewer wasted words"
|
|
- Every page is read by a large, expensive AI model, and that model's
|
|
"thinking time" counts against its answer budget — we've seen it spend its
|
|
whole budget thinking and return a cut-off answer.
|
|
- **Improvement:** use cheaper models for simple pages (schedules, title
|
|
sheets), save the expensive model for dense drawings; keep tuning the
|
|
thinking budget knobs; reuse cached answers when the same plan set is
|
|
re-run. *(The truncation bug itself is now fixed.)*
|
|
|
|
### 4. Smarter detective work — "catch conflicts that span piles" ✅ *shipped*
|
|
- The Detectives only saw one topic pile at a time, so a contradiction spanning
|
|
two piles — or a mistake on a single sheet — could slip through.
|
|
- **Done:** the Detectives now also flag contradictions *within* a single sheet,
|
|
a new Drawing Checker proofreads every sheet on its own, and after the Brain
|
|
sorts everything it can send doubtful findings back for a zoomed-in second
|
|
look (wave 6.5) and drop the ones that don't hold up.
|
|
- **Still open:** revisit the hard cap on how many topic piles are kept on very
|
|
large sets.
|
|
|
|
### 5. Human time — "review less, but review what matters"
|
|
- Today the review queue is built from rules about severity and confidence.
|
|
- **Improvement:** learn from your past Confirm/Reject decisions to sort the
|
|
queue better — the system already records your feedback, so it can get
|
|
smarter over time about what actually needs your eyes.
|
|
|
|
### 5b. Explaining itself — "why did you think that?" ✅ *shipped*
|
|
- **Done:** every finding on the review screen has a chat panel, plus one for
|
|
the run as a whole. Ask why a unit was read as ground-mounted, or why a whole
|
|
discipline never got looked at, and it traces the answer back through what it
|
|
actually recorded — quoting the note it read off the sheet, or naming the
|
|
stage that skipped the pages. It cannot change a finding; that stays yours.
|
|
- **Done:** when you correct it in conversation ("that's not a floor drain,
|
|
it's a power floor box"), the correction is filed as structured data rather
|
|
than a free-text comment.
|
|
- **Still open:** nothing reads those filed corrections back yet. The next step
|
|
is priming a new run with what reviewers corrected on previous sets, so the
|
|
same misread does not come back on the next job.
|
|
|
|
### 6. Trust — "show the receipts" ✅ *partially shipped*
|
|
- **Done:** the fact-checker already pulls a zoomed-in crop of the exact spot on
|
|
the sheet when it re-reads a finding.
|
|
- **Still open:** attach that crop to the finding in the final report so a
|
|
non-technical reader can verify it in seconds without opening the PDF.
|
|
|
|
---
|
|
|
|
*Technical reference for the curious: the pipeline lives in
|
|
`backend/agents/runner.py` (the waves above are the "Agent wave N" stages), the
|
|
team's shared notebook is `backend/agents/memory.py`, the review queue is
|
|
`backend/review/gate.py` + `backend/review/finalizer.py`, and the review chat is
|
|
`backend/review/chat.py` + `backend/review/chat_context.py`.*
|