feat: text-layer grounding (extractor authority, guard rescue tier, verifier oracle + hi-DPI crops)

- backend/text_layer.py: PyMuPDF text-layer extraction, fuzzy evidence bbox matching, 300-DPI crop rendering, coverage-gap signal - extractor (classic + agent): TEXT LAYER block appended at call sites; grounding guard gains text-layer rescue tier (grounding=text_layer stamp) - verifier: {text_layer} oracle excerpt + evidence-located hi-DPI crops replacing full-page images (fallback preserved, I2 guard intact) - coverage gaps: text-bearing pages with zero extraction -> failed-scope gap findings (agent) / log-only (classic) - config knobs: TEXT_LAYER_ENABLED/MIN_CHARS/MAX_CHARS, VERIFY_TEXT_MAX_CHARS, VERIFY_HI_DPI_CROPS, VERIFY_CROP_DPI, VERIFY_CROP_MARGIN_PTS - tests: 22 new (text_layer unit, grounding/render, runner-level flow) Spec: docs/superpowers/specs/2026-08-12-text-layer-grounding-design.md
2026-08-12 14:27:00 -05:00
parent 349b357e5c
commit 570300324f
14 changed files with 1665 additions and 16 deletions
@@ -0,0 +1,936 @@
+# Evidence Verification + Cross-Sheet Correlation Implementation Plan
+
+> **For Hermes:** Use subagent-driven-development skill to implement this plan task-by-task.
+
+**Goal:** Stop vision-extraction misreads (e.g. "(2) 2x6 STUD PACK" vs the actual "(5) 2x6") from becoming confident downstream findings, and correlate the same physical element across sheets (S101/S205/S401) so no stage reasons from one sheet's text in isolation.
+
+**Architecture:** Three independently shippable phases on the agent pipeline (`backend/agents/runner.py`):
+1. **Disputed-value detection** — deterministic post-link pass that flags contradictory extracted values inside a cluster and surfaces them to the critic/specialist prompts.
+2. **Cross-sheet xref linking** — linker gains detail-reference/tag buckets that join assertions across levels (today `(level, family)` bucketing splits S101/S205/S401 apart).
+3. **Evidence verification wave (5b)** — a bounded vision fact-check agent re-reads the cited sheet images for high-severity / disputed findings before the Brain merge, annotates or suppresses findings built on phantom text.
+
+**Tech Stack:** Python 3.14, pytest (`tests/`), existing `call_json` LLM wrapper (supports `images_b64`, `reasoning_effort`, `reasoning_max_tokens`).
+
+---
+
+## Current context / root cause (from job 959e16407573)
+
+Finding `validated_issues[3]` ("FRONT PERSPECTIVE detail on Sheet S401", HSS16x4 on
+"(2) 2x6 STUD PACK", severity critical) is a **false positive built on a wave-1 vision
+misread**. The sheet actually shows a (5) 2x6 stud pack (matching S205/S101). Chain of failure:
+
+1. Wave 1 (`SheetExtractorAgent`) froze the misread into text. From then on it is "ground truth".
+2. Wave 3 linker (`backend/agents/linker.py:33` `build_link_scopes`) buckets by
+   `(level, family)`. S101 (foundation), S205 (details), S401 (sections) get different
+   `level` values, so assertions about the same front-wall header never share a link scope
+   or cluster. No cross-sheet corroboration happened.
+3. Wave 5 `ConstructabilityAgent` (`backend/agents/construct_agent.py:53`) calls
+   `call_json` **with no images** — in this job 120/120 constructability calls were `+0img`.
+   It reasoned arithmetically from the misread text ("2 x 1.5in = 3in < 4in -> unbuildable").
+   It even held the "(5) 2x6 STUD PACK" assertion in the same scope but labeled it
+   "Ambiguous column size specification" instead of arbitrating.
+4. Nothing between wave 5 and the report ever looks at a sheet image again. Only the wave-4
+   conflict critic receives images, and only for its own cluster's pages.
+
+Also confirmed in this log (separate known bug, fixed in Task 7 while we're here): wave-4
+conflict critic truncates on Gemini thinking tokens because `conflict_critic.py:59` passes
+`max_tokens=config.REASON_MAX_TOKENS` (4096) with no reasoning budget — 13/121 calls hit
+`finish_reason=length`.
+
+## Assumptions
+
+- Assertions carry `id`, `attribute`, `value`, `source_text`, `location_key`
+  (`room`/`grid`/`detail_reference`/`tag`/`level`) — see `linker._payload` and
+  `_serialize.slim_assertion`.
+- Extractor assertions already carry a `confidence` field (per test fixtures).
+- `validate_issue` in `backend/pipeline/_stage.py` guarantees each finding an `issue_id`.
+- Test convention: `unittest.mock.patch("backend.agents.<module>.call_json", ...)` —
+  see `tests/agents/test_sheet_extractor_fallback.py`. Run tests with
+  `.venv/bin/python -m pytest tests/ -x -q`.
+- `AgentResult.error` defaults to `""` (not None) in assertions.
+
+---
+
+## Phase 1 — Disputed-value detection + prompt hardening
+
+### Task 1: `find_disputes` pure function (TDD)
+
+**Objective:** Detect "same attribute, different values" inside one cluster's assertions.
+
+**Files:**
+- Create: `backend/agents/disputes.py`
+- Test: `tests/agents/test_disputes.py`
+
+**Step 1: Write failing test**
+
+```python
+# tests/agents/test_disputes.py
+from backend.agents.disputes import annotate_clusters, find_disputes
+
+
+def _a(id_, attribute, value):
+    return {"id": id_, "attribute": attribute, "value": value,
+            "source_text": value}
+
+
+def test_find_disputes_flags_same_attribute_different_values():
+    assertions = [
+        _a("a1", "stud_pack_size", "(2) 2x6 STUD PACK"),
+        _a("a2", "stud_pack_size", "(5) 2x6 STUD PACK"),
+        _a("a3", "beam_size", "HSS16X4X5/8"),
+    ]
+    disputes = find_disputes(assertions)
+    assert len(disputes) == 1
+    assert disputes[0]["attribute"] == "stud_pack_size"
+    assert disputes[0]["values"] == ["(2) 2x6 STUD PACK", "(5) 2x6 STUD PACK"]
+    assert disputes[0]["assertion_ids"] == ["a1", "a2"]
+
+
+def test_find_disputes_ignores_agreeing_values_and_blanks():
+    assertions = [
+        _a("a1", "beam_size", "HSS16X4X5/8"),
+        _a("a2", "beam_size", "  hss16x4x5/8 "),   # same after normalize
+        _a("a3", "", "orphan"),                     # no attribute -> skipped
+        _a("a4", "beam_size", ""),                  # no value -> skipped
+    ]
+    assert find_disputes(assertions) == []
+
+
+def test_annotate_clusters_writes_disputed_attributes():
+    clusters = [
+        {"key": "c1", "assertions": [
+            _a("a1", "stud_pack_size", "(2) 2x6"),
+            _a("a2", "stud_pack_size", "(5) 2x6"),
+        ]},
+        {"key": "c2", "assertions": [_a("a3", "x", "1"), _a("a4", "x", "1")]},
+    ]
+    assert annotate_clusters(clusters) == 1
+    assert clusters[0]["disputed_attributes"][0]["attribute"] == "stud_pack_size"
+    assert "disputed_attributes" not in clusters[1]
+```
+
+**Step 2: Run test to verify failure**
+
+Run: `.venv/bin/python -m pytest tests/agents/test_disputes.py -v`
+Expected: FAIL — `ModuleNotFoundError: backend.agents.disputes`
+
+**Step 3: Implement**
+
+```python
+# backend/agents/disputes.py
+"""Deterministic detection of contradictory extracted values within a cluster.
+
+Extraction is a vision pass: quantities and sizes can be misread ("(2) 2x6" vs
+"(5) 2x6"). Cluster members are supposed to describe the same real-world
+element, so two members asserting different values for the same attribute are
+a probable misread. Flag these so downstream text-only stages treat the value
+as unverified instead of reasoning from one reading.
+"""
+
+import re
+from typing import Dict, List
+
+
+def _norm(value) -> str:
+    return re.sub(r"\s+", " ", str(value or "").strip().lower())
+
+
+def find_disputes(assertions: List[Dict]) -> List[Dict]:
+    """Same attribute with >= 2 distinct normalized values = disputed."""
+    groups: Dict[str, Dict[str, set]] = {}
+    for assertion in assertions:
+        attribute = _norm(assertion.get("attribute"))
+        value = _norm(assertion.get("value"))
+        if not attribute or not value:
+            continue
+        groups.setdefault(attribute, {}).setdefault(value, set()).add(
+            assertion.get("id")
+        )
+    disputes = []
+    for attribute, values in sorted(groups.items()):
+        if len(values) < 2:
+            continue
+        disputes.append({
+            "attribute": attribute,
+            "values": sorted(values),
+            "assertion_ids": sorted(
+                aid for ids in values.values() for aid in ids if aid
+            ),
+        })
+    return disputes
+
+
+def annotate_clusters(clusters: List[Dict]) -> int:
+    """Attach disputed_attributes to each cluster that has any. Returns count."""
+    annotated = 0
+    for cluster in clusters:
+        disputes = find_disputes(cluster.get("assertions") or [])
+        if disputes:
+            cluster["disputed_attributes"] = disputes
+            annotated += 1
+    return annotated
+```
+
+**Step 4: Run test to verify pass**
+
+Run: `.venv/bin/python -m pytest tests/agents/test_disputes.py -v`
+Expected: 3 passed
+
+**Step 5: Commit**
+
+```bash
+git add backend/agents/disputes.py tests/agents/test_disputes.py
+git commit -m "feat: deterministic disputed-value detection for cluster assertions"
+```
+
+---
+
+### Task 2: Wire `annotate_clusters` into the runner + serialization
+
+**Objective:** Disputes must be visible to the wave-4 critic and wave-5 constructability prompts.
+
+**Files:**
+- Modify: `backend/agents/runner.py` (after `memory.replace("clusters", clusters)`, ~line 116)
+- Modify: `backend/pipeline/_serialize.py` (`slim_clusters`, line 46)
+- Test: `tests/agents/test_disputes.py` (append)
+
+**Step 1: Write failing test**
+
+```python
+def test_slim_clusters_preserves_disputed_attributes():
+    from backend.pipeline._serialize import slim_clusters
+    cluster = {"key": "c1", "assertions": [],
+               "disputed_attributes": [{"attribute": "a", "values": ["1", "2"],
+                                         "assertion_ids": ["x", "y"]}]}
+    slim = slim_clusters([cluster])[0]
+    assert slim["disputed_attributes"][0]["values"] == ["1", "2"]
+```
+
+**Step 2: Run test to verify failure**
+
+Run: `.venv/bin/python -m pytest tests/agents/test_disputes.py::test_slim_clusters_preserves_disputed_attributes -v`
+Expected: FAIL — `KeyError: 'disputed_attributes'`
+
+**Step 3: Implement**
+
+In `backend/pipeline/_serialize.py` `slim_clusters`, add the key:
+
+```python
+def slim_clusters(clusters: List[Dict]) -> List[Dict]:
+    return [
+        {
+            "key": c.get("key"),
+            "location": c.get("location"),
+            "disciplines": c.get("disciplines"),
+            "kind": c.get("kind"),
+            **({"disputed_attributes": c["disputed_attributes"]}
+               if c.get("disputed_attributes") else {}),
+            "assertions": [slim_assertion(a) for a in c.get("assertions", [])],
+        }
+        for c in clusters
+    ]
+```
+
+In `backend/agents/runner.py`, right after `clusters = [...]` / `object_graph = build_object_graph(clusters)` (before `memory.replace("clusters", clusters)`):
+
+```python
+from backend.agents.disputes import annotate_clusters
+...
+    object_graph = build_object_graph(clusters)
+    disputed_count = annotate_clusters(clusters)
+    if disputed_count:
+        orchestrator.log(
+            f"[Link] {disputed_count} clusters carry disputed extracted values"
+        )
+```
+
+(Check `Orchestrator` for the actual log method name — `orchestrator.stage(...)` exists;
+if no `.log`, use the module's existing logging/print convention. Adjust to match.)
+
+**Step 4: Run tests**
+
+Run: `.venv/bin/python -m pytest tests/agents/ -v`
+Expected: all pass (including existing `test_runner_review_gate.py`)
+
+**Step 5: Commit**
+
+```bash
+git add backend/agents/runner.py backend/pipeline/_serialize.py tests/agents/test_disputes.py
+git commit -m "feat: surface disputed extracted values to critic and specialist prompts"
+```
+
+---
+
+### Task 3: Prompt hardening — extracted text is fallible
+
+**Objective:** Tell text-only specialists how to handle disputed/unverified values so they stop asserting buildability conclusions from a single (possibly misread) number.
+
+**Files:**
+- Modify: `backend/prompts.py` `CONSTRUCTABILITY_SYSTEM_PROMPT` (line 533) and `CONSTRUCTABILITY_USER_INSTRUCTION` (line 551)
+
+**Step 1: Edit prompts**
+
+Append to `CONSTRUCTABILITY_SYSTEM_PROMPT` Rules list (after line 547, before "Use plain ASCII"):
+
+```
+- Assertions are machine-extracted from sheet images and may contain misread values,
+  especially quantities and member sizes (e.g. "(2) 2x6" vs "(5) 2x6").
+- When the cluster lists disputed_attributes, or two evidence items disagree on a
+  numeric value, do NOT assert a buildability conclusion from one reading. Report the
+  ambiguity itself (category "detail_gap", confidence "low") and state that the value
+  needs verification against the sheet.
+```
+
+Append to `CONSTRUCTABILITY_USER_INSTRUCTION` after the `Cross-discipline conflicts already found: {conflicts}` line:
+
+```
+Disputed extracted values in this cluster (possible vision misreads - treat as unverified): {disputes}
+```
+
+**Step 2: Wire the `{disputes}` placeholder in `construct_agent.py`**
+
+In `backend/agents/construct_agent.py` `run()`, extend the `substitutions` dict:
+
+```python
+            substitutions = {
+                "assertions": dumps(cluster["assertions"]),
+                "clusters": dumps(slim_clusters([cluster])),
+                "conflicts": dumps(scope.payload.get("conflicts") or []),
+                "disputes": dumps(cluster.get("disputed_attributes") or []),
+            }
+```
+
+**Step 3: Run full test suite (prompt edits can break runner tests that snapshot prompts)**
+
+Run: `.venv/bin/python -m pytest tests/ -q`
+Expected: all pass
+
+**Step 4: Commit**
+
+```bash
+git add backend/prompts.py backend/agents/construct_agent.py
+git commit -m "feat: constructability prompt treats disputed extracted values as unverified"
+```
+
+---
+
+## Phase 2 — Cross-sheet xref linking
+
+### Task 4: detail-reference / tag xref buckets in the linker (TDD)
+
+**Objective:** Assertions sharing a `detail_reference` or a member `tag` get linked across levels, so S101/S205/S401 details of the same physical element land in one scope.
+
+**Files:**
+- Modify: `backend/agents/linker.py` (`build_link_scopes`, line 33)
+- Test: `tests/agents/test_linker_xref.py`
+
+**Step 1: Write failing test**
+
+```python
+# tests/agents/test_linker_xref.py
+from backend.agents.base import AgentScope
+from backend.agents.linker import build_link_scopes
+
+
+def _sheet(number, page, level, assertions):
+    return {"sheet_number": number, "page_number": page,
+            "discipline": "Structural", "level": level,
+            "assertions": assertions}
+
+
+def _assertion(id_, ref=None, tag=None, level=None):
+    return {"id": id_, "attribute": "stud_pack_size", "value": "(5) 2x6",
+            "source_text": "(5) 2x6 STUD PACK",
+            "location_key": {"detail_reference": ref, "tag": tag,
+                             "level": level}}
+
+
+def test_xref_scope_joins_same_detail_reference_across_levels():
+    sheets = [
+        _sheet("S101", 10, "foundation", [_assertion("a1", ref="A/S205")]),
+        _sheet("S205", 20, "roof", [_assertion("a2", ref="A/S205")]),
+        _sheet("S401", 30, "roof", [_assertion("a3", ref="A/S205")]),
+    ]
+    scopes = build_link_scopes(sheets)
+    xref = [s for s in scopes if s.scope_id.startswith("xref:")]
+    assert xref, "expected a cross-level detail-reference scope"
+    ids = {a["id"] for s in xref for a in s.payload["assertions"]}
+    assert ids == {"a1", "a2", "a3"}
+
+
+def test_xref_scope_requires_two_distinct_sheets():
+    sheets = [
+        _sheet("S401", 30, "roof", [_assertion("a1", ref="A/S205"),
+                                    _assertion("a2", ref="A/S205")]),
+    ]
+    scopes = build_link_scopes(sheets)
+    assert not [s for s in scopes if s.scope_id.startswith("xref:")]
+
+
+def test_xref_scope_joins_shared_member_tag():
+    sheets = [
+        _sheet("S102", 5, "roof", [_assertion("a1", tag="HSS16X4X5/8")]),
+        _sheet("S401", 30, "unknown", [_assertion("a2", tag="HSS16X4X5/8")]),
+    ]
+    scopes = build_link_scopes(sheets)
+    xref = [s for s in scopes if s.scope_id.startswith("xref:")]
+    assert xref
+```
+
+**Step 2: Run test to verify failure**
+
+Run: `.venv/bin/python -m pytest tests/agents/test_linker_xref.py -v`
+Expected: FAIL — no `xref:` scopes produced
+
+**Step 3: Implement**
+
+Rewrite `build_link_scopes` in `backend/agents/linker.py` (keep the existing
+`(level, family)` bucketing, add the xref pass):
+
+```python
+def _xref_keys(assertion: Dict) -> List[str]:
+    """Cross-level join keys: detail references and member tags."""
+    location = assertion.get("location_key") or {}
+    keys = []
+    ref = re.sub(r"\s+", "", str(location.get("detail_reference") or "")).upper()
+    if ref:
+        keys.append(f"detail:{ref}")
+    tag = re.sub(r"\s+", "", str(location.get("tag") or "")).upper()
+    if re.match(r"^[A-Z]{2,}\d", tag):  # member marks: HSS16X4X5/8, W12X26, ...
+        keys.append(f"tag:{tag}")
+    return keys
+
+
+def build_link_scopes(sheets: List[Dict]) -> List[AgentScope]:
+    """Partition facts by level and object/tag family, then enforce a hard cap.
+
+    A second pass joins assertions that share a detail_reference or member tag
+    ACROSS levels, so plan/detail/section sheets describing the same physical
+    element are linked together even though their levels differ.
+    """
+    buckets: Dict[Tuple[str, str], List[Dict]] = defaultdict(list)
+    xref: Dict[str, List[Dict]] = defaultdict(list)
+    for sheet in sheets:
+        for assertion in sheet.get("assertions", []):
+            enriched = {
+                **assertion,
+                "discipline": sheet.get("discipline") or "Unknown",
+                "sheet_number": sheet.get("sheet_number"),
+                "page_number": sheet.get("page_number"),
+            }
+            level = str((assertion.get("location_key") or {}).get("level")
+                        or sheet.get("level") or "unknown").lower()
+            buckets[(level, _family(assertion))].append(enriched)
+            for key in _xref_keys(assertion):
+                xref[key].append(enriched)
+
+    scopes: List[AgentScope] = []
+    cap = max(2, config.AGENT_LINK_MAX_ASSERTIONS)
+    for (level, family), assertions in sorted(buckets.items()):
+        for offset in range(0, len(assertions), cap):
+            chunk = assertions[offset:offset + cap]
+            if len(chunk) < 2:
+                continue
+            scopes.append(AgentScope(
+                scope_id=f"{level}:{family}:{offset // cap + 1}",
+                payload={"assertions": chunk, "level": level, "family": family},
+            ))
+    for key, assertions in sorted(xref.items()):
+        sheets_present = {a.get("sheet_number") for a in assertions}
+        if len(assertions) < 2 or len(sheets_present) < 2:
+            continue
+        scopes.append(AgentScope(
+            scope_id=f"xref:{key}",
+            payload={"assertions": assertions[:cap],
+                     "level": "xref", "family": key},
+        ))
+    return scopes
+```
+
+**Step 4: Run tests**
+
+Run: `.venv/bin/python -m pytest tests/agents/test_linker_xref.py tests/agents/ -v`
+Expected: all pass (watch existing runner tests for scope-count coupling)
+
+**Step 5: Commit**
+
+```bash
+git add backend/agents/linker.py tests/agents/test_linker_xref.py
+git commit -m "feat: cross-level xref link scopes via detail_reference and member tag"
+```
+
+---
+
+## Phase 3 — Evidence verification wave (5b)
+
+### Task 5: Config knobs + verify prompts
+
+**Objective:** Add the tuning surface and prompts for the vision fact-check agent.
+
+**Files:**
+- Modify: `backend/config.py` (near line 44, with the other AGENT_* knobs)
+- Modify: `backend/prompts.py` (append near the CONFLICT prompts, ~line 460)
+- Modify: `backend/.env.example`
+
+**Step 1: Add config knobs to `backend/config.py`**
+
+```python
+AGENT_VERIFY_MODEL         = os.getenv("AGENT_VERIFY_MODEL", "") or MODEL
+AGENT_VERIFY_CONCURRENCY   = int(os.getenv("AGENT_VERIFY_CONCURRENCY", "4"))
+AGENT_VERIFY_MAX_CHECKS    = int(os.getenv("AGENT_VERIFY_MAX_CHECKS", "20"))
+AGENT_VERIFY_SEVERITIES    = {
+    s.strip().lower()
+    for s in os.getenv("AGENT_VERIFY_SEVERITIES", "critical,high").split(",")
+    if s.strip()
+}
+AGENT_VERIFY_REASONING_EFFORT = os.getenv("AGENT_VERIFY_REASONING_EFFORT", "low").strip()
+VERIFY_MAX_TOKENS          = int(os.getenv("VERIFY_MAX_TOKENS", "8192"))
+```
+
+Append to `backend/.env.example`:
+
+```
+# Wave 5b evidence verification (vision fact-check of cited sheet text)
+AGENT_VERIFY_MAX_CHECKS=20
+AGENT_VERIFY_SEVERITIES=critical,high
+AGENT_VERIFY_REASONING_EFFORT=low
+VERIFY_MAX_TOKENS=8192
+```
+
+**Step 2: Add prompts to `backend/prompts.py`**
+
+```python
+VERIFY_SYSTEM_PROMPT = """You are a meticulous construction document checker verifying machine-extracted evidence against the actual drawing sheet images.
+For each evidence item you are given the sheet it was extracted from and the verbatim text the extractor claims appears there.
+Judge each item against the images:
+- confirmed: the text (or an obvious equivalent) appears on the cited sheet and means what the finding claims.
+- corrected: the sheet shows a DIFFERENT value than the extracted text. Give the actual verbatim text.
+- not_found: nothing like the extracted text appears on the cited sheet.
+Be strict about numbers, quantities, and member sizes: "(2) 2x6" and "(5) 2x6" are different values. HSS16x4 and HSS16x16 are different values.
+Use plain ASCII only.
+Respond only with valid JSON."""
+
+VERIFY_USER_INSTRUCTION = """Verify this finding's evidence against the attached sheet images.
+Respond ONLY with a valid JSON object - no markdown fences, no explanation:
+{ "verdicts": [ { "sheet": "string", "source_text": "the evidence text judged", "verdict": "confirmed | corrected | not_found", "actual_text": "verbatim sheet text when corrected, else null", "notes": "string or null" } ] }
+Finding: {finding}"""
+```
+
+**Step 3: Sanity check**
+
+Run: `.venv/bin/python -c "from backend import config, prompts; print(config.AGENT_VERIFY_MAX_CHECKS, config.AGENT_VERIFY_SEVERITIES); print(prompts.VERIFY_SYSTEM_PROMPT[:40])"`
+Expected: `20 {'critical', 'high'}` and prompt text
+
+**Step 4: Commit**
+
+```bash
+git add backend/config.py backend/prompts.py backend/.env.example
+git commit -m "feat: config knobs and prompts for evidence verification wave"
+```
+
+---
+
+### Task 6: `EvidenceVerifierAgent` + runner wave 5b (TDD)
+
+**Objective:** Re-read cited sheet images for selected findings; annotate verified findings, suppress refuted ones before the Brain merge.
+
+**Files:**
+- Create: `backend/agents/verifier.py`
+- Modify: `backend/agents/runner.py` (new wave between wave 5 and wave 6, ~line 167)
+- Modify: `backend/agents/construct_agent.py` line 67 (stamp `cluster_key` for dispute-based selection)
+- Test: `tests/agents/test_verifier.py`
+
+**Step 1: Write failing test**
+
+```python
+# tests/agents/test_verifier.py
+from unittest.mock import patch
+
+from backend.agents.base import AgentScope, AgentUsage
+from backend.agents.verifier import (
+    EvidenceVerifierAgent, apply_verdicts, select_findings,
+)
+
+
+def _finding(sev="critical", issue_id="i1", sheets=("S401",), cluster_key=None):
+    f = {"issue_id": issue_id, "severity": sev, "confidence": "high",
+         "source_stage": "constructability", "sheets": list(sheets),
+         "description": "HSS16x4 on (2) 2x6 STUD PACK is unbuildable",
+         "evidence": [{"sheet": "S401", "source_text": "(2) 2x6 STUD PACK",
+                       "asserted_value": "3-inch width"}]}
+    if cluster_key:
+        f["cluster_key"] = cluster_key
+    return f
+
+
+def test_select_findings_by_severity_and_dispute():
+    findings = [_finding("critical"), _finding("low", "i2"),
+                _finding("medium", "i3", cluster_key="c9")]
+    clusters = [{"key": "c9", "disputed_attributes": [{"attribute": "a"}]}]
+    selected = select_findings(findings, clusters, max_checks=20,
+                               severities={"critical", "high"})
+    assert [f["issue_id"] for f in selected] == ["i1", "i3"]
+
+
+def test_select_findings_respects_cap():
+    findings = [_finding("critical", f"i{n}") for n in range(30)]
+    selected = select_findings(findings, [], max_checks=5,
+                               severities={"critical"})
+    assert len(selected) == 5
+
+
+def test_run_attaches_verdicts_and_marks_refuted():
+    agent = EvidenceVerifierAgent(usage=AgentUsage())
+    scope = AgentScope(scope_id="verify:0", payload={
+        "finding_index": 0,
+        "finding": _finding(),
+        "images_b64": ["QUJD"],
+    })
+    verdicts = {"verdicts": [
+        {"sheet": "S401", "source_text": "(2) 2x6 STUD PACK",
+         "verdict": "corrected", "actual_text": "(5) 2x6 STUD PACK",
+         "notes": "callout reads (5)"},
+    ]}
+    with patch("backend.agents.verifier.call_json", return_value=verdicts):
+        result = agent.run(scope)
+    assert not result.error
+    artifact = result.artifacts[0]
+    assert artifact["finding_index"] == 0
+    assert artifact["status"] == "refuted"      # no evidence confirmed
+    assert artifact["verdicts"][0]["actual_text"] == "(5) 2x6 STUD PACK"
+
+
+def test_apply_verdicts_annotates_and_suppresses():
+    findings = [_finding("critical", "i1"), _finding("high", "i2")]
+    from backend.agents.base import AgentResult
+    results = [AgentResult(scope_id="verify:0", artifacts=[
+        {"finding_index": 0, "status": "refuted", "verdicts": []},
+        {"finding_index": 1, "status": "confirmed", "verdicts": []},
+    ])]
+    suppressed = apply_verdicts(findings, results)
+    assert suppressed == [findings[0]]
+    assert findings[0]["verification"]["status"] == "refuted"
+    assert findings[1]["verification"]["status"] == "confirmed"
+```
+
+**Step 2: Run test to verify failure**
+
+Run: `.venv/bin/python -m pytest tests/agents/test_verifier.py -v`
+Expected: FAIL — `ModuleNotFoundError: backend.agents.verifier`
+
+**Step 3: Implement `backend/agents/verifier.py`**
+
+```python
+"""Wave 5b: vision fact-check of extracted evidence against cited sheet images.
+
+Downstream specialists are text-only; a wave-1 vision misread ("(2) 2x6" vs
+"(5) 2x6") otherwise becomes immutable ground truth. For high-severity or
+dispute-linked findings, re-read the cited sheets and adjudicate each evidence
+item: confirmed / corrected / not_found. Findings whose evidence is entirely
+unconfirmed are suppressed before the Brain merge.
+"""
+
+from typing import Dict, List, Optional, Set
+
+from backend import config
+from backend.agents.base import AgentResult, AgentScope, AgentUsage, failure
+from backend.llm import call_json
+from backend.pipeline._serialize import dumps
+from backend.pipeline._stage import collect_list, render
+from backend.prompts import VERIFY_SYSTEM_PROMPT, VERIFY_USER_INSTRUCTION
+
+_SEVERITY_RANK = {"critical": 0, "high": 1, "medium": 2, "low": 3}
+_VERDICTS = ("confirmed", "corrected", "not_found")
+
+
+def select_findings(
+    findings: List[Dict],
+    clusters: List[Dict],
+    max_checks: int,
+    severities: Set[str],
+) -> List[Dict]:
+    """Severity-gated selection plus any finding tied to a disputed cluster."""
+    disputed_keys = {
+        cluster.get("key") for cluster in clusters
+        if cluster.get("disputed_attributes")
+    }
+    selected = [
+        finding for finding in findings
+        if str(finding.get("severity") or "").lower() in severities
+        or finding.get("cluster_key") in disputed_keys
+    ]
+    selected.sort(key=lambda f: _SEVERITY_RANK.get(
+        str(f.get("severity") or "").lower(), 9))
+    return selected[:max_checks]
+
+
+def _valid_verdict(item: Dict) -> Optional[Dict]:
+    if not isinstance(item, dict):
+        return None
+    verdict = str(item.get("verdict") or "").lower()
+    if verdict not in _VERDICTS:
+        return None
+    return {
+        "sheet": item.get("sheet") or "",
+        "source_text": item.get("source_text") or "",
+        "verdict": verdict,
+        "actual_text": item.get("actual_text"),
+        "notes": item.get("notes"),
+    }
+
+
+def _status(verdicts: List[Dict]) -> str:
+    if not verdicts:
+        return "unverified"
+    confirmed = sum(1 for v in verdicts if v["verdict"] == "confirmed")
+    if confirmed == len(verdicts):
+        return "confirmed"
+    if confirmed == 0:
+        return "refuted"
+    return "mixed"
+
+
+class EvidenceVerifierAgent:
+    name = "verify"
+
+    def __init__(self, usage: AgentUsage) -> None:
+        self.usage = usage
+
+    def run(self, scope: AgentScope) -> AgentResult:
+        try:
+            finding = scope.payload["finding"]
+            instruction = render(
+                VERIFY_USER_INSTRUCTION, {"finding": dumps(finding)}
+            )
+            parsed = call_json(
+                system_prompt=VERIFY_SYSTEM_PROMPT,
+                user_text=instruction,
+                images_b64=scope.payload.get("images_b64") or [],
+                max_tokens=config.VERIFY_MAX_TOKENS,
+                model=config.AGENT_VERIFY_MODEL,
+                reasoning_effort=config.AGENT_VERIFY_REASONING_EFFORT or None,
+                usage_tracker=self.usage,
+                usage_stage="agent.verify",
+            )
+            verdicts = collect_list(parsed, "verdicts", _valid_verdict)
+            return AgentResult(scope_id=scope.scope_id, artifacts=[{
+                "finding_index": scope.payload["finding_index"],
+                "status": _status(verdicts),
+                "verdicts": verdicts,
+            }])
+        except Exception as exc:
+            return failure(scope, exc)
+
+
+def apply_verdicts(
+    findings: List[Dict], verify_results: List[AgentResult]
+) -> List[Dict]:
+    """Annotate findings with verification; return refuted ones to suppress."""
+    by_index: Dict[int, Dict] = {}
+    for result in verify_results:
+        for artifact in result.artifacts:
+            by_index[artifact["finding_index"]] = artifact
+    suppressed = []
+    for index, finding in enumerate(findings):
+        artifact = by_index.get(index)
+        if not artifact:
+            continue
+        finding["verification"] = {
+            "status": artifact["status"],
+            "verdicts": artifact["verdicts"],
+        }
+        if artifact["status"] == "refuted":
+            finding["confidence"] = "low"
+            suppressed.append(finding)
+    return suppressed
+```
+
+**Step 4: Stamp `cluster_key` on constructability findings**
+
+In `backend/agents/construct_agent.py` line 66-67, change:
+
+```python
+            for finding in findings:
+                finding.update(agent=self.name, scope_id=scope.scope_id)
+```
+
+to:
+
+```python
+            for finding in findings:
+                finding.update(agent=self.name, scope_id=scope.scope_id,
+                               cluster_key=cluster.get("key"))
+```
+
+**Step 5: Wire wave 5b into `backend/agents/runner.py`**
+
+After `memory.extend("findings", specialist_findings)` (line 167) and before
+`gap_findings` / wave 6:
+
+```python
+    orchestrator.stage("Agent wave 5b: evidence verification")
+    sheet_to_page = {
+        sheet.get("sheet_number"): sheet.get("page_number") for sheet in sheets
+    }
+    verify_targets = select_findings(
+        specialist_findings, clusters,
+        max_checks=config.AGENT_VERIFY_MAX_CHECKS,
+        severities=config.AGENT_VERIFY_SEVERITIES,
+    )
+    target_indexes = {id(f): i for i, f in enumerate(specialist_findings)}
+    verify_scopes = [
+        AgentScope(
+            scope_id=f"verify:{target_indexes[id(finding)]}",
+            payload={
+                "finding_index": target_indexes[id(finding)],
+                "finding": finding,
+                "images_b64": [
+                    page_to_b64[sheet_to_page[name]]
+                    for name in (finding.get("sheets") or [])
+                                [:config.AGENT_CONFLICT_MAX_IMAGES]
+                    if sheet_to_page.get(name) in page_to_b64
+                ],
+            },
+        )
+        for finding in verify_targets
+    ]
+    verify_results = orchestrator.run_scopes(
+        EvidenceVerifierAgent(usage), verify_scopes,
+        config.AGENT_VERIFY_CONCURRENCY,
+    )
+    suppressed = apply_verdicts(specialist_findings, verify_results)
+    if suppressed:
+        suppressed_ids = {id(f) for f in suppressed}
+        specialist_findings = [
+            f for f in specialist_findings if id(f) not in suppressed_ids
+        ]
+        memory.replace("suppressed", suppressed)
+        memory.extend("findings", specialist_findings)  # see note below
+```
+
+NOTE for implementer: `memory.extend("findings", ...)` already ran with the
+un-suppressed list. Adjust ordering so verification happens BEFORE
+`memory.extend("findings", specialist_findings)` — i.e. move the extend to after
+wave 5b — so the Brain never sees refuted findings. Keep `gap_findings` logic
+unchanged. Also add imports at top of runner.py:
+
+```python
+from backend.agents.verifier import (
+    EvidenceVerifierAgent, apply_verdicts, select_findings,
+)
+```
+
+And in the report dicts (both the `require_review` branch ~line 217-225 and the
+wave-7 branch ~line 293-300), populate suppressed issues:
+
+```python
+        "suppressed_issues": memory.snapshot().get("suppressed") or [],
+```
+
+Finally, `verify` results cost shows up as `agent.verify` in
+`summary.cost_by_stage` automatically via `usage_stage="agent.verify"`.
+
+**Step 6: Update existing runner tests**
+
+`tests/agents/test_runner_review_gate.py` monkeypatches `BrainAgent` and
+`convert_pdf_to_images` but lets waves 1-5 run against... check how LLM calls
+are stubbed there (likely `call_json` returns None -> empty artifacts, which is
+fine). The new wave must no-op cleanly when `select_findings` returns `[]`
+(zero scopes -> `run_scopes` returns `[]` per orchestrator.py:60). Verify by
+running the suite; if a runner test now fails because verification selects a
+stubbed finding, monkeypatch `select_findings` to `lambda *a, **k: []` in that
+test file's `_patch_brain` helper.
+
+**Step 7: Run full suite**
+
+Run: `.venv/bin/python -m pytest tests/ -q`
+Expected: all pass
+
+**Step 8: Commit**
+
+```bash
+git add backend/agents/verifier.py backend/agents/runner.py backend/agents/construct_agent.py tests/agents/test_verifier.py tests/agents/test_runner_review_gate.py
+git commit -m "feat: wave 5b evidence verification - vision fact-check before Brain merge"
+```
+
+---
+
+### Task 7 (small, related): reasoning budget for the conflict critic
+
+**Objective:** Fix the wave-4 truncation found in this same job (13/121 calls hit `finish_reason=length` at the 4096 cap with ~3.7k thinking tokens).
+
+**Files:**
+- Modify: `backend/agents/conflict_critic.py:55-63`
+- Modify: `backend/pipeline/conflict_checker.py:85` (same pattern, classic path)
+
+**Step 1: Apply the extractor's reasoning-knob pattern**
+
+```python
+            parsed = call_json(
+                system_prompt=CONFLICT_SYSTEM_PROMPT,
+                user_text=instruction,
+                images_b64=images,
+                max_tokens=config.REASON_MAX_TOKENS,
+                model=config.AGENT_CONFLICT_MODEL,
+                reasoning_effort=config.EXTRACT_REASONING_EFFORT or None,
+                reasoning_max_tokens=config.EXTRACT_REASONING_MAX_TOKENS or None,
+                usage_tracker=self.usage,
+                usage_stage="agent.conflict",
+            )
+```
+
+(Match the exact kwarg names `SheetExtractorAgent` uses — check
+`backend/agents/extractors.py` for whether it passes `None` when the budget is
+0, and mirror that guard.)
+
+**Step 2: Run tests**
+
+Run: `.venv/bin/python -m pytest tests/ -q`
+Expected: all pass
+
+**Step 3: Commit**
+
+```bash
+git add backend/agents/conflict_critic.py backend/pipeline/conflict_checker.py
+git commit -m "fix: reasoning budget for conflict critic (wave-4 max_tokens truncation)"
+```
+
+---
+
+## Tests / validation
+
+1. `.venv/bin/python -m pytest tests/ -q` — full suite green.
+2. **Targeted repro of the original failure:** pull the S401 page image from job
+   959e16407573's output dir on sits-docker (or re-render the PDF page), then run
+   one `EvidenceVerifierAgent` scope locally against the finding JSON from
+   `validated_issues[3]`. Expected: verdict `corrected`,
+   `actual_text: "(5) 2x6 STUD PACK"`, status `refuted`.
+3. **End-to-end:** rerun the same Cypress TX PDF through the pipeline (local with
+   `LLM_CACHE`/`LLM_RAW_DUMP` per the conflict-checker skill). Expected:
+   - log shows `Agent wave 5b: evidence verification` with a bounded number of calls;
+   - the S401 stud-pack finding is either absent from `validated_issues` and present
+     in `suppressed_issues` with verification verdicts, or downgraded to low confidence;
+   - `agent.verify` appears in `summary.cost_by_stage`;
+   - zero `finish_reason=length` lines in wave 4 (Task 7).
+4. Cost check: wave 5b adds at most `AGENT_VERIFY_MAX_CHECKS` (20) vision calls —
+   for this job's profile that is well under $1.
+
+## Risks, tradeoffs, open questions
+
+- **Dispute false positives:** cluster members with legitimately different values
+  (e.g. two doors in one door cluster) will produce `disputed_attributes`. Mitigation:
+  prompts treat disputes as "unverified", not "wrong"; only severity-gated findings
+  burn verification calls. Tune later by restricting `find_disputes` to numeric-ish
+  values if noise is high.
+- **Verifier can also misread.** It is one model checking another with the same eyes.
+  Mitigation: verdict requires `actual_text` verbatim evidence for `corrected`, and
+  only fully-unconfirmed findings are suppressed (mixed keeps the finding with a note).
+- **xref cost:** extra link scopes. Bounded by the >= 2 distinct sheets gate and the
+  existing assertion cap; expect a handful of extra scopes per set.
+- **Suppression in review mode:** refuted findings land in `suppressed_issues` — the
+  review UI/finalizer must tolerate that list being non-empty (it is currently always
+  `[]` in agent mode). Open question: surface suppressed items in the human review
+  queue as informational, or keep them report-only?
+- **Open question:** should wave-4 conflict findings (which already saw images) also be
+  verification-eligible? Plan says no (they had the pixels); revisit if critics show
+  the same misread pattern.