Conflict_Checker/.hermes/plans/2026-08-18_120500-extraction-coverage-guarantee.md

Extraction Coverage Guarantee — Implementation Plan

For Hermes: Use subagent-driven-development skill to implement this plan task-by-task.

Goal: Eliminate dark sheets (missed pages) and vision-misread content by making wave-1 extraction coverage-guaranteed: deterministic coverage measurement, a text-first retry ladder, deterministic fallback extraction, and text-layer sheet identity recovery.

Architecture: For text-bearing sheets the authoritative alphanumeric content already exists in the PyMuPDF text layer (backend/text_layer.py). Today the LLM transcribes from pixels and we merely detect failure post-hoc (coverage_gaps logs; nothing retries). This plan flips wave 1 to: run the vision pass (unchanged, always, on every page) → measure text coverage deterministically per page → if below floor, ADD a text-only structuring pass (LLM segments the text layer, no image, no misreads possible) and MERGE its objects into the vision results — vision keeps everything it found, text structuring fills what it missed → if still below floor, emit deterministic stub objects straight from the text layer so NO text-bearing page ever contributes zero objects. Sheet identity is recovered from the text layer when the LLM drops the header. Vision stays the only source for graphical content (symbols, geometry, line work) and the only path for scanned pages.

Tech Stack: Python 3.14, PyMuPDF (already a dep), existing call_json LLM plumbing, pytest.

---

Root-Cause Diagnosis (why this keeps happening)

Confirmed against Cypress job 3e01d5baba32 (38-page Verizon set) and code:

Missed sheets (pages 8 = S202 wood notes, 10 = S204 lap-splice tables, 18 = A102 REFLECTED CEILING PLAN — zero assertions each):

1. The extractor prompt (backend/prompts.py:277) is biased toward physical "construction objects" (rooms, doors, fixtures). Notes/table-dense sheets have few, so the model returns a bare array with ONE generic summary object (log: wrapping bare objects array (1 items, no sheet header)).
2. The grounding guard (backend/pipeline/extractor.py:178 _is_grounded) drops that summary object as ungrounded → 0 objects.
3. _wrap_bare_list (backend/agents/extractors.py:33) converts the 1-item bare array into a valid dict, so the compact retry (extractors.py:71) NEVER fires — it only triggers when parsing fully fails. A 1-object page counts as "success".
4. coverage_gaps() (backend/text_layer.py:211) only LOGS the gap and adds a failed_scope note. No retry, no fallback. The page is silently dark for every downstream wave.
5. Sheet identity comes ONLY from the LLM reading the title block in the image. 7/38 Cypress pages ended with sheet_number=None (4 of them WITH assertions: pages 22, 30, 31, 37), so they can't join sheet-keyed scopes and corrupt missing_expected_sheets downstream.

Completely incorrect information:

1. Vision misreads of dense alphanumeric content (the "(2) vs (5) 2x6 STUD PACK" family). The wave-1.5 rescue tier catches invented numbers but is a SET subset test — it cannot catch SWAPPED numbers (documented in docs/superpowers/specs/2026-08-12-text-layer-grounding-design.md).
2. Gemini thinking tokens count against max_tokens → recovered truncated JSON silently drops tail objects (bottom/right of sheet vanishes). Nothing flags the page as degraded.
3. The model paraphrases source_text; the guard only checks digit-run/token overlap, so plausible-but-wrong values pass.
4. JSON parse error (giving up) → classic path returns an empty "extraction failed" sheet (extractor.py:282-291); the page vanishes from analysis while sheets_analyzed still counts it.

Cornerstone principles for the fix:

1. If a page has a text layer, the truth is already deterministic and free. The LLM's job on such pages is STRUCTURING, not TRANSCRIPTION. Every extracted alphanumeric claim must trace to the text layer; anything that can't is vision-only and gets stamped as such.
2. The vision pass is never skipped and never replaced. These are construction documents: symbols, device/fixture locations, geometry, and line work exist only in the image. The text-only rung and the fallback rung are strictly ADDITIVE — they merge into the vision results (deduped by normalized source_text), so a rescue can only add coverage, never subtract graphical content.

---

Task 1: Coverage metric module (backend/text_coverage.py)

Objective: Deterministic per-page coverage measurement: what fraction of the text layer is actually represented in extracted objects.

Files:

• Create: backend/text_coverage.py
• Test: tests/test_text_coverage.py

Step 1: Write failing test

# tests/test_text_coverage.py
from backend.text_coverage import text_coverage, segment_text_layer, fallback_objects

def test_coverage_full():
    text = "NOTE 1\nALL LUMBER NO. 2 SOUTHERN PINE\nNOTE 2\nUSE 5/8\" PLYWOOD"
    objects = [{"source_text": "ALL LUMBER NO. 2 SOUTHERN PINE"},
               {"source_text": "USE 5/8\" PLYWOOD"}]
    cov = text_coverage(text, objects)
    assert cov["covered_lines"] == 2
    assert cov["total_lines"] == 2
    assert cov["ratio"] == 1.0

def test_coverage_zero_on_empty_objects():
    cov = text_coverage("LINE A\nLINE B\nLINE C", [])
    assert cov["ratio"] == 0.0 and cov["total_lines"] == 3

def test_coverage_ignores_short_and_numeric_noise_lines():
    text = "15\"\n19\"\nA\nB\nREAL NOTE ABOUT FRAMING HERE"
    cov = text_coverage(text, [{"source_text": "REAL NOTE ABOUT FRAMING HERE"}])
    # short/noise lines (< MIN_LINE_CHARS or pure dimension ticks) excluded
    assert cov["total_lines"] == 1 and cov["ratio"] == 1.0

def test_segment_notes_and_rows():
    text = "WOOD CONSTRUCTION\n1. \nALL SAWN LUMBER TO BE SOUTHERN PINE.\n2. \nROOF SHEATHING 5/8\" PLYWOOD."
    segs = segment_text_layer(text)
    assert any("ALL SAWN LUMBER" in s for s in segs)
    assert any("ROOF SHEATHING" in s for s in segs)

def test_fallback_objects_verbatim_and_stamped():
    objs = fallback_objects("1. \nALL SAWN LUMBER TO BE SOUTHERN PINE.", page_number=8)
    assert len(objs) == 1
    assert objs[0]["source_text"] == "ALL SAWN LUMBER TO BE SOUTHERN PINE."
    assert objs[0]["grounding"] == "text_layer_fallback"
    assert objs[0]["confidence"] == "low"

def test_merge_objects_keeps_vision_and_unions_text():
    vision = [
        {"source_text": "2X6 WD STUD @ 16\" O.C.", "object_type": "wall"},
        {"source_text": None, "graphical_basis": "light fixture symbol, grid C-4",
         "object_type": "lighting_fixture"},  # graphical: exists only in image
    ]
    text = [
        {"source_text": "2X6 WD STUD @ 16\" O.C.", "object_type": "wall"},  # dup
        {"source_text": "ALL LUMBER NO. 2 SOUTHERN PINE", "object_type": "general_note"},
    ]
    merged = merge_objects(vision, text)
    assert len(merged) == 3                       # dup dropped, note added
    assert any(o.get("graphical_basis") for o in merged)   # graphical kept
    assert merged[0]["object_type"] == "wall"     # vision order preserved

def test_merge_objects_dedupes_by_normalized_text():
    a = [{"source_text": "RTU-1: 5 TON, 1600 CFM"}]
    b = [{"source_text": "rtu 1 5 ton 1600 cfm"}]   # same content, different case/punct
    assert len(merge_objects(a, b)) == 1

Step 2: Run test to verify failure

Run: .venv/bin/python -m pytest tests/test_text_coverage.py -v Expected: FAIL — ModuleNotFoundError: backend.text_coverage

Step 3: Implement backend/text_coverage.py

"""text_coverage.py - deterministic extraction-coverage measurement.

The coverage guarantee: for any page with a usable text layer, measure how
much of that layer ended up represented in extracted objects. Pages below
the floor route into the extraction retry ladder (agents/extractors.py and
pipeline/extractor.py). fallback_objects() is the last rung: stub objects
segmented straight from the text layer so no text-bearing page goes dark.
"""

import re
from typing import Dict, List

# Lines below this many meaningful chars are noise (dimension ticks, grid
# bubbles, single letters) and excluded from the coverage denominator.
MIN_LINE_CHARS = 12
# Pure dimension/elevation ticks like 15" or 8' - 0" carry no prose content.
_TICK_RE = re.compile(r"^[\d\s'\"/.,-]+$")
_WORD_RE = re.compile(r"[a-z0-9]+")


def _meaningful_lines(text: str) -> List[str]:
    lines = []
    for raw in (text or "").splitlines():
        line = " ".join(raw.split())
        if len(line) < MIN_LINE_CHARS or _TICK_RE.match(line):
            continue
        lines.append(line)
    return lines


def _norm(text: str) -> str:
    return " ".join(_WORD_RE.findall((text or "").lower()))


def text_coverage(page_text: str, objects: List[Dict]) -> Dict:
    """Fraction of meaningful text-layer lines whose normalized form appears
    in the concatenated normalized source_text of extracted objects."""
    lines = _meaningful_lines(page_text)
    if not lines:
        return {"total_lines": 0, "covered_lines": 0, "ratio": 1.0}
    haystack = " ".join(
        _norm(str(o.get("source_text") or o.get("object_description")
                  or o.get("value") or ""))
        for o in objects if isinstance(o, dict)
    )
    covered = sum(1 for ln in lines if _norm(ln) and _norm(ln) in haystack)
    return {
        "total_lines": len(lines),
        "covered_lines": covered,
        "ratio": covered / len(lines) if lines else 1.0,
    }


def segment_text_layer(text: str) -> List[str]:
    """Segment a page text layer into note-sized blocks: numbered notes and
    contiguous prose runs. PyMuPDF emits each note number on its own line
    ('1. ', '2. ') followed by wrapped text lines; rejoin number->body and
    merge continuation lines until the next number or blank-line break."""
    segments: List[str] = []
    buf: List[str] = []
    number_re = re.compile(r"^(\d{1,2}[.)]?|[A-Z]\d{0,2}[.)]?)\s*$")

    def flush():
        joined = " ".join(buf).strip()
        if len(joined) >= MIN_LINE_CHARS:
            segments.append(joined)
        buf.clear()

    for raw in (text or "").splitlines():
        line = raw.strip()
        if not line:
            flush()
            continue
        if number_re.match(line):
            flush()
            buf.append(line.rstrip(".)"))
            continue
        buf.append(line)
        # wrapped-note heuristic: a line starting a new sentence after a
        # period ends the segment
        if line.endswith(".") and len(" ".join(buf)) > 120:
            flush()
    flush()
    return segments


def fallback_objects(page_text: str, page_number: int,
                     max_objects: int = 200) -> List[Dict]:
    """Last-rung deterministic extraction: one stub object per text segment,
    source_text verbatim from the text layer. confidence=low and
    grounding=text_layer_fallback make their provenance explicit downstream."""
    objs = []
    for idx, seg in enumerate(segment_text_layer(page_text)[:max_objects]):
        objs.append({
            "object_id": f"p{page_number}-tl{idx}",
            "object_type": "general_note",
            "category": "general",
            "tag": None,
            "name": seg[:80],
            "description": seg,
            "attributes": {},
            "location_key": {},
            "source_text": seg,
            "graphical_basis": None,
            "review_uses": ["code_review", "constructability_review"],
            "confidence": "low",
            "grounding": "text_layer_fallback",
        })
    return objs


def merge_objects(vision_objs: List[Dict], text_objs: List[Dict]) -> List[Dict]:
    """Union of vision and text-structured objects. Vision results come first
    and are never dropped (graphical_basis objects exist only in the image).
    Text objects are appended unless their normalized source_text is already
    represented. A merge can only add coverage, never subtract it."""
    merged = list(vision_objs or [])
    seen = {_norm(str(o.get("source_text") or ""))
            for o in merged if isinstance(o, dict)}
    seen.discard("")
    for obj in text_objs or []:
        if not isinstance(obj, dict):
            continue
        key = _norm(str(obj.get("source_text") or ""))
        if key and key in seen:
            continue
        seen.add(key)
        merged.append(obj)
    return merged

Step 4: Run test to verify pass

Run: .venv/bin/python -m pytest tests/test_text_coverage.py -v Expected: 7 passed

Step 5: Commit

git add backend/text_coverage.py tests/test_text_coverage.py
git commit -m "feat: deterministic text-layer coverage metric + fallback extraction"

---

Task 2: Sheet identity recovery from the text layer

Objective: When the LLM drops/misreads the sheet header, recover sheet_number (and discipline via existing discipline_from_sheet_number) deterministically from the text layer instead of leaving None.

Files:

• Modify: backend/text_coverage.py (add recover_sheet_number)
• Test: tests/test_text_coverage.py (add tests)

Step 1: Write failing test

def test_recover_sheet_number_from_title_block():
    text = ("WALL SECTIONS\n...\nSheet Information\nS301\n"
            "Issue Date 05.29.26\nProject Number 25177")
    assert recover_sheet_number(text) == "S301"

def test_recover_sheet_number_none_when_absent():
    assert recover_sheet_number("just some notes about lumber") is None

def test_recover_prefers_discipline_pattern_over_dates():
    # 05.29.26 and 25177 must never match
    text = "Issue Date 05.29.26\nProject Number 25177\nA102 REFLECTED CEILING PLAN"
    assert recover_sheet_number(text) == "A102"

Step 2: Run to verify failure

Run: .venv/bin/python -m pytest tests/test_text_coverage.py::test_recover_sheet_number_from_title_block -v Expected: FAIL — ImportError

Step 3: Implement in backend/text_coverage.py

# Sheet ids: 1-2 uppercase letters + 2-3 digits + optional decimal suffix
# (S301, A102, M200, E500, LS101, P100, G000). Deliberately excludes pure
# numbers (dates, project numbers) and long alphanumerics (member marks).
_SHEET_ID_RE = re.compile(r"\b([A-Z]{1,2}\d{2,3}(?:\.\d+)?)\b")
_TITLE_HINT_RE = re.compile(
    r"(?i)sheet\s*(?:information|no|number)?|"
    r"(floor plan|ceiling plan|elevations?|sections?|details?|schedule|"
    r"notes|legend|plan)")

def recover_sheet_number(page_text: str) -> Optional[str]:
    """Deterministic sheet id from the text layer. Strategy: collect every
    sheet-id-shaped token, prefer ones appearing near title words or in the
    last ~15%% of the page (title block lives at the drawing edge)."""
    text = page_text or ""
    cands = _SHEET_ID_RE.findall(text)
    if not cands:
        return None
    tail = text[int(len(text) * 0.85):]
    for cand in reversed(_SHEET_ID_RE.findall(tail)):
        return cand
    return cands[0]

(Add from typing import Optional to the imports.)

Step 4: Run to verify pass

Run: .venv/bin/python -m pytest tests/test_text_coverage.py -v Expected: all pass (10 tests)

Step 5: Commit

git add backend/text_coverage.py tests/test_text_coverage.py
git commit -m "feat: deterministic sheet-number recovery from text layer"

---

Task 3: Text-only structuring prompt (no image)

Objective: Second rung of the ladder: give the LLM the raw text layer and ask it to segment EVERY note/row/callout into objects with verbatim source_text. No image = no vision misreads for alphanumerics; far cheaper than the vision pass.

Files:

• Modify: backend/prompts.py (append after EXTRACTOR_USER_INSTRUCTION, ~line 282)
• Test: tests/agents/test_extraction_ladder.py (prompt-content assertions only; rendering tested in Task 4)

Step 1: Write failing test

# tests/agents/test_extraction_ladder.py
from backend.prompts import TEXT_STRUCTURING_SYSTEM_PROMPT, TEXT_STRUCTURING_USER_INSTRUCTION

def test_text_structuring_prompt_demands_verbatim_and_completeness():
    assert "verbatim" in TEXT_STRUCTURING_USER_INSTRUCTION.lower()
    assert "every" in TEXT_STRUCTURING_USER_INSTRUCTION.lower()
    assert "{text_layer}" in TEXT_STRUCTURING_USER_INSTRUCTION

Step 2: Run to verify failure

Run: .venv/bin/python -m pytest tests/agents/test_extraction_ladder.py -v Expected: FAIL — ImportError

Step 3: Append to backend/prompts.py

# ---------------------------------------------------------------------------
# Stage 2b - text-only structuring (extraction retry ladder, rung 2)
# ---------------------------------------------------------------------------

TEXT_STRUCTURING_SYSTEM_PROMPT = """You are a construction document structuring engine.
You receive the deterministic text layer extracted from one drawing sheet. It is complete and authoritative.
Your ONLY job is to segment it into structured objects. You are NOT reading an image. You must NOT invent, complete, or correct any text.
Rules:
- Every numbered note, schedule row, callout, tag, legend entry, and title-block field becomes its own object.
- source_text must be copied VERBATIM from the input, character-for-character. Never paraphrase.
- Cover the ENTIRE input. Omitting a note is a failure. When unsure of an object's type, use general_note with confidence low.
- Numbers, model numbers, dimensions, and tags must appear in source_text exactly as in the input.
Respond only with valid JSON."""

TEXT_STRUCTURING_USER_INSTRUCTION = """Segment this sheet's text layer into structured construction objects.
Respond ONLY with a valid JSON object - no markdown fences:
{ "sheet": { "sheet_number": "string or null", "sheet_title": "string or null", "discipline": "string or null", "drawing_type": "string or null", "level": "string or null", "scale": "string or null" }, "objects": [ { "object_id": "string", "object_type": "room | door | window | wall | finish | ceiling | dimension | grid | callout | keynote | general_note | equipment | plumbing_fixture | mechanical_equipment | electrical_device | lighting_fixture | structural_element | schedule_reference | symbol | abbreviation", "category": "architectural | structural | mechanical | electrical | plumbing | code | general", "tag": "string or null", "name": "string or null", "description": "string or null", "attributes": { "attribute_name": "attribute_value" }, "location_key": { "room_number": "string or null", "grid": "string or null", "detail_reference": "string or null" }, "source_text": "VERBATIM text copied from the input", "graphical_basis": null, "review_uses": [ "schedule_comparison", "cross_discipline_coordination", "code_review", "constructability_review" ], "confidence": "high | medium | low" } ], "unresolved_items": [] }
Optional sheet hint: {sheet_hint}

TEXT LAYER (segment ALL of it):
{text_layer}"""

NOTE the two render sites you will add in Tasks 4-5 substitute {sheet_hint} and {text_layer} with str.replace directly (NOT via render()/call_stage) — this matches the wave-1.5 pattern and avoids the classic-path literal-placeholder leak documented in the project pitfalls.

Step 4: Run to verify pass

Run: .venv/bin/python -m pytest tests/agents/test_extraction_ladder.py -v Expected: 1 passed

Step 5: Commit

git add backend/prompts.py tests/agents/test_extraction_ladder.py
git commit -m "feat: text-only structuring prompt for extraction retry ladder"

---

Task 4: Retry ladder in the AGENT path (SheetExtractorAgent)

Objective: Replace the binary parse-fail retry with a coverage-driven ladder: vision pass → coverage check → text-only structuring pass → deterministic fallback. Also recover sheet identity and mark truncation-degraded pages.

Files:

• Modify: backend/agents/extractors.py:63-86 (SheetExtractorAgent.run)
• Modify: backend/config.py (new knobs, below)
• Test: tests/agents/test_extraction_ladder.py

New config knobs (backend/config.py, follow existing env pattern):

EXTRACT_COVERAGE_FLOOR = float(os.getenv("EXTRACT_COVERAGE_FLOOR", "0.6"))
EXTRACT_TEXT_RETRY_ENABLED = os.getenv("EXTRACT_TEXT_RETRY_ENABLED", "true").lower() == "true"
EXTRACT_FALLBACK_ENABLED = os.getenv("EXTRACT_FALLBACK_ENABLED", "true").lower() == "true"
EXTRACT_FALLBACK_MAX_OBJECTS = int(os.getenv("EXTRACT_FALLBACK_MAX_OBJECTS", "200"))

Step 1: Write failing test

from backend.agents.extractors import SheetExtractorAgent
from backend.agents.base import AgentScope, AgentUsage

def _page(n=8, text="1. \nALL SAWN LUMBER IN CONTACT WITH SOIL TO BE SOUTHERN PINE, PRESSURE TREATED.\n2. \nROOF SHEATHING: 5/8\" PLYWOOD, C-D GRADE, STRUCTURAL I."):
    return {"page_number": n, "base64": "AAAA", "text_layer": text}

def test_ladder_falls_back_when_vision_returns_nothing(agent_monkeypatch):
    # vision pass returns 1 summary object that the guard drops;
    # text-structuring disabled to exercise the deterministic rung
    agent_monkeypatch.setattr("backend.agents.extractors.call_json",
                              lambda **kw: [{"name": "general notes", "value": "notes"}])
    agent_monkeypatch.setattr("backend.config.EXTRACT_TEXT_RETRY_ENABLED", False)
    agent = SheetExtractorAgent(AgentUsage())
    scope = AgentScope(scope_id="sheet:8", payload={"page": _page(), "sheet_hint": ""})
    result = agent.run(scope)
    sheet = result.artifacts[0]
    assert sheet["assertions"], "dark sheet must be impossible with fallback enabled"
    assert all(a.get("grounding") == "text_layer_fallback" for a in sheet["assertions"])
    assert sheet["coverage"]["ratio"] >= 0.6

def test_ladder_merge_preserves_graphical_objects(agent_monkeypatch):
    # vision finds a graphical symbol + misreads nothing; text rung adds notes.
    # The graphical object MUST survive the merge.
    calls = {"n": 0}
    def fake_call_json(**kw):
        calls["n"] += 1
        if kw.get("images_b64"):  # vision pass
            return {"sheet": {}, "objects": [
                {"object_id": "g1", "object_type": "lighting_fixture",
                 "name": "pendant at grid C-4", "source_text": None,
                 "graphical_basis": "16in pendant symbol at grid C-4"}]}
        return {"sheet": {}, "objects": [  # text-structuring pass
            {"object_id": "t1", "object_type": "general_note",
             "source_text": "ALL SAWN LUMBER IN CONTACT WITH SOIL TO BE SOUTHERN PINE, PRESSURE TREATED.",
             "name": "lumber note"}]}
    agent_monkeypatch.setattr("backend.agents.extractors.call_json", fake_call_json)
    agent = SheetExtractorAgent(AgentUsage())
    scope = AgentScope(scope_id="sheet:8", payload={"page": _page(), "sheet_hint": ""})
    sheet = agent.run(scope).artifacts[0]
    assert any(a.get("graphical_basis") for a in sheet["assertions"])
    assert any("SAWN LUMBER" in (a.get("source_text") or "") for a in sheet["assertions"])

def test_ladder_recovers_sheet_number_from_text_layer(agent_monkeypatch):
    agent_monkeypatch.setattr(
        "backend.agents.extractors.call_json",
        lambda **kw: {"sheet": {}, "objects": [
            {"object_id": "o1", "name": "RCP note",
             "source_text": "GYP. BD. CEILING 8'-11 3/8\" A.F.F. TYP. FOR ALL STOREFRONT",
             "attributes": {"height": "8'-11 3/8\""}}]})
    agent = SheetExtractorAgent(AgentUsage())
    scope = AgentScope(scope_id="sheet:18",
                       payload={"page": _page(18, "REFLECTED CEILING PLAN\nA102\nGYP. BD. CEILING 8'-11 3/8\" A.F.F. TYP. FOR ALL STOREFRONT"),
                                "sheet_hint": ""})
    sheet = agent.run(scope).artifacts[0]
    assert sheet["sheet_number"] == "A102"

(Monkeypatch fixture: plain unittest.mock.patch context or pytest monkeypatch; follow tests/agents/test_text_layer_flow.py patterns for scope/result construction — check AgentScope/AgentResult signatures in backend/agents/base.py before writing.)

Step 2: Run to verify failure

Run: .venv/bin/python -m pytest tests/agents/test_extraction_ladder.py -v Expected: FAIL — assertions on coverage/sheet_number fail (ladder not implemented)

Step 3: Implement the ladder in backend/agents/extractors.py

Replace SheetExtractorAgent.run (lines 63-86) with:

    def _text_structuring_call(self, page, sheet_hint):
        from backend.prompts import (TEXT_STRUCTURING_SYSTEM_PROMPT,
                                     TEXT_STRUCTURING_USER_INSTRUCTION)
        instruction = (TEXT_STRUCTURING_USER_INSTRUCTION
                       .replace("{sheet_hint}", str(sheet_hint or ""))
                       .replace("{text_layer}",
                                (page.get("text_layer") or "")
                                [:config.TEXT_LAYER_MAX_CHARS]))
        return call_json(
            system_prompt=TEXT_STRUCTURING_SYSTEM_PROMPT,
            user_text=instruction,
            images_b64=None,
            max_tokens=config.EXTRACT_MAX_TOKENS,
            model=config.AGENT_EXTRACT_MODEL,
            usage_tracker=self.usage,
            usage_stage="agent.extract_text",
            reasoning_effort=config.EXTRACT_REASONING_EFFORT or None,
            reasoning_max_tokens=config.EXTRACT_REASONING_MAX_TOKENS or None,
        )

    def run(self, scope: AgentScope) -> AgentResult:
        from backend.text_coverage import (fallback_objects, merge_objects,
                                           recover_sheet_number, text_coverage)
        try:
            page = scope.payload["page"]
            hint = scope.payload.get("sheet_hint") or ""
            page_text = page.get("text_layer")
            instruction = EXTRACTOR_USER_INSTRUCTION.replace(
                "{sheet_hint}", str(hint)) + _text_layer_block(page)

            # Rung 1: vision pass (unchanged behaviour, incl. compact retry)
            parsed = _wrap_bare_list(self._call(instruction, page),
                                     page["page_number"])
            if not isinstance(parsed, dict):
                print(f"[Extract] Page {page['page_number']}: full extraction "
                      f"failed, retrying compact")
                parsed = _wrap_bare_list(
                    self._call(instruction + _COMPACT_RETRY_SUFFIX, page),
                    page["page_number"])
            if not isinstance(parsed, dict):
                parsed = {"sheet": {}, "objects": []}

            sheet = _normalize_sheet(parsed, page["page_number"],
                                     page_text=page_text)
            cov = text_coverage(page_text or "", sheet["assertions"])
            sheet["coverage"] = cov

            # Rung 2: text-only structuring when coverage is below floor.
            # MERGE, never replace: vision keeps every object it found
            # (graphical_basis content exists only in the image); the text
            # pass fills in the text content the vision pass missed.
            if (page_text and config.EXTRACT_TEXT_RETRY_ENABLED
                    and cov["ratio"] < config.EXTRACT_COVERAGE_FLOOR):
                print(f"[Extract] Page {page['page_number']}: coverage "
                      f"{cov['ratio']:.0%} < floor - text-only structuring pass")
                parsed2 = _wrap_bare_list(
                    self._text_structuring_call(page, hint), page["page_number"])
                if isinstance(parsed2, dict):
                    sheet2 = _normalize_sheet(parsed2, page["page_number"],
                                              page_text=page_text)
                    before = len(sheet["assertions"])
                    sheet["assertions"] = merge_objects(sheet["assertions"],
                                                        sheet2["assertions"])
                    # Fill header gaps the vision pass left null
                    for key in ("sheet_number", "sheet_title", "discipline",
                                "level", "scale", "drawing_type"):
                        if not sheet.get(key) and sheet2.get(key):
                            sheet[key] = sheet2[key]
                    cov = text_coverage(page_text, sheet["assertions"])
                    sheet["coverage"] = cov
                    print(f"[Extract] Page {page['page_number']}: merged "
                          f"{len(sheet['assertions']) - before} text-structured "
                          f"object(s), coverage now {cov['ratio']:.0%}")

            # Rung 3: deterministic fallback - dark sheets are impossible.
            # Also merged (deduped) so stub notes never double up with
            # objects the earlier rungs already captured.
            if (page_text and config.EXTRACT_FALLBACK_ENABLED
                    and cov["ratio"] < config.EXTRACT_COVERAGE_FLOOR):
                stubs = fallback_objects(page_text, page["page_number"],
                                         config.EXTRACT_FALLBACK_MAX_OBJECTS)
                stubs = _normalize_sheet({"sheet": {}, "objects": stubs},
                                         page["page_number"],
                                         page_text=page_text)["assertions"]
                before = len(sheet["assertions"])
                sheet["assertions"] = merge_objects(sheet["assertions"], stubs)
                print(f"[Extract] Page {page['page_number']}: fallback merged "
                      f"{len(sheet['assertions']) - before} text-layer stub(s)")
                sheet["coverage"] = text_coverage(page_text,
                                                  sheet["assertions"])

            # Identity recovery: never leave a text-bearing page sheet-less
            if not sheet.get("sheet_number") and page_text:
                recovered = recover_sheet_number(page_text)
                if recovered:
                    sheet["sheet_number"] = recovered
                    sheet["discipline"] = (
                        __import__("backend.pipeline.extractor",
                                   fromlist=["discipline_from_sheet_number"])
                        .discipline_from_sheet_number(recovered)
                        or sheet.get("discipline") or "Unknown")
                    print(f"[Extract] Page {page['page_number']}: sheet number "
                          f"recovered from text layer -> {recovered}")

            return AgentResult(scope_id=scope.scope_id, artifacts=[sheet])
        except Exception as exc:
            return failure(scope, exc)

Step 4: Run to verify pass

Run: .venv/bin/python -m pytest tests/agents/test_extraction_ladder.py -v Expected: all pass

Step 5: Commit

git add backend/agents/extractors.py backend/config.py tests/agents/test_extraction_ladder.py
git commit -m "feat: coverage-driven extraction retry ladder (agent path)"

---

Task 5: Same ladder in the CLASSIC path (pipeline/extractor.py)

Objective: The classic pipeline (_extract_one, backend/pipeline/extractor.py:273-293) must get the identical ladder — two render paths share everything, per the documented project trap.

Files:

• Modify: backend/pipeline/extractor.py:273-293
• Test: tests/test_text_layer_flow.py or new tests/test_extraction_ladder_classic.py

Step 1: Write failing test — mirror Task 4's tests against _extract_one directly (monkeypatch backend.pipeline.extractor.call_json).

Step 2: Run to verify failure

Run: .venv/bin/python -m pytest tests/test_extraction_ladder_classic.py -v Expected: FAIL

Step 3: Implement — same ladder shape as Task 4 but inside _extract_one; the text-structuring call here uses default model (no model= kwarg, matching existing _extract_one call_json usage). Keep the existing "extraction failed" empty-sheet shape for pages with NO text layer (scanned pages stay vision-only and may legitimately return empty).

Step 4: Run to verify pass

Run: .venv/bin/python -m pytest tests/test_extraction_ladder_classic.py -v Expected: all pass

Step 5: Commit

git add backend/pipeline/extractor.py tests/test_extraction_ladder_classic.py
git commit -m "feat: coverage-driven extraction retry ladder (classic path)"

---

Task 6: Verbatim-source stamping upgrade

Objective: When a text layer exists, check each object's source_text against the page text with the existing fuzzy machinery; stamp grounding="vision_unverified" when it doesn't match so the wave-5b verifier prioritizes it. Cheap upgrade, reuses text_layer._tokens — no new call sites.

Files:

• Modify: backend/pipeline/extractor.py (_normalize_sheet, ~line 182 where grounding is stamped)
• Test: extend tests/test_extractor_text_grounding.py

Step 1: Failing test — object whose source_text is NOT a fuzzy substring of the page text keeps the object (guard passes via digits) but gets stamped vision_unverified.

Step 2: Run, expect FAIL.

Step 3: Implement — in _normalize_sheet, when page_text is present and no grounding stamp yet: normalized source_text (via backend.text_coverage._norm) not substring of normalized page text → grounding = "vision_unverified" (counted in the existing log line as a third counter).

Step 4: Run, expect PASS.

Step 5: Commit

git add backend/pipeline/extractor.py tests/test_extractor_text_grounding.py
git commit -m "feat: stamp vision-unverified source_text against text layer"

---

Task 7: Surface coverage in the report

Objective: report.summary gains per-job extraction-quality visibility so "is extraction healthy?" is answerable without log spelunking.

Files:

• Modify: backend/agents/runner.py (where summary is assembled) and/or backend/pipeline/report.py
• Test: extend the runner-level stub test (tests/agents/test_wave5b_suppression.py pattern)

Step 1: Failing test — runner-level: summary contains extraction_coverage = {"pages_below_floor": [...], "mean_ratio": float, "fallback_pages": [...]}.

Step 2-4: Implement by aggregating the coverage dicts Task 4/5 attach to each sheet; NO new ProjectMemory keys (closed registry trap) — compute at report assembly from the sheets list already in scope.

Step 5: Commit

git add backend/agents/runner.py backend/pipeline/report.py tests/
git commit -m "feat: extraction coverage summary in report"

---

Task 8: Full suite + Cypress validation run

Step 1: .venv/bin/python -m pytest tests/ -q — expected: all pass (128 + new).

Step 2: Push branch, wait for Gitea Actions sha- build, deploy per the skill's deploy runbook (compose pull + up -d --force-recreate).

Step 3: Resubmit the exact Cypress PDF (docker cp'd source.pdf preserved at /tmp/cypress-source.pdf on sits-docker): curl -F file=@source.pdf -F pipeline_mode=agent https://conchecker.scoutitsystems.com/check

Step 4: Acceptance criteria (compare against job 3e01d5baba32):

• Zero text-bearing pages with 0 assertions (was: pages 8, 10, 18).
• sheet_number present on >= 37/38 pages (was: 31/38).
• A102 RCP content (ceiling heights, tape lights, sconces) present in assertions.
• Spot-check: no regression in validated-issue quality — suppressed_issues and validated_issues counts within noise of the prior run; cost delta reported (expect +1 cheap text-only call per low-coverage page, ~$0 on healthy pages).
• report.summary.extraction_coverage.pages_below_floor is empty or every entry is a genuinely scanned page.

---

Files Touched (summary)

• Create: backend/text_coverage.py
• Modify: backend/prompts.py, backend/config.py, backend/agents/extractors.py, backend/pipeline/extractor.py, backend/agents/runner.py, backend/pipeline/report.py
• Tests: tests/test_text_coverage.py, tests/agents/test_extraction_ladder.py, tests/test_extraction_ladder_classic.py, extensions to tests/test_extractor_text_grounding.py and the runner-level stub test

Risks, Tradeoffs, Open Questions

• Fallback flood risk: 200 low-confidence stubs/page could flood downstream scopes. Mitigations: EXTRACT_FALLBACK_MAX_OBJECTS cap, confidence=low (specialists already weight confidence), fallback only fires below the coverage floor (3/38 pages on Cypress). If Brain merge gets noisy, lower the cap or restrict fallback to pages where rungs 1+2 BOTH return 0 objects.
• Cost: rung 2 adds one text-only call per low-coverage page (~12k input chars, no image) — negligible vs the 65k-token vision pass. Healthy pages skip it entirely.
• Sheet-id regex false positives: member marks like W12X26 are excluded by the 2-3-digit shape, but "S301" inside a detail reference ("2/S301") will match. Tail-of-page preference mitigates; wrong-but-present sheet_number is still strictly better than None for scope keying (sheet_index wave can correct it).
• Open question: should rung 2 route to the local model (aimax LM Studio) instead of the cloud extractor model to make retries free? Config knob AGENT_EXTRACT_TEXT_MODEL would allow it; not included in this plan (YAGNI until cost data from the validation run says otherwise).
• Explicit non-goal: graphical-only content (symbol geometry, line work) stays vision-based — text-layer-first cannot see it. Scanned PDFs (no text layer) keep today's behaviour plus the existing failed_scopes gap note.