1 changed files with 166 additions and 0 deletions
@@ -0,0 +1,166 @@
 # Text-Layer Grounding — Design Spec
 **Date:** 2026-08-12 · **Branch:** `agent-mode` · **Status:** approved by user (2026-08-12)
 ## Problem
 The pipeline is vision-only for extraction, but most CAD-produced drawing sets
 carry a real vector text layer. Two worst documented failure modes are text
 problems being solved with pixels:
 1. **Wave-1 text misreads propagate immutably** — e.g. job `959e16407573`:
   vision read "(2) 2x6 STUD PACK" where the sheet says "(5)"; text-only
   downstream specialists treated the misread as ground truth → confident
   false-positive findings.
 2. **Silent extraction loss** — failed/under-extracted pages are invisible
   (job `475a6f184dd1`: 42% extraction loss), producing false
   `missing_expected_sheets` warnings and missed conflicts.
 Priority (user, 2026-08-12): reduce false positives **and** missed items;
 more accurate conflicts.
 ## Approach
 Extract the PDF text layer deterministically (PyMuPDF) once per job, and make
 it a first-class citizen at three points: extractor grounding, the grounding
 guard, and the wave-5b verifier (as text oracle + high-DPI evidence crops).
 Inspired by `hamzaabduljabbar/construction-drawing-analyzer` (patterns only —
 its license is source-available/no-resale; all code here is original).
 ## Components
 ### 1. New module `backend/text_layer.py` (deterministic, no LLM)
 - `extract_text_layers(pdf_path) -> Dict[int, dict]` — per 1-based page:
  `{"text": str, "words": [{"text", "bbox": (x0,y0,x1,y1)}, ...],
  "has_text_layer": bool}`. Pages with < `TEXT_LAYER_MIN_CHARS` of text are
  `has_text_layer=False` (scanned/raster sheets stay vision-only; logged).
 - `find_evidence_bbox(words, needle) -> bbox | None` — best-effort fuzzy
  substring match of an evidence `source_text` against word sequence; returns
  union rect of matched words.
 - `render_crop(pdf_path, page_number, bbox, dpi, margin_pts) -> bytes` —
  PyMuPDF `page.get_pixmap(clip=rect, dpi=dpi)` → JPEG bytes.
 Both runners call `extract_text_layers` right after `convert_pdf_to_images`
 and attach `page["text_layer"] = <text or None>` to each page dict. Word
 positions stay in a separate `page_words: Dict[int, list]` runner-local map
 (not attached to page dicts — they get serialized).
 ### 2. Extractor grounding (both pipelines)
 - Static paragraph added to `_EXTRACTOR_SYSTEM_TEMPLATE` in
  `backend/prompts.py` (no new placeholder): when a TEXT LAYER block is
  present in the user message it is **authoritative for alphanumeric content**
  (counts, dimensions, member tags, notes); the image is for geometry,
  symbols, linework, and anything absent from the text layer.
 - Text-layer content is **appended programmatically** at each extractor call
  site (classic `extractor.py::_extract_one`, agent
  `extractors.py::SheetExtractorAgent.run`) — NOT a new `{placeholder}` in the
  shared template (two-render-path trap: `render()` silently leaves missing
  keys as literals). Block capped at `TEXT_LAYER_MAX_CHARS`.
  Format: `\n\nTEXT LAYER (authoritative for alphanumeric content — trust it
  over the image for numbers, tags, and note text):\n<text>`
 ### 3. Grounding guard rescue tier (`pipeline/extractor.py::_normalize_sheet`)
 Current guard drops an object when its primary value's digit-runs aren't in
 its own `source_text`. New tier, only when a text layer exists for the page:
 - digits ⊆ source_text → keep (unchanged)
 - digits ⊆ page text layer but ⊄ source_text → keep, stamp
  `grounding: "text_layer"` on the assertion (recall rescue — vision quoted
  imperfectly but the value is real page text)
 - otherwise → drop (unchanged)
 `_is_grounded` gains an optional `page_text` param; existing callers/tests
 unaffected. Dropped/ rescued counts logged per page.
 ### 4. Verifier: text oracle + high-DPI crops (wave 5b)
 Wherever verify scopes are built (agent runner confirmed; classic runner to be
 checked — integrate at both if present):
 - Scope payload gains `text_layer_excerpt`: concatenated text of the finding's
  cited sheets, capped at `VERIFY_TEXT_MAX_CHARS`. `VERIFY_USER_INSTRUCTION`
  gains a `{text_layer}` placeholder with instructions to treat it as
  deterministic page text (verdicts may cite it as `actual_text`). **Both
  render sites** (agent verifier + any classic-path render) must substitute it
  — grep the template name across `backend/agents/` and `backend/pipeline/`.
 - When `VERIFY_HI_DPI_CROPS` and the page has words: for each evidence item,
  `find_evidence_bbox` on the cited page's words; on hit, `render_crop` at
  `VERIFY_CROP_DPI` with margin → crop images replace full-page images (up to
  `AGENT_CONFLICT_MAX_IMAGES`). On any miss/failure → fall back to the current
  full-page image. Zero-resolved-images ⇒ scope skipped (I2 guard preserved).
 ### 5. Coverage signal (recall)
 After extraction in both runners: for each page with `has_text_layer=True`
 whose extraction failed or returned 0 objects, log
 `[TextLayer] Page N: text layer present (M chars) but no objects extracted —
 possible extraction gap` and add the page to the existing gap-finding path
 (agent: `orchestrator.stats.failed_scopes`-style finding; classic: log only).
 ## Config knobs (`backend/config.py`, env-overridable, documented in `.env.example`)
 | Key | Default | Effect |
 |-----|---------|--------|
 | `TEXT_LAYER_ENABLED` | `true` | Master switch |
 | `TEXT_LAYER_MIN_CHARS` | `20` | Below this per page → `has_text_layer=False` |
 | `TEXT_LAYER_MAX_CHARS` | `12000` | Cap per sheet injected into extractor prompt |
 | `VERIFY_TEXT_MAX_CHARS` | `8000` | Cap of text-layer excerpt in verify scope |
 | `VERIFY_HI_DPI_CROPS` | `true` | Evidence-located crops in verifier |
 | `VERIFY_CROP_DPI` | `300` | Crop render DPI |
 | `VERIFY_CROP_MARGIN_PTS` | `36` | Padding around evidence bbox (PDF points) |
 ## Known traps (from project history — designed around)
 - **Two render paths:** no new `{placeholder}` in extractor templates; the one
  new placeholder (`{text_layer}` in VERIFY_USER_INSTRUCTION) substituted at
  every render site; a render test asserts no `{...}` literals remain.
 - **ProjectMemory closed registry:** no new memory keys. Text artifacts dump
  via plain file writes under `outputs/<job>/text/` (agent: under `agent/`).
 - **`slim_clusters`:** no new cluster fields — unchanged.
 - **I2 zero-image path:** crops replace full-page images only on confident
  bbox match; never reduce image count to zero.
 - **Base64 hygiene:** page dicts already carry base64; `text_layer` strings
  must not leak into `clusters.json` dumps — reuse `_without_base64` pattern
  if assertions ever carry page refs (they don't today).
 ## Dependencies
 `PyMuPDF>=1.23` added to `requirements.txt` (Docker image rebuild picks it up;
 pdf2image/poppler unchanged).
 ## Testing
 - `tests/test_text_layer.py` — build tiny PDFs with PyMuPDF in-test:
  extraction, `has_text_layer` thresholds, `find_evidence_bbox` hit/miss,
  `render_crop` dimensions.
 - Extractor guard: rescue-tier unit tests (keep-with-flag, still-drop,
  unchanged behavior without text layer).
 - Prompt render test: extractor + verify instructions fully substituted at
  every site (both pipelines).
 - Runner-level (pattern from `tests/agents/test_wave5b_suppression.py`):
  stubbed waves, assert text layer reaches extract scopes and verify scopes
  (excerpt present, crop fallback on no-match), full `run_agent_pipeline`.
 - Full `pytest tests/` green before push.
 ## Validation (post-deploy)
 Re-run the Cypress set (source PDF persists at
 `/app/backend/outputs/959e16407573/source.pdf` on sits-docker) per the
 documented re-run workflow. Success criteria:
 1. The "(2) vs (5)"-class findings are not generated, or are verifier-refuted
   with text-layer evidence cited.
 2. Coverage-gap log lines appear for any page with text but no objects.
 3. No new `finish_reason=length` in waves 1/4; cost delta reported vs
   baseline job.
 ## Out of scope (future PRs)
 - Legend/symbol-library wave injected into extractor + critic prompts.
 - Deterministic schedule-row recall pass (text-layer tables → assertions).
 - pdf-markup export for the review UI.
 - Takeoff/polygon geometry (belongs to AI_Takeoffs, not this product).