woogi/Conflict_Checker
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Add required human review gate to the Agent pipeline.
Docker Release / build-and-push (push) Successful in 1m10s
Details
Docker Release / release (push) Skipped
Details

Browse Source 
Agent web jobs now stop after Brain consolidation and enter needs_review
with a persisted review queue (blocking: high-severity, low-confidence,
sensitive-category findings; audit sample of clean clusters). Humans
decide confirm/reject/unsure/needs_clarification via new review API and
frontend queue; a finalizer applies decisions (rejections suppressed with
reason codes), performs bounded targeted reruns for clarifications,
drafts RFIs only for kept issues, and only then marks the job done and
sends the final email. Two-phase email (review-required, then final
report), per-decision feedback labels with redacted aggregate metrics,
restart recovery from job artifacts, and CLI --no-review bypass.
Classic pipeline unchanged. 65 non-LLM tests.
This commit is contained in:
John Wilganowski
2026-07-28 19:23:57 +00:00
parent ac328d34fd
commit 1c1d2ff21b
27 changed files with 3344 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+46
View File
@@ -171,6 +171,52 @@ The `AGENT_*_CONCURRENCY` and scope-cap variables in `backend/.env.example`
bound fan-out and prompt size. Agent mode intentionally ignores the hybrid/local
text option in v1.
### Agent mode: required human review
By default (`AGENT_REQUIRE_REVIEW=true`) an Agent run **stops after the Brain
consolidation wave** and waits for a human before anything ships:
```
Brain merge -> needs_review -> review UI (/?job=<id>) -> finalize -> final report
```
The job lifecycle adds review states: `needs_review` (queue built, waiting),
`reviewing` (decisions submitted), `finalizing` (targeted reruns + RFI writers
running), then `done` — or `finalization_error` if finalization fails. Open the
job in the web UI to work the queue: blocking items (high/critical severity,
low confidence, sensitive categories) must be decided; clean-cluster items are
non-blocking spot-checks.
Email is **two-phase**: a "review required" notice goes out when the job enters
`needs_review` (with a link to the review UI); the final conflict report email
is only sent after finalization completes. The unreviewed report never leaves
the server.
**Privacy boundary:** all review artifacts (queue, decisions, final report) are
job-local under `outputs/<job_id>/review/`. Cross-job review-feedback
aggregation, when built, excludes verbatim `source_text`, images, and comments
unless `REVIEW_AGGREGATE_INCLUDE_TEXT=true`.
Config knobs (see `backend/.env.example`):
| Key | Default | Effect |
|-----|---------|--------|
| `AGENT_REQUIRE_REVIEW` | `true` | `false` = Agent jobs skip the gate entirely (old behavior: RFIs, final report, one email) |
| `AGENT_REVIEW_AUDIT_SAMPLE` | `5` | Max clean clusters added to the queue as spot-checks |
| `REVIEW_AGGREGATE_INCLUDE_TEXT` | `false` | Allow future aggregate feedback to include source text/images/comments |
From the CLI, `--no-review` bypasses the gate for that run (it overrides
`AGENT_REQUIRE_REVIEW=true`):
```bash
python cli/run_check.py samples/your_set.pdf --mode agent --no-review --out out/agent-run
```
**Deployment note:** the review endpoints (`/jobs/{id}/review-decisions`,
`/jobs/{id}/finalize-review`) are **state-changing and sensitive** — they accept
human decisions that alter the final report. Do **not** expose the UI/API
publicly without reverse-proxy auth or a shared access token in front of it.
Web UI (upload + view):
```bash