Project 02 / 10 · Agents

Contract Risk Reviewer

Multi-agent system that reviews contracts like a junior counsel — clause extraction, risk scoring and plain-English recommendations in minutes, not billable hours.

01 / Context

The problem

Contract review needs both repeatable rules and enough flexibility to catch non-standard wording. The reviewer combines a deterministic risk catalog with an optional LLM pass, producing traceable findings instead of uncited classifications.

02 / System workflow

From input to output

  1. 01PDF / TXT
  2. 02Clause extraction
  3. 03Rule check
  4. 04Optional LLM check
  5. 05Typed risk report

03 / Architecture

Inside the system

A1

A LangGraph pipeline extracts clauses, applies deterministic checks, runs an optional Gemini review, and assembles the final report.

A2

Every finding is validated through Pydantic and includes severity, verbatim evidence, page or section anchors, and a suggested redline.

A3

Reports export to PDF, DOCX, or JSON; a separate hybrid retrieval path supports free-form questions over the contract.

04 / Engineering decisions

What shaped the build

Decision 01

Graceful degradation

The rules engine always produces a complete report. Missing keys, quota failures, malformed model output, or unavailable LangGraph never block review.

Decision 02

Precision-first baseline

Deterministic rules carry the reproducible guarantees, while the LLM layer is explicitly used to improve recall on varied language.

Decision 03

Visible provenance

The interface identifies whether a result came from rules alone or the combined LLM and rules path.

05 / Evidence

Measured, not implied

100%

Curated precision

Measured on 53 labeled clauses including 16 hard negatives.

0%

Hard-negative FPR

Standard clauses that resemble risks were not incorrectly flagged.

26.9%

CUAD recall

Honest precision-first baseline across 1,150 mapped clauses from 510 contracts.

06 / Delivery

What shipped

  • 01The lightweight Streamlit application runs offline with the deterministic engine and can enable Gemini when a key is available.
  • 02CI runs tests and evaluation gates across Python 3.10–3.12.
  • 03A Docker image and slim dependency set support reliable Streamlit Cloud or Hugging Face deployment.

Constraint documented

The rule engine intentionally has low recall on diverse real-world wording. Its CUAD score documents the exact gap the optional LLM extraction pass is designed to close.