What Large Language Models Actually Do to a Contract — and What They Don’t

Ask a room of lawyers what a large language model does when it “reviews” a contract and you will get two answers, both wrong. The optimists describe something close to a diligent associate. The sceptics describe autocomplete with a marketing budget. The truth sits in an awkward middle, and knowing exactly where it sits is the difference between a tool that saves you a day and a tool that costs you a client.

This piece is an attempt at an honest technical account, written for practitioners rather than engineers. No mathematics, but no hand-waving either.

What actually happens when you upload an agreement

A modern contract review system is not one model. It is a pipeline, and most of the pipeline is not a language model at all.

First, the document has to become text. For a native PDF exported from Word this is mechanical. For a scanned counterpart signed in 2014, photographed at an angle, it means optical character recognition, and OCR introduces errors at a rate that depends heavily on scan quality. A good system reports its OCR confidence; a bad one silently produces findings from text that says something the page does not.

Second, the text has to be segmented. This is more subtle than it sounds. A commercial agreement is a hierarchical document with numbered clauses, sub-clauses, schedules, annexes, cross-references and a definitions section that governs the meaning of everything downstream. Splitting it into arbitrary blocks of a few hundred words — which is what a naive implementation does — destroys exactly the structure a lawyer relies on. Clause 14.3 stops knowing that it is subject to clause 14.1.

Third, and only third, the language model appears. Its job is classification and extraction: given this segment, what kind of provision is it, and what are its operative parameters? Is this an indemnity? Mutual or one-way? Capped? Subject to which exclusions?

Fourth, the outputs are compared against a reference — market-standard language, or your own playbook — and scored for deviation and materiality. This step is mostly rules and retrieval, not generation.

What the model is genuinely good at

Three things, and they matter more than the sceptics allow.

Recognition despite variation. A limitation of liability clause can be drafted a thousand ways. Keyword search finds the ones that say “limitation of liability”. A language model finds the one buried in clause 22.7 that says neither party’s aggregate financial exposure arising from the performance or non-performance of its obligations shall exceed a defined sum — and correctly identifies it as the cap. This robustness to paraphrase is the single biggest practical gain over the previous generation of tools.

Structured extraction. Turning prose into fields is genuinely hard and genuinely useful. “Either party may terminate on ninety (90) days’ written notice, save that during the Initial Term such right shall not apply” becomes: termination for convenience, mutual, 90 days, written notice required, suspended during initial term. That structure is what lets you query a portfolio rather than read it.

Consistency at volume. A model applies the same standard to document 1 and document 4,000. A team of six reviewers, three of whom joined last month, does not. On large reviews this is often worth more than raw accuracy: variance is what produces the finding nobody caught.

Four failure modes to design around

Now the uncomfortable part. These failures are not rare edge cases; they are structural properties of how the technology works, and any deployment that does not account for them will eventually produce an embarrassing result.

1. Fluent invention

A language model generates plausible continuations. When the source does not contain the answer, the mechanism that produces correct text and the mechanism that produces confident nonsense are the same mechanism. This is why grounding matters: a system that must cite the passage supporting each finding cannot invent a clause that is not there, because there is nothing to cite. A system that produces a free-text summary with no anchors can and eventually will.

2. Absence blindness

Models are far better at characterising what is present than at noticing what is missing. Ask “is there an assignment clause?” and the honest answer requires reasoning over the entire document, not over a retrieved passage. Missing-clause detection therefore has to be built as its own checklist-driven step, run against the full clause inventory. If your vendor cannot explain how missing clauses are detected, assume they are not.

3. Definition drift

Contracts are self-referential. “Confidential Information” means whatever clause 1 says it means, and clause 1 may exclude precisely the category you care about. When segments are analysed in isolation, defined terms are read at face value. Good systems resolve definitions before classification; the difference shows up on exactly the documents where it matters most, which are the heavily negotiated ones.

4. Confidence that does not track accuracy

Models are frequently most fluent when least reliable, particularly on unusual drafting. A calibrated system reports lower confidence on genuinely hard passages. An uncalibrated one reports 95% on everything, which is the same as reporting nothing.

The useful question is not “how often is it right?” but “when it is wrong, can I tell?” A tool that is right 90% of the time and flags its uncertainty is safe. A tool that is right 96% of the time with uniform confidence is not.

What this means for how you deploy it

Three practical consequences follow.

Use it for the first pass, not the last. The economics of review are dominated by locating relevant provisions, not by deciding what to do about them. Automating location is where the return is. Automating the decision is where the liability is.

Require citations, always. A finding without a source passage is an assertion. Insist that every output can be opened in place against the underlying document, and build your review workflow so that checking is one click rather than a search.

Measure reviewer disagreement, not just model accuracy. The metric that predicts whether a deployment succeeds is how often reviewers override the tool and whether those overrides cluster. Clustering means a systematic weakness you can fix — usually with a playbook change rather than a model change.

Where this is going

The direction of travel is towards systems that do less generation and more retrieval, verification and structured reasoning — because that is what makes output checkable. The interesting research is not in making models more fluent. It is in making them able to say, precisely and reliably, “I could not find this, and here is what I looked at.”

For practitioners, the takeaway is unglamorous but durable: treat these systems as extremely fast, extremely consistent, occasionally overconfident junior colleagues whose work you would never send out unread. That framing gets you most of the value and almost none of the risk.