Watercooler
November 17, 2026
4 min read
4 min read

Legal AI Now Needs AI to Check Its AI. Who Checks That One?

AI drafts, AI checks, AI grades. Six hard truths about legal AI verification for in-house teams, and the one boring control that still works.

Legal AI Now Needs AI to Check Its AI. Who Checks That One?

Legal AI has reached its nesting doll era. We now have AI to draft the answer, AI to check the answer, AI to argue with the checker, and AI to grade the whole thing. Somewhere at the bottom of the doll, presumably, is a tired human with a bar number.

If you have sat through any vendor demo this quarter, you have seen the pattern. Cite-checkers that confirm an authority exists and says what the draft claims it says. Multi-model chains where one provider drafts and another audits. Adversarial "red team" passes. Masking and redaction layers that strip sensitive values before anything leaves the building. Benchmarks scored by panels of models voting on each other's work.

The checking layer is starting to look as big as the drafting layer. That is not a joke, it is a budget line.

Hard truths from the break room

1. "Human in the loop" now includes several robots

The loop got crowded. That is not necessarily bad — making different models challenge each other is sensible engineering, and a second opinion catches things a single confident model will not. But a chain of reviewers is not the same as a reviewer. Somebody still has to own the final answer, and no model has ever been deposed.

2. Hallucinated law is not only the other side's problem

The genre of news story where a lawyer files a brief full of citations that do not exist is no longer novel. In June 2023 a federal judge in Manhattan fined two lawyers and their firm $5,000 after they submitted a brief built on cases a chatbot invented, and there is now a public database that does nothing but track court decisions involving hallucinated citations.

The comfortable version of that story is that it happens to someone else. The uncomfortable version is that your own team is drafting with the same tools, on deadline, with less review than outside counsel gets.

It is hard to tut at the other side's brief when your own draft is in the evidence bag.

3. A benchmark is a marketing asset until you read the methodology

More vendors are publishing benchmark scores for contract review and research tasks. Read the methodology and you will often find that the outputs were graded by another model rather than by lawyers, or that the benchmark was built by someone with a product in the race.

None of that makes the numbers worthless. It makes the methodology the interesting part: who wrote the criteria, who scored the output, what the task actually was, and whether it resembles anything your team does on a Tuesday. If a deck leads with a score and buries the method, the score is a claim, not a result.

4. Nobody budgeted for the checker

Every extra model in the chain costs tokens, latency and patience. Run a four-stage review on every NDA and you have quietly multiplied your usage, and possibly your invoice, for a document nobody was worried about in the first place. Ask vendors to show consumption per task before you roll out, not after.

5. Each checker is another vendor review

The verification tool also processes your data. That means another DPA, another security questionnaire, another sub-processor list, another renewal. The irony of adding three vendors to reduce risk is not lost on the person who has to diligence all three.

6. The real control is still boring

Cite-check against a real authority database rather than a model's memory. Define which tasks AI is allowed to do unsupervised, which need review, and what gets escalated. Put a name on the sign-off. A clear workflow plus a sceptical lawyer beats a five-model chain that nobody reads.

"Trust, but verify" has become "verify, then verify the verifier, then ask legal ops which budget code the verifier goes under."

What to actually do on Monday

  • Write a one-page verification standard. What gets cite-checked, who reviews what, and which work product never leaves the department without a second human read.
  • Ask vendors how they check their own output. "We use a better model" is not an answer. "Here is our verification step, here is what it caught, here is what it missed" is.
  • Point research tools at a real database. A general-purpose model recalling case law from memory is a creative writing exercise with footnotes.
  • Match the control to the stakes. Four-model review on a filing, one quick human read on a routine NDA. Verification should scale with consequence, not with enthusiasm.
  • Log your near-misses. The fake citation someone caught before it went out is the most useful training material you will ever have. Write it down, anonymise it, share it at the team meeting.
  • Keep a name on it. Whatever your stack looks like, one person signs off on anything that leaves the department.

The bottom line

AI checking AI is real progress, and a multi-model review chain is a better safety net than hope and a deadline. Just do not let the checking layer become an elaborate way of not checking: the sign-off still belongs to a human who can be asked about it under oath. More on verification, cite-checking and legal research and practical guidance in the CorporateLegal.tech directory.

This article is general information, not legal advice.

Sources

  1. courtlistener.com
  2. damiencharlotin.com
Aggregator X Webflow Template - Icon
CorporateLegal.tech

The CorporateLegal.tech AI editorial team covers the trends, tools and hard-won lessons shaping modern corporate legal departments.