Depth Is a Disagreement

Judgment rarely fails for want of a sharper instrument. More often it fails for want of a second view, taken from somewhere else. That is the argument of this essay, and it is older than the machines we are now worrying about.

One field experiment puts the worry into focus. Researchers asked 228 participants, a mix of MIT Solve evaluators and students, to screen submissions from a set of 48, and used an expert panel’s judgments as the benchmark. The experiment compared unaided screening with AI recommendations shown alone or alongside a generated justification. With the justification, evaluators were more likely to reject submissions the expert panel would have advanced. The Harvard Business Review piece that reports the study¹ draws the uncomfortable conclusion: the explanation, meant to help people check the machine, helped them stop checking.

I recognise the temptation in my own work. I spend my days running AI agents, and an agent rarely returns a bare result. It returns a result and a paragraph of fluent reasoning, and the paragraph reads like evidence. It is one eye describing what it saw.

The star that would not give its distance

For two centuries after Galileo, astronomers knew how a star’s distance could be measured and could not manage it. A star should shift very slightly against the stars behind it as the earth travels around the sun, and the shift was too small for anyone to catch.

Friedrich Bessel caught it in 1838. Using a precision heliometer, he measured 61 Cygni against nearby background stars again and again for more than a year, and found the small wobble that the earth’s orbit put into its position. From that wobble came a distance of about ten light years. The modern figure is eleven and a bit. His instrument mattered, and so did the thing the instrument was for: comparing the same star from two ends of a very long baseline. However good the instrument, a single vantage point returns a flat sky.

We carry the same principle in our heads. The English essayist G.K. Chesterton, describing what keeps an ordinary person sane, said that person’s sight is “stereoscopic, like his physical sight: he sees two different pictures at once and yet sees all the better for that.” Closing one eye removes one source of depth while the world stays perfectly sharp. That is the state of an evaluator reading a confident explanation. Nothing looks blurred, and something has gone missing.

Two eyes, one picture

So we can make a second view a routine part of reviewing an agent’s work. The important words are different family. In a 2025 study of correlated errors², models from the same provider, or with the same base architecture, tended to make the same mistakes, and judges tended to inflate the accuracy of weaker models, especially those from their own provider or family. A model grading its sibling is one eye agreeing with itself.

Order matters as much as family. A second model that reads the first answer before thinking is mostly checking that answer’s homework. We give the second model the original task and the source material first, let it form its own view, and only then show it the first answer and ask it to find what is wrong.

Two views can also build something together. In Together AI’s layered system, responses from different models contributed more than repeated responses from one model³. That study merged the answers into one. My own practice adds a step before any merging: I read where the two families part company, because the disagreement tells me what to check next.

There is a catch, and the same correlated-errors study names it: “larger and more accurate models have highly correlated errors, even with distinct architectures and providers.” Two strong models from different houses can still share a blind spot. Crowds behave the same way. When people could see each other’s estimates, researchers found⁴ the crowd lost the “diversity of opinions” that made it wise. A second eye only helps while it still stands somewhere else, so we keep checking that it does.

That is the useful way to read this week’s news. Mistral says its Large 4 public preview was trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in its own European datacenters⁵, and it promises the weights by the end of the month. A model built in a separate place by a separate team is a candidate for a second reader. Whether it catches the mistakes our current model misses is something we find out by testing it on our own work.

Keeping the baseline

For a leader who does not run agents, the translation is direct. The second reader of an AI recommendation should stand somewhere else: another discipline, other incentives, another model family, another decade of experience. A colleague who trusts the same tool for the same reasons may share our blind spots, so it is worth asking what they checked on their own.

The fix is cheap and old. Whether the decision sits in a Montreal boardroom or a two-person startup, we can start on Monday with one decision that matters this month. We put it to two models from two different houses, neither seeing the other’s answer, and read the two replies side by side before reading either explanation. Then we take the one claim they disagree on and check it against the original evidence ourselves, before anyone signs.