AI hallucinations are getting harder to catch

You know that feeling when you’re putting together a sizeable report, maybe with contributions from other departments or even external stakeholders? Deadlines loom, everyone’s hunched over their computers, buried in draft after draft and sprinting for the finish line.

That’s usually when a Cosy Crow slips through, something that sounds so right nobody thinks to check it. Convincing, and not quite true, the way a crow can mimic another bird, a car alarm, even a human voice, well enough to fool anyone. By the time it’s caught, whoever signed off is left eating crow.

More to crow about

Wherever LLM-assisted drafting has become part of the workflow, and it has across almost every industry, Cosy Crows are turning up, including at some of the most credentialed organisations in the world.

In 2025, Deloitte Australia famously let a few past. Weeks after they delivered a government report, a university professor found around 20 references that either did not exist or were not factual¹.

Also in 2025, KPMG’s global report on agentic AI met the same fate: of 45 citations, only five checked out².

Closer to home, in March 2025, the Singapore High Court ordered two lawyers to pay S$5,000 each after their closing submissions cited cases that turned out to be entirely fictitious, invented by an AI tool during research³. Another lawyer was fined after a junior colleague’s AI-assisted research produced a citation that didn’t exist⁴.

It’s not a bug, so it can’t be patched

It would be reassuring to think that all Cosy Crows can be tackled with the next model update. They can’t be. Researchers at OpenAI published findings in 2025 showing hallucinations happen because of how LLM models are trained⁵.

Performance gets graded much like a multiple-choice exam: a wrong guess can still be marked right, while answering “I don’t know” guarantees zero marks. So the model learns exactly what any exam-taker would learn: never leave an answer blank.

Newer or more powerful models don’t reliably fix this. OpenAI’s own next-generation reasoning models actually hallucinated more often than the ones they replaced, not less⁶. Anthropic’s CEO Dario Amodei has said AI may hallucinate “in more surprising ways”⁷.

As LLMs get better at sounding right, the mistakes that slip through get more fluent and much harder for a tired or rushed reviewer to spot.

What the big AI labs are doing about it
OpenAI has started re-scoring its models so a confident wrong guess counts against them more heavily than admitting uncertainty⁵. Google DeepMind built a benchmark called FACTS Grounding to measure how closely models stick to the facts in front of them⁹. Anthropic has been mapping the internal “circuits” inside Claude to understand exactly what triggers a hallucination in the first place, essentially studying the model’s thinking to catch the moment it starts making things up⁸.

There is progress. But, every lab’s published data still points to the same conclusion, that none of today’s LLMs can be fully trusted to check themselves.

How to catch Cosy Crows

Most corporate reports are run in-house without a formal publishing structure and a clear, single owner for each essential role. While none of what’s tabled below guarantees a clean report, it could meaningfully improve your odds of catching more Cosy Crows.

Cosy Crows can slip past a hardworking team not just because of a lack of vigilance but because of structure, who owns what and when.

In our article, (working title)How to Run Your Annual Report Like a Newsroom, we look at how your in-house adhoc publishing team can borrow the structure of pro publishers so your next deadline sprint does not end with another one slipping through.

Don’t be left eating crow

If your team’s using AI to help draft reports, proposals or sustainability disclosures, tell us what you’re working on. We’ll help you build a proper AI content audit around it.