# How we achieve a near-zero OCR hallucination rate > Hallucinations are a fact of how LLMs work. For mission-critical documents, even one is too many. Here's the defense-in-depth system we use to drive them toward zero. - Canonical: https://www.datalab.to/blog/anti-hallucination - Published: 2026-06-30 - Authors: Vik Paruchuri Hallucinations are a product of how transformer models are trained, and how they generate text. If you're casually chatting with a model, the occasional made-up detail is tolerable. But when you're processing invoices, contracts, lab reports, or financial statements, a single fabricated number can carry real legal and financial consequences. Over the last few months we've invested heavily in keeping our hallucination rate as low as possible, and in building tools to enable manual review of any remaining potential hallucinations. Together, our systems achieve a near-zero hallucination rate on our internal benchmark. This post walks through the system that makes that possible. It's several independent layers, each catching what the previous one misses.

Raw OCR output hallucination risk on hard pages: high

1 · Better models & training fewer hallucinations at the source

2 · Page-level checks catch & regenerate bad pages

3 · Block-level checks heal a region, or mark it illegible

4 · Word confidence surface anything left for human review

Final output surviving hallucination rate: ≈ 0%

Defense in depth — each layer narrows what slips through.

## It starts with the model The cheapest hallucination to handle is the one that never happens. So the first line of defense is the OCR model itself — we train ours specifically to minimize hallucination risk, and we do research on architecture and training methods that make rare or ambiguous text easier to read correctly. One example we've published openly: **character-level tokenization**. By having the model generate one character at a time, we let it spend a constant, predictable amount of compute on every character, without the strong "autocomplete-the-common-word" bias that trips up BPE models. The payoff is that rare sequences — an unusual part number, a foreign name — get transcribed far more faithfully. Some of this work is open source in [Surya OCR 2](https://github.com/datalab-to/surya). Still, no model is perfect. On the hardest inputs — dense handwriting, degraded scans, unusual layouts — even a strong model occasionally fails. On an internal benchmark set of our most degeneration-prone pages, raw model output contains a hallucination on roughly **6 in 10 pages**. Next, we'll go into more detail on how we mitigate those.

Surviving hallucination / degeneration rate on the stress set, by layer of defense.

## Page-level checks After a page is OCRed, we score it on several quality axes that are good at exposing the signatures of a hallucinated page. If a page scores below threshold, we regenerate and re-check, repeating until it clears or we escalate to the next layer. In the benchmark set, the page-level layer flags about **67%** of pages as suspect, and resolves **a quarter** of those on its own. The rest get escalated for closer inspection. ## Block-level checks When a page can't be cleared at the page level, we zoom in. We split it into its individual blocks — a table, a paragraph, a signature block — and examine each one independently. Working at this granularity matters: a single bad region on an otherwise-perfect page is easy to miss at the page level but obvious up close, and re-reading just that region is both more accurate and far cheaper than redoing the whole page. For each suspect region we attempt to **heal** it — re-read it until we get a stable, confident transcription. When we can heal it, the corrected text flows through transparently. When we _can't_ — when the region is genuinely illegible and we can't produce a reading we're confident in, we mark it **illegible** instead. A wrong number that looks right is the most dangerous output a document system can produce. An explicit "we couldn't read this" is something your team — or your downstream automation — can catch, route, and handle.

Across all regions on the stress set

recovered

illegible

97.5% emitted as confident text 2.5% flagged illegible, never guessed

Even on the hardest pages, nearly every region is recovered; the small remainder is marked illegible rather than fabricated.

## Word confidence The final layer puts you in control. We recently shipped [word bounding boxes and confidence scores](/blog/word-bounding-boxes-and-confidence) — for **every word** on the page, you get its exact location and how confident the model is in the prediction. You can see at a glance where the model was confident and where it wasn't, and review the uncertain spots before they ever reach your downstream work. Word boxes on a scanned document in the Datalab playground. Every word is boxed in green; one low-confidence word is boxed in red.

Word boxes on a scanned document in the Datalab playground. Every word is boxed in green; one low-confidence word is boxed in red.

_Every word boxed and scored in the playground — the single low-confidence word is flagged in red for review._ ## Seeing it in action It's easiest to understand on real handwriting. Here are three samples from our stress set, run through the live system. **Recovering difficult cursive.** A handwritten letter from 1847. The script is hard, but it's legible, so we read it faithfully:

A handwritten letter in 19th-century cursive, dated New-York, Aug. 31, 1847. — Difficult but legible cursive — transcribed faithfully.

**Refusing to guess.** And a page where the handwriting is genuinely unreadable — a dense, struck-through manuscript draft. Rather than fabricate a plausible-looking transcription, the system flags it:

A dense, heavily struck-through handwritten manuscript draft that is largely unreadable. — When resampling can't agree on a confident reading, we flag the region instead of inventing one.

## How we measure it We maintain an internal benchmark built specifically around failure: **121 single-page samples mined from real-world documents**, each one a page we've actually seen push OCR into a hallucination. It's adversarial by design, not a representative sample of everyday work (where hallucinations are rare). We run every page multiple times and measure the fraction of runs that come back degenerate. Lower is better. | Configuration | Pages that degenerate | | ------------------------------ | --------------------- | | Raw model, single pass | ~61% | | + one sampling-level safeguard | ~41% | | Full multi-stage system | **≈ 0%** | Each safeguard we add moves the number down, and they compound. Running the **full system end-to-end** over 60 of these worst-case pages, we found **no genuine surviving hallucinations** — and across the **920 regions** on those pages, **97.5% were recovered as confident text** while **2.5% were flagged illegible**. ## The result Put together, these layers turn an unavoidable property of LLMs into something you can actually rely on. On our benchmark set of the hardest, most failure-prone pages: - Raw model output degenerates on roughly **6 in 10** pages. - After the full system, the surviving hallucination rate is **≈ 0%**. - Of everything we flag, **97.5%** is recovered automatically and the rest is marked illegible. On real customer documents — including handwritten records that broke earlier versions of our pipeline — the same system eliminates blanked pages and runaway repetition entirely, recovering legible content and clearly marking the rest. Hallucinations will always be a property of how these models work. Our job is to make sure they never quietly become _your_ problem. If you're processing mission-critical documents and want OCR you can trust — in the cloud or fully on-prem — [reach out to our team](/contact) or [try it yourself](/auth/sign_up).