Indra OCR 1.1: handwriting, 80+ languages and pages at any angle

Today we’re releasing Indra OCR 1.1. The first release was fast on clean print. 1.1 is still small (a 1M-parameter OCR model, 7M parameters for the whole default pipeline), it still runs on CPU with no GPU, and it reads a lot more: handwriting, 80+ languages on scans, and pages at any angle.

On OmniParseBench, the headline goes from 62.3 to 66.7, and from 67.0 to 71.4 with --math.

math: with --math. All scores come from OmniParseBench’s own harness, the same one behind the public leaderboard. 1.0’s scores differ slightly from our first post, which used our own harness.

The gains show up across the board, including tables (+4.8) and degraded scans (+4.7).

Handwriting

1.0 didn’t read handwriting. 1.1 does: OmniParseBench handwriting tests go from 8.3% to 25.2%.

Here’s a handwritten cursive letter from OmniParseBench. 1.0 turned it into a table of fragments. 1.1 reads every word:

Indra OCR 1.1 output

The committee will meet again on Tuesday evening at seven.

Enclosed please find the receipt for the amount paid in full. Please forward the enclosed papers to the county clerk before Friday.

Indra OCR 1.0 output

the emmiee wil meer aqain - Iueeday evening asaeven,

80+ languages

1.0 could OCR Latin, Cyrillic, Greek, Devanagari, Bengali and Chinese. Everything else came only from a digital PDF’s text layer. 1.1 OCRs Arabic, Hebrew, Japanese, Korean, Thai, Lao, Khmer, Myanmar, Sinhala, the Indic scripts, Georgian, Armenian and Ethiopic too. On held-out printed pages in 95 languages, 1.1 reaches a character F1 of 0.90 or higher on 83 of them.

Indra OCR 1.1 output

코로나19 확산방지 대응을 위한 사목서신(8)

주님의 평화!

성령께서 우리를 부르시어 주님의 몸된 교회를 이루게 하시고, 거룩한 길로 이끌어주심을 감사하고 찬양합니다.

Indra OCR 1.0 output

旦旦中19 卧社增本 研名音 华卧 斗号州召(8)

The same happens with Thai script. 1.0 forced this academic calendar into Latin letters. 1.1 reads it, table rows included:

Indra OCR 1.1 output

ปฏิทินการศึกษา ระดับประกาศนียบัตรวิชาชีพ ปีการศึกษา 2563 มหาวิทยาลัยเทคโนโลยีราชมงคลพระนคร

(ฉบับแก้ไข)

1. วันเปิดภาคการศึกษา	1 ก.ค. 63	16 พ.ย. 63		29 มี.ค 64
2. วันปิดภาคการศึกษา	11 พ.ย. 63	22 มี.ค. 64		31 พ.ค. 64

…

Indra OCR 1.0 output

Ugfunnsfnen szðuusznnnüoTnsivntn Unnsfnen 2563 un nonáunalulamvusnanszuns (aWuunlo)

And with Hebrew script, which 1.0 read as letters and digits:

Indra OCR 1.1 output

1 ברישא ברא יי בלשמא ולארעא והות כתיב 2 ועל ארעא הוה חשוכא …

Indra OCR 1.0 output

bb) 2 bpb pb) fDObb fobbb = fob fDb 1 fD) fOOp bb bb FOODb) …

Per script, on OmniParseBench:

Most scripts have few tests, so one test moves a score by 2 to 8 points. Myanmar and Sinhala are new too. Latin-script languages gain as well, such as Vietnamese (65.4 to 80.8) and Hausa (69.6 to 91.3).

Pages at any angle

Scans and phone photos are rarely straight. 1.1 straightens pages tilted up to 45 degrees, and turns sideways and upside-down pages upright, so a page at any angle reads. In our tests, a page tilted 30 degrees went from almost no text in 1.0 to the same accuracy as an upright page. Light text on a dark background, like slides and dark table headers, reads too.

Here’s a memo tilted 30 degrees. 1.0 returned nothing. 1.1 finds the angle and reads the page exactly as it reads the upright page:

Indra OCR 1.1 output

EXECUTIVE OFFICE OF THE PRESIDENT OFFICE OF MANAGEMENT AND BUDGET WASHINGTON, D.C. 20503

December 18, 2019

MEMORANDUM FOR USERS OF OMB CIRCULAR NO. A-11

FROM: Hee Jun Chief, Budget Concepts Branch SUBJECT: Revisions to OMB Circular No. A-11 dated June 28, 2019

We have updated the electronic version of OMB Circular No. A-11 to reflect some technical revisions.

…

Indra OCR 1.0 output

(empty)

Here’s a table printed sideways on a journal page. 1.0 got nothing useful from it. 1.1 turns it upright and reads the whole table:

Indra OCR 1.1 output

Table 2. Impact of cancer on COVID-19 outcomes in closed cases.

Characteristics		Discharge		p-value	…
Age	Mean ± SD	58.0 ± 15.1	63.8 ± 13.2	0.026	…
Gender	Female	80 (57.1)	20 (47.6)	0.29	…

…

Chinese layout

Chinese text blocks are no longer mistaken for tables. On OmniDocBench, 1,651 Chinese and English pages read in OCR mode, the text error on Chinese pages drops by 19%, and Chinese pages that wrongly got a table go from 129 to 89 of 493:

Speed

1.1 reads far more, and it still runs on CPU. On OmniParseBench, one page at a time on an x86 machine with 4 vCPUs, the median page takes 212 ms, or 357 ms with --math. That’s about 16% more time per page than 1.0 on the same machine. Digital PDFs with a good text layer still skip OCR and stay fast.

On an x86 AMD EPYC (Genoa) machine with no GPU, in OCR mode with one single-threaded worker per core, 1.1 reads about 44 pages per second on 32 cores. That’s 15x Tesseract and 10x PaddleOCR’s PP-OCR text models on the same machine type:

300 OmniParseBench pages, OCR mode. PP-OCR text models: RapidOCR PP-OCRv6 small.

Quickstart

pip install -U indra-ocr
indra-ocr paper.pdf                   # markdown to stdout
indra-ocr scan.png --format json      # lines, boxes, layout blocks
indra-ocr notes.jpg --math            # math as LaTeX

Limitations

• Handwriting: clear writing reads well. Cursive and faint pencil are harder.
• Complex tables: tables with merged cells or multi-level headers can lose some structure.
• Diacritics: in a few Latin-script languages with many diacritics, such as Czech, Turkish and Kurmanji, a letter can come out with a similar diacritic.
• Text-layer-only scripts: Tibetan, Mongolian and Cantonese-only characters come from a PDF’s text layer, not from OCR.

The full list is in the README.

Try it

• Code: github.com/datalab-to/indra
• Models: huggingface.co/datalab-to/indra-ocr
• Benchmark: OmniParseBench

Tell us what you run it on, and where it breaks.