# Datalab benchmarks

> Datalab measures its parsers on OmniParseBench, an open benchmark of 16,288
> pass/fail tests on 2,937 pages in 92 languages, scored by open code with no
> model as judge. Each OmniParseBench score is the mean of the text, tables,
> and layout pass rates. olmOCR-bench results are reported with each model's
> own postprocessing.

- Page: https://www.datalab.to/benchmarks
- Dataset: https://huggingface.co/datasets/datalab-to/omni_parse_bench
- Scorer and docs: https://github.com/datalab-to/omni_parse_bench
- Current results (Datalab production runs, 2026-10-23): https://github.com/datalab-to/omni_parse_bench/blob/main/docs/results.md

## Overall (OmniParseBench)

OmniParseBench is our open parsing benchmark: 16,288 pass/fail tests on 2,937 pages in 92 languages. The headline is the mean of three family scores (text, tables and layout), and an open-code scorer checks every test with no model judging the outputs.

Datalab accurate scores 94.1. Coverage: 16,288 tests · 2,937 pages · 92 languages. Leaderboard run: 2026-10-08. A row with a date in its note is from a later run of that model.

| Parser | Score | Note |
| --- | ---: | --- |
| Datalab accurate | 94.1 | production run, Oct 23 |
| Gemini 3.8 Flash* | 92.9 | |
| Datalab balanced | 92.6 | production run, Oct 23 |
| Reducto | 90.8 | |
| LlamaParse | 90.4 | |
| Mistral OCR | 89.5 | |
| Claude Sonnet 5.5* | 88.7 | |
| GPT-5.6 Sol | 87.8 | |
| Extend | 83.3 | |
| Azure | 83.0 | |

Source: https://www.datalab.to/blog/omni-parse-bench

## Tables (OmniParseBench · table tag)

Every OmniParseBench test that touches a table: 6,767 tests from 875 documents. A table_cell test passes only when a cell sits under the right column and row headings and beside the right neighbors, so merged cells, spanning headers and split tables all count.

Datalab accurate scores 95.6. Coverage: 6,767 tests · 875 documents. Leaderboard run: 2026-10-08. A row with a date in its note is from a later run of that model.

| Parser | Score | Note |
| --- | ---: | --- |
| Datalab accurate | 95.6 | production run, Oct 23 |
| Extend | 94.7 | |
| Datalab balanced | 94.2 | production run, Oct 23 |
| Gemini 3.8 Flash* | 93.8 | |
| LlamaParse | 93.7 | |
| Reducto | 92.7 | |
| Claude Sonnet 5.5* | 92.0 | |
| GPT-5.6 Sol | 91.6 | |
| Mistral OCR | 91.5 | |
| Azure | 86.1 | |

Source: https://www.datalab.to/blog/omni-parse-bench

## Multi-column (OmniParseBench · multi_column tag)

Every OmniParseBench test on a multi-column page: 3,517 tests from 635 documents. Order tests check that text comes out in reading order across columns, and layout tests check that headings, text and tables are told apart.

Datalab accurate scores 94.9. Coverage: 3,517 tests · 635 documents. Leaderboard run: 2026-10-08. A row with a date in its note is from a later run of that model.

| Parser | Score | Note |
| --- | ---: | --- |
| Datalab accurate | 94.9 | production run, Oct 23 |
| Datalab balanced | 92.7 | production run, Oct 23 |
| Gemini 3.8 Flash* | 92.3 | |
| LlamaParse | 91.4 | |
| Reducto | 91.3 | |
| Mistral OCR | 89.6 | |
| Extend | 88.3 | |
| GPT-5.6 Sol | 87.4 | |
| Claude Sonnet 5.5* | 86.5 | |
| Azure | 83.8 | |

Source: https://www.datalab.to/blog/omni-parse-bench

## olmOCR-bench (olmOCR-bench · Chandra 2.5)

olmOCR-bench unit-tests OCR output on arXiv papers, old scans, tables, multi-column pages and tiny text. Chandra 2.5 is the model behind the Datalab API. We report each model with its own postprocessing, because benchmark-specific postprocessing flatters scores.

Chandra 2.5 scores 86.9. Coverage: olmocr 0.4.27 · olmOCR-bench revision 54a96a6 · headers and footers left out. Leaderboard run: 2026-10-14. A row with a date in its note is from a later run of that model.

| Parser | Score | Note |
| --- | ---: | --- |
| Chandra 2.5 | 86.9 | Datalab API |
| Chandra OCR 2 | 85.8 | open source |
| LightOnOCR-3-4B | 85.4 | our reproduction |

Source: https://www.datalab.to/blog/chandra-2-5

* Used in creating and verifying at least some OmniParseBench tests.