Of course vendors only publish benchmarks they win. I collected 33 of them in one place so you can see them side by side.
Written 16 November 2026. Everything below reflects the leaderboards as captured on 15 and 16 November 2026. Because live leaderboards change over time, each benchmark links both to a Wayback Machine snapshot from the capture date and to its current live page.
Disclosure: I’m a contractor at Artifex, which makes PyMuPDF and PyMuPDF4LLM. This post is written in my personal capacity and is not an Artifex publication. An earlier, shorter version appeared on the Artifex blog.
TL;DR
- This post collects 33 benchmarks published by companies that sell PDF-to-Markdown or document extraction products, where the company’s own product appears on the leaderboard.
- On 29 of the 33 benchmarks, the publisher’s product is ranked #1.
- On 2 more, the publisher ranks #2 or #3 behind general-purpose frontier models (GPT or Gemini variants), while the benchmark still frames the publisher’s own product as the more practical choice on cost and reliability.
- That brings the total to 31 of 33 (about 94%). Only 2 benchmarks rank the publisher below a direct competitor or publish no overall winner: Baidu’s Real5-OmniDocBench and Parsebridge’s PDF Parser Benchmark.
- This is not a surprising finding. Why would a vendor spend time and money publishing a benchmark where its own product does not come out on top? Even with completely honest evaluation, there is an obvious selection bias in what gets published.
- Across every benchmark, I only look at the headline number, or the default metric used to rank parsers on the page. Many boards also include sub-metrics, category tabs, or cost and latency breakdowns where rankings shift.
- Every benchmark below includes the original leaderboard screenshot, a link to the live page, and a Wayback Machine snapshot. If you want to amend an existing benchmark or add a new one, email me at contact@hashiromer.com.
If you shop for a document parser in 2026, you will quickly notice that almost every vendor has its own benchmark. Some are standalone leaderboard websites with their own domains. Others live in GitHub READMEs, Hugging Face dataset cards, or product launch blog posts. Many release open datasets, evaluation scripts, and per-document scores.
Put them side by side and you see the exact result you would expect: the company that publishes the benchmark is almost always at the top of it.
There is nothing mysterious about this. No company has an incentive to build, run, and promote a benchmark where its product finishes fourth. Even without any thumb on the scale, there is an implicit selection bias in what reaches the public: internal evaluations where a product trails its rivals stay internal, or serve as an engineering checklist until the product is strong enough on that task to publish. Vendors also naturally design benchmarks around the document types, layouts, and failure modes they built their systems to solve.
Still, because these benchmarks are scattered across dozens of blogs, dataset cards, and microsites, it is useful to have them all in one place. I collected 33 vendor-published benchmarks, archived their leaderboards on the Wayback Machine on 15 and 16 November 2026, and lined them up below, ordered from the most open and comprehensive living leaderboards to smaller launch-post evaluations.
Across the collection, the headline results fall into three buckets:
- The publisher is #1 outright (29 benchmarks): #1 ParseBench 2, #2 ExtractBench, #4 PulseBench-Tab, #6 RealDoc-Bench, #8 WholeDoc Bench, #9 OmniExtractBench, #10 DocuBench, #11 SCORE-Bench, #12 OmniAI OCR benchmark, #13 DocParsingBench, #14 IDP Leaderboard, #15 BizDocBench, #16 LongArray-Extract, #17 OpenDataLoader Bench, #18 DP-Bench, #19 RD-TableBench, #20 DeepRead extraction composite, #21 LongExtractionBench, #22 ChartParse-Bench, #23 OCRFlux-bench, #24 Parse Index, #26 Handwriting OCR Benchmark 2026, #27 ParseArena / PAIR, #28 Five-hard-document Open Benchmark, #29 120-document OCR benchmark, #30 NuMind internal evaluations, #31 Legal document parsing benchmark, #32 Mistral OCR internal evaluations, and #33 Enterprise Document Benchmark.
- A frontier generalist edges out #1 on raw score, while the publisher is framed as the practical choice (2 benchmarks): #5 OmniParseBench and #7 ExtractBench Pro. Here, the only systems ahead of the publisher are heavyweight general-purpose models or coding agents (Gemini 4 or Codex GPT-6 Astra). Alongside the scores, the tables highlight cost, page refusals, or retry caveats that position the publisher’s product as the sensible option for real-world production.
- Exceptions where a rival parser beats the publisher or no single winner is crowned (2 benchmarks): #3 Real5-OmniDocBench and #25 PDF Parser Benchmark 2026.
Scope and methodology
What is included
A benchmark is included in this list if both of the following hold:
- The publisher is commercial. It sells the tested product through a paid API, a hosted platform, or a commercial licence.
- The publisher’s own product is on the leaderboard.
What I left out or merged:
- Third-party benchmarks reposted by vendors. Academic or independent benchmarks like OmniDocBench and olmOCR-Bench frequently appear inside vendor launch posts. I only count the evaluations the vendor created or commissioned itself. For example, in Mistral’s OCR 4 release post, I count Mistral’s internal Crawl Multilingual evaluation and human-preference study, not the olmOCR-Bench half of its chart.
- Older versions of the same benchmark. ParseBench 1 is shown alongside ParseBench 2 for context, but it is not counted as a separate entry.
- Composites built from public datasets. When a vendor builds a composite leaderboard out of existing datasets (such as DeepRead’s composite, Nanonets’ IDP Leaderboard, or Nace’s Parse Index), it counts once as that vendor’s composite benchmark.
How I found them
There was no special process. I tried to find as many benchmarks as I could that fit the criteria above: a commercial vendor published a leaderboard, and its own product is on it. Every benchmark I found that fits is included, wherever its publisher ranks.
This list is almost certainly incomplete. If you know of a benchmark that fits the criteria and is missing here, please email me (see Amendments and additions) and I will add it.
Headline and default numbers only
Many benchmarks report dozens of columns: sub-scores for tables or reading order, per-domain slices, latency percentiles, and cost per page. To keep comparisons consistent and reflect what a reader sees first, I only look at the headline number on each benchmark, or the default metric used to sort and rank parsers when you open the page.
Where a benchmark presents two or three headline charts side by side without combining them into a single number (for example, accuracy and completion rate on LongExtractionBench, or TEDS, TEDS-S, and NID on DP-Bench), I note all of those headline metrics. Where a benchmark publishes separate category tables and intentionally avoids any overall ranking (such as Parsebridge’s benchmark), I record it as having no overall rank.
What would count as an exception
The pattern I am checking is that a vendor’s benchmark shows the vendor’s own product in the best possible light. For that claim to mean anything, it has to be possible for a benchmark to break it. Both conditions below are judged on the metric the benchmark uses to rank parsers when you open the page, the same headline or default metric I use everywhere else in this post. A benchmark is an exception if either of these is true on that metric:
- A competing parser or extraction product ranks above the publisher’s product.
- At least one system ranked above the publisher costs the same as the publisher’s product or less. If ten systems rank above the publisher, one cheaper or equally priced system among them is enough.
So a publisher that is #1 fits the pattern. A publisher that is #2 or #3 also fits, but only if nothing above it is a competing product and everything above it costs more.
Why does a more expensive model above the publisher not count? In practice these are general-purpose models like Gemini or GPT. Google and OpenAI are not competing for document parsing customers through these leaderboards, and nobody from those teams will show up to defend Gemini’s score on a parsing benchmark. Placing a more expensive general model just above your own parser costs the publisher very little. It even helps the publisher’s story: only a model that costs several times more beats them, and only by a small margin. A competing parser ranked above you is different. That competitor will notice, share the result, and use it to win customers.
Because of selection bias, I would expect exceptions to be very rare, and they are. Only 2 of the 33 benchmarks are exceptions: Real5-OmniDocBench, where two competing open OCR models rank above Baidu’s PaddleOCR-VL, and Parsebridge’s PDF Parser Benchmark. Parsebridge has no single ranking metric, and its page says its numbers “do not rank the parsers for every document”, so I checked both tables whose rows are ordered by a metric: speed and table accuracy. Competing parsers rank above Docling on both. The rule is close to a foregone conclusion, but these two benchmarks show that it can fail.
Being an exception is not the same as being a better benchmark, though. A vendor that ranks someone else first is good evidence that it did not publish only the results it wins, but that says nothing about how rigorous or useful the benchmark is. A five-PDF test where the publisher loses is not automatically more trustworthy than a benchmark with thousands of pages, a public dataset, and open scoring code where the publisher wins.
Parsing vs. extraction
The 33 benchmarks cover two closely related tasks, and the scorecard labels each entry accordingly:
- Parsing (19 benchmarks): turning a document into text, Markdown, or structural elements (reading order, tables, charts, layout regions, OCR). Scores compare the parser’s output against a ground-truth reference.
- Extraction (10 benchmarks): pulling specific fields or records out of a document into a target schema (usually JSON). Scores check whether the extracted values match the ground truth.
- Both (4 benchmarks): benchmarks that evaluate parsers through a downstream extraction step (OmniAI, Tensorlake, Docsumo), or composite leaderboards that blend parsing and extraction datasets (Nanonets’ IDP Leaderboard).
Every screenshot preserves the original published row order and scores. For longer leaderboards, the screenshot shows only the top rows.
The scorecard
Each benchmark name in the table below links to its Wayback Machine snapshot from 15 and 16 November 2026. The Live page column links to the current URL, where rankings may have changed since capture.
| # | Benchmark | Publisher's rank | Ranked above the publisher | Live page |
|---|---|---|---|---|
| 1 | ParseBench 2 | #1 | No one | Live |
| 2 | ExtractBench | #1 | No one | Live |
| 3 | Real5-OmniDocBench | #3 | MiniCPM-OCR-1.0, OvisOCR3 (small open OCR models) | Live |
| 4 | PulseBench-Tab | #1 | No one | Live |
| 5 | OmniParseBench | #3#1 on live benchmark page | Gemini 4 Argon, Gemini 4 Carbon | Live |
| 6 | RealDoc-Bench | #1 | No one | Live |
| 7 | ExtractBench Pro | #2 | Codex GPT-6 Astra (xhigh) | Live |
| 8 | WholeDoc Bench | #1 | No one | Live |
| 9 | OmniExtractBench | #1 | No one | Live |
| 10 | DocuBench | #1 | No one | Live |
| 11 | SCORE-Bench | #1Adjusted CCT | No one | Live |
| 12 | OmniAI OCR benchmark | #1 | No one | Live |
| 13 | DocParsingBench | #1 | No one | Live |
| 14 | IDP Leaderboard | #1 | No one | Live |
| 15 | BizDocBench | #1 | No one | Live |
| 16 | LongArray-Extract | #1 | No one | Live |
| 17 | OpenDataLoader Bench | #1 | No one | Live |
| 18 | DP-Bench | #1all 3 accuracy columns | No one | Live |
| 19 | RD-TableBench | #1 | No one | Live |
| 20 | DeepRead extraction composite | #1 | No one | Live |
| 21 | LongExtractionBench | #1both headline metrics | No one | Live |
| 22 | ChartParse-Bench | #1both headline tests | No one | Live |
| 23 | OCRFlux-bench | #1 | No one | Live |
| 24 | Parse Index | #1 | No one | Live |
| 25 | PDF Parser Benchmark 2026 | Not #1 on any test | MinerU, Marker, MarkItDown, others | Live |
| 26 | Handwriting OCR Benchmark 2026 | #1 | No one | Live |
| 27 | ParseArena / PAIR | #1 | No one | Live |
| 28 | Five-hard-document Open Benchmark | #1sister product #2 | No one | Live |
| 29 | 120-document OCR benchmark | #1both headline metrics | No one | Live |
| 30 | NuMind internal evaluations | #1 | No one | Live |
| 31 | Legal document parsing benchmark | #1both headline metrics | No one | Live |
| 32 | Mistral OCR internal evaluations | #1both owned tests | No one | Live |
| 33 | Enterprise Document Benchmark | #1 | No one | Live |
Summary by task type
| Outcome | Parsing | Extraction | Both | Total |
|---|---|---|---|---|
| Publisher is #1 outright | 16 | 9 | 4 | 29 |
| Frontier generalist is #1 on raw score; publisher is framed as the practical choice | 1 | 1 | 0 | 2 |
| Exceptions | 2 | 0 | 0 | 2 |
| Publisher is #1 overall or framed as the practical choice | 17 of 19 | 10 of 10 | 4 of 4 | 31 of 33 |
All 33 benchmarks
1. ParseBench 2 (LlamaIndex) · parsing
ParseBench 2 evaluates 55 entries (49 distinct parsers across 27 vendors) over 2,645 documents (4,317 page-level test cases). On the default Overall ranking, LlamaParse holds three of the top five rows. LlamaParse (Agentic Plus) takes #1 at 90.1, ahead of Claude Opus 5.5 (85.8) and Reducto r-1 (85.2).

Source: parsebench.ai · score CSV · Wayback
The earlier version, ParseBench 1, tested a different set of models with different scores, and also had LlamaParse (Agentic Plus) in first place:

Source: parsebench.ai/1 · Wayback
2. ExtractBench (LlamaIndex) · extraction
ExtractBench covers 370 documents (4,869 pages) and 54 ranked entries (48 distinct systems). On the default Overall value F1 score, two LlamaExtract configurations tie for #1 at 96.2. Reducto Deep Extract and Pulse (Effort) sit 0.1 points behind at 96.1.

Source: extractbench.ai · score CSV · Wayback
3. Real5-OmniDocBench (Baidu / PaddlePaddle) · parsing · exception
Baidu sells PaddleOCR-VL document parsing commercially through Baidu Cloud. On its own Real5-OmniDocBench (1,355 pages photographed and scanned across five capture conditions, 35 models tested), PaddleOCR-VL-1.6 ranks #3 on the default Overall column at 93.19, behind OpenBMB’s MiniCPM-OCR-1.0 (93.75) and Alibaba’s OvisOCR3 (93.71). Neither model ahead of Baidu is an expensive frontier generalist; both are small open OCR models (1.3B and 0.9B parameters) from competing labs, making this one of the two exceptions in the collection.

Source: PaddlePaddle/Real5-OmniDocBench · Wayback
4. PulseBench-Tab (Pulse) · parsing
PulseBench-Tab evaluates 1,820 tables across 380 documents in 9 languages. On the default Overall T-LAG F1 ranking, Pulse Ultra 2 is #1 at 93.5%, 4.5 points ahead of Claude Fable 5.1 (89.0%). According to the benchmark documentation, accuracy is averaged over samples that were processed successfully, while coverage is reported in a separate column. In the benchmark’s GitHub README table, Pulse’s score dates from April 2026 while several competitor rows were re-run in September 2026.

Source: benchmark.runpulse.com · Wayback
5. OmniParseBench (Datalab) · parsing · beaten only by frontier generalists, framed as the practical choice
OmniParseBench (2,937 pages, 16,288 unit tests) shows two slightly different rankings depending on which official page you open, and both present Datalab as the clear practical winner:
- On the main benchmark webpage (
datalab.to/benchmarks), Datalab accurate is ranked #1 outright at 94.1%, ahead of Gemini 3.8 Flash (92.9%).

- In the newer Chandra 2.7 release post (published 12 November 2026) and the GitHub README, Datalab adds Google’s frontier Gemini 4 Argon (95.8) and Gemini 4 Carbon (95.6) at the top of the table, placing Datalab accurate (Chandra 2.7) at #3 overall (94.4). Even here, the table visually bolds Datalab’s Chandra 2.7 scores while attaching a footnote (
†) to both Gemini 4 rows. In the full README table, those Gemini 4 runs cost $49.91 and $58.66 per 1,000 pages, required a custom reader setup with failed-page reruns, and still logged 5 to 8 page refusals. By contrast, Datalab accurate costs $7.50 per 1,000 pages (and balanced costs $3.00) with zero refusals, making Datalab’s parser the obvious practical choice for production use.

Source: datalab.to/benchmarks · Chandra 2.7 release · Wayback: benchmarks page, release post
6. RealDoc-Bench (Extend) · parsing
RealDoc-Bench evaluates 1,356 question-answering fields across 581 documents alongside a 1,500-page layout detection dataset. On the website, Extend leads both the QA task (95.7%) and the layout task (0.847 adjusted F1), taking three of the top four spots on QA.


The updated table in Extend’s GitHub README expands the roster to 15 rows with newer GPT models and lists Extend Parse 2.0 at 96.0%, still in first place:

Source: extend.ai/resources/realdocbench · GitHub README · Wayback
7. ExtractBench Pro (LlamaIndex) · extraction · beaten only by a pricier frontier agent, framed as the practical choice
ExtractBench Pro tests 17 configurations across 100 complex documents (2,622 pages). On the default ranking metric (percentage of Perfect Documents on the time-limited Challenge=all view, with Value F1 as tiebreak), OpenAI’s Codex GPT-6 Astra (a general-purpose coding agent running at xhigh reasoning) takes #1 at 58.0%, just two points ahead of LlamaExtract Agentic Plus (Max Effort) at 56.0%.
Right next to the overall score, the leaderboard displays a COST column that frames LlamaExtract as the more practical solution: Codex GPT-6 Astra is the only top entry in the highest $$$$$ tier (about $0.95 per page in the published score CSV), whereas LlamaExtract Max Effort sits a tier lower at $$$$ ($0.37 per page, roughly 2.6 times cheaper) and standard LlamaExtract Agentic Plus (#4 at 38.0%) is rated $ ($0.07 per page). A brute-force coding agent can edge into first place when cost is no object, while LlamaExtract stands out as the practical option.

Source: extractbench.ai/pro · score CSV · Wayback
8. WholeDoc Bench (Nutrient) · parsing
WholeDoc Bench tests 19 configurations across 480 documents (633 pages, with a 10-document public sample). On the default Overall score, Nutrient Agentic (86.2) and Nutrient Understand (80.8) take #1 and #2, followed by Claude Fable 5.1 at 67.1. A large share of that 19-point lead comes from the Barcode column, where both Nutrient entries score 100 while most competitors score between 2.4 and 3.6.

Source: nutrient.io/benchmarks/parse · Wayback
9. OmniExtractBench (Datalab) · extraction
Across 620 documents and 10 tested configurations, Datalab’s “accurate” mode ranks #1 at 93.85% accuracy and its “balanced” mode ranks #2 at 93.48%, just 0.01 points ahead of Reducto deep_extract v2 (93.47%).

Source: omni_extract_bench README · Wayback
10. DocuBench (DocuPipe) · extraction
DocuBench tests 10 configurations across 72 documents with open schemas and scoring code on GitHub. On the headline macro-average field accuracy chart, DocuPipe takes #1 and #2 with its “high effort” (97.02%) and “standard” (96.14%) modes, ahead of Claude Sonnet 5 at 91.73%.

Source: docupipe.ai/benchmarks/docubench · summary.json · Wayback
11. SCORE-Bench (Unstructured) · parsing
Unstructured’s benchmarks page compares its pipelines with Reducto, LlamaParse, Docling, Snowflake, Databricks, NVIDIA, and five off-the-shelf frontier models on more than 1,000 enterprise pages, scored with its SCORE metrics. In the headline Overall Outcome table, Unstructured is listed first and ranks #1 on Adjusted CCT, which the page calls “the core measure of text accuracy”, at 0.880, ahead of LlamaParse VLM (0.835) and Reducto Agentic (0.812). It is also #1 on both table cell accuracy columns, #2 on tokens added (behind Claude Opus-4.6 off the shelf), and #4 on element alignment, where Snowflake Layout Mode leads at 0.608. The underlying SCORE-Bench dataset of 224 documents, with per-pipeline tables, is published on Hugging Face.

Source: unstructured.io/benchmarks · SCORE-Bench dataset · Wayback
12. OmniAI OCR benchmark (OmniAI) · parsing + extraction
The oldest leaderboard in the collection (last updated in February 2025), OmniAI’s 1,000-document benchmark ranks 8 models by downstream JSON extraction accuracy. OmniAI is #1 at 0.92, followed by Gemini 2.0 Flash at 0.86.

Source: getomni-ai/ocr-leaderboard · Wayback
13. DocParsingBench (SoMark) · parsing
SoMark, which offers commercial licensing and private deployment for its OXR parser, evaluates 17 models on 1,342 pages (after 58 exclusions from a 1,400-page set). On the Overall headline score, SoMark’s OXR-1.0 ranks #1 at 87.76, 2.6 points ahead of PaddleOCR-VL-1.6 (85.17).

Source: SoMarkAI/OXR benchmark.md · Wayback
14. IDP Leaderboard (Nanonets) · parsing + extraction
While the domain name (idp-leaderboard.org) does not mention a vendor, the site’s official links and canonical metadata point to Nanonets. It averages results across olmOCR-Bench, OmniDocBench, and seven Nanonets IDP datasets. On the default Overall composite score across 26 models, Nanonets OCR-3 ranks #1 at 85.9. The page does not state when scores were last updated, and the frontier models on its roster (GPT-5.4, Gemini-3-Pro) are a generation older than those on more recently refreshed boards.

Source: idp-leaderboard.org · Wayback
15. BizDocBench (DocSlicer) · parsing
Across 77 business documents and 8 tools, DocSlicer ranks #1 on the headline BizDocBench Score at 0.88, with Docling second at 0.70. The headline composite blends extraction quality with processing speed and document coverage.

Source: docslicer.ai · score CSV · Wayback
16. LongArray-Extract (Extend) · extraction
Built around 45 synthetic clinical, financial, and legal PDFs containing long arrays, the original comparative report ranks Extend MAX #1 at 99.2% aggregate accuracy, ahead of Reducto Deep Extract at 97.4%. Extend’s newer Operator-1 release post reports Operator-1 at 99.3% on the same benchmark, showing only Extend’s own rows without a full competitor table.

Source: original report · Operator-1 release · Wayback
17. OpenDataLoader Bench (Hancom-associated) · parsing
Hancom sells document parsing commercially, and the entry at the top of OpenDataLoader Bench is the open-source parser maintained by that project. Across 200 PDFs with hand-annotated Markdown ground truth and 12 tested configurations, opendataloader [hybrid] ranks #1 on the Overall headline score at 0.907, ahead of Nutrient (0.885) and Docling (0.882).

Source: opendataloader.org/docs/benchmark · Wayback
18. DP-Bench (Upstage) · parsing
Across 200 pages and 7 entries, Upstage’s DP-Bench reports three quality metrics (table TEDS, TEDS-S, and reading-order NID) alongside processing speed, without a single combined score. Upstage holds the bolded #1 score in all three quality columns, while Microsoft leads the speed column.

Source: upstage/dp-bench · Wayback
19. RD-TableBench (Reducto) · parsing
Published in November 2024 across 1,000 hand-labelled tables, RD-TableBench ranks Reducto #1 at 90.2% mean table similarity, 7.5 points ahead of Azure Document Intelligence (82.7%).

Source: reducto.ai/blog/sota-table-parsing · Wayback
20. DeepRead extraction composite (DeepRead) · extraction
DeepRead combines three public datasets: DocILE (500 documents), Kleister-Charity (40 documents), and a 20-document subset of micro1’s LongExtractBench. On the default Average column, DeepRead deep-extract ranks #1 at 92.0% and DeepRead extract is #2 at 91.1%. Extend Operator ties DeepRead on Charity and beats it on LongExtract, but DeepRead’s margin on DocILE puts it first on the overall average.

Source: deepread.tech/benchmarks · Wayback
21. LongExtractionBench (commissioned by Reducto, published by micro1) · extraction
Commissioned by Reducto and run by micro1 across 225 long documents, LongExtractionBench reports completion count and field-level accuracy side by side. Reducto Deep Extract leads both headline numbers: it is the only system to complete all 225 documents, and it posts the highest leaf accuracy at 99.3%.

Source: micro1.ai/benchmark/long-extraction · Reducto post · Wayback
22. ChartParse-Bench (Unsiloed) · parsing
Tested on 58 synthetic Matplotlib charts across seven pipelines, Unsiloed reports two headline tables: Mean Absolute Error (where lower is better) and Output Point Count (how many data points each parser extracted). Unsiloed ranks #1 on both, with the lowest error (6.56% vs. 14.72% for second-place Reducto) and the highest extracted point count (105,989).


Source: unsiloed.ai blog · Wayback
23. OCRFlux-bench (ChatDOC) · parsing
ChatDOC sells a commercial PDF parser API and also releases the open-source OCRFlux model. Evaluated on 2,000 single-page documents (1,000 English, 1,000 Chinese) against three other small open models, ChatDOC’s OCRFlux-3B ranks #1 on both headline metrics: 0.967 Edit Distance Similarity (EDS) and 0.861 table TEDS.

Source: chatdoc-com/OCRFlux · Wayback
24. Parse Index (Nace.AI) · parsing
Parse Index is a composite of public OmniDocBench scores, ParseBench’s Content Faithfulness sub-score (rather than ParseBench’s official Overall score), and an internal financial benchmark. Nace Document Intelligence + Drex ranks #1 at 0.834, ahead of Nace High and LlamaParse Agentic (tied at 0.820). The table’s Evaluated column shows that the winning Nace + Drex row was run on 11 November 2026, whereas every competitor row is dated 15 September 2026.

Source: nace.ai/perception · Wayback
25. PDF Parser Benchmark 2026 (Parsebridge) · parsing · exception
Parsebridge sells a commercial hosted Docling API, and includes Docling as one of the six parsers in its five-PDF benchmark. Docling does not finish first on any table.
- On processing speed, Docling is 10th of 12 configurations, taking 229 to 244 seconds on the 60-page report compared with 4.5 seconds for MarkItDown. Only Unstructured hi_res (236.6 seconds, inside Docling’s range) and MinerU advanced (1,093 seconds) are slower. The screenshot shows the top 10 rows.
- On table extraction accuracy, Docling extracts 20 of 22 rows, trailing MinerU (advanced) and Marker, which both extract all 22.
One important caveat: Parsebridge benchmarked local open-source builds rather than its own hosted API, and it does not publish a single composite quality score.


Source: parsebridge.com/pdf-parsing-benchmark · Wayback
26. Handwriting OCR Benchmark 2026 (Parseur) · extraction
Parseur comes in #1 on its handwriting benchmark, extracting 209 of 209 fields (100%) across seven handwritten forms taken from the public ExtractBench dataset. Scoring was done manually, and Extend was tested in its default configuration, which returned only the first record of a multi-form PDF.

Source: parseur.com/blog/best-handwriting-ocr-tools · Wayback
27. ParseArena / PAIR (Reducto) · parsing
Published in November 2026, Reducto’s ParseArena uses blinded pairwise judgments (via its PAIR judge framework) against a Gemini Layout reference across 2,000 pages, with three judgments per page. Reducto’s flagship r-1 model ranks #1 out of 21 configurations with a 63.3% win rate, while its smaller r-1 mini ranks #4 at 55.4%.

Source: reducto.ai/blog/reducto-r-1-mini-parse-model · Wayback
28. Five-hard-document Open Benchmark (Airparser) · extraction
Tested on five difficult documents across five tools, Airparser ranks #1 at 100.0% and its sister product Parsio (built by the same company) ranks #2 at 99.6%. The headline accuracy metric is calculated only across the fields each tool actually returned; for example, Parseur (which ranked #1 on its own handwriting benchmark above) places #3 here at 98.5% while returning 6 or 7 of the 12 target fields. Airparser notes on the page that this is a stress test rather than a universal ranking.

Source: airparser.com/open-benchmark · Wayback
29. 120-document OCR benchmark (Docsumo) · parsing + extraction
Comparing three systems on 120 documents with a shared GPT-4o extraction step, Docsumo leads both headline metrics: 84.8% field extraction accuracy and human preference on 116 of the 120 documents. The post notes that the competitors tested (LandingAI ADE and Mistral OCR) are their 2025 versions; a September 2026 edit to the blog post kept the original April 2025 results.

Source: docsumo.com blog · Wayback
30. NuMind internal evaluations (NuMind) · extraction
On its ~600-document internal structured extraction benchmark, NuMind’s NuExtract 3.4 4B ranks #1 at 0.651. The roster is limited to small open-weight models of 9B parameters or fewer, with no frontier API models or commercial document parsers included.

Source: huggingface.co/numind/NuExtract3 · Wayback
31. Legal document parsing benchmark (LexSelect) · parsing
Evaluated across 24 public-record legal documents (917 pages, 21,660 labelled regions), LexSelect reports two headline metrics: “Found + Named (Standard Layout)” and “Found + Named + Citable (Citable Standard)”. LexSelect ranks #1 on both (0.735 and 0.563), ahead of Reducto in second place (0.569 and 0.311).

Source: lexselect.io · Wayback
32. Mistral OCR internal evaluations (Mistral) · parsing
In Mistral’s OCR 4 release post, the performance chart has two halves: the left side reposts third-party olmOCR-Bench numbers (not counted here), while the right side shows Mistral’s internal Crawl Multilingual evaluation across 36 languages, where Mistral OCR 4 ranks #1 at 98%. Notably, the two halves of the chart use different competitor rosters: Gemini 3.1 Pro and GPT 5.5 appear on the olmOCR-Bench side, but not on the Crawl Multilingual side.

Mistral also publishes a human-preference evaluation where Mistral OCR 4 wins every pairwise matchup reported, from 65.7% against Databricks to 82.3% against AWS Textract:

Source: mistral.ai/news/ocr-4 · Wayback
33. Enterprise Document Benchmark (Tensorlake) · parsing + extraction
Comparing three providers on an internal 100-page enterprise corpus (with GPT-4o held fixed downstream for JSON extraction), Tensorlake ranks #1 on both headline metrics: 91.7% JSON F1 and 86.79% table TEDS, ahead of AWS Textract and Azure Document Intelligence.

Source: tensorlake.ai/blog/benchmarks · Wayback
How the same vendors rank on each other’s benchmarks
Another way to read the collection is to follow the most frequently tested vendors across each other’s leaderboards. Six vendors appear repeatedly across these benchmarks. The table below records the row position of each vendor’s best-placed entry on its own benchmarks versus on rival vendors’ benchmarks (using the screenshots, with full-leaderboard positions below what a screenshot shows noted in parentheses where available).
| Vendor | On its own benchmark(s) | On rival vendors’ benchmarks (best row position) |
|---|---|---|
| LlamaIndex | ParseBench 2: #1 · ExtractBench: #1 · ExtractBench Pro: #2 | RealDoc-Bench QA: #3 · ParseArena: #3 · Parse Index: #3 · OmniParseBench (site): #5 · OmniExtractBench: #6 · SCORE-Bench: #6 · PulseBench-Tab: #7 · RD-TableBench: #6 · ChartParse-Bench: #7 (last) · LongArray-Extract: #8 · LexSelect: #8 (last) · (WholeDoc Bench full table: #12) |
| Reducto | RD-TableBench: #1 · ParseArena: #1 · LongExtractionBench: #1 | LongArray-Extract: #2 · LexSelect: #2 · ChartParse-Bench: #2 · ExtractBench: #3 · OmniExtractBench: #3 · ParseBench 2: #4 · OmniParseBench (site): #4 · DeepRead: #4 · DocuBench: #4 · RealDoc-Bench QA (updated table): #4 · SCORE-Bench: #5 · PulseBench-Tab: #10 · (WholeDoc Bench full table: #11 · ExtractBench Pro full table: #14 · Parse Index full table: #14, last) |
| Extend | RealDoc-Bench: #1 · LongArray-Extract: #1 | DeepRead: #3 · LexSelect: #4 · ChartParse-Bench: #4 · Parseur Handwriting: #4 · ExtractBench Pro: #5 · OmniExtractBench: #5 · DocuBench: #6 · ParseBench 2: #8 · WholeDoc Bench: #8 · PulseBench-Tab: #8 · ParseArena: #8 · OmniParseBench (site): #9 · (ExtractBench full table: #11 · Parse Index full table: #11) |
| Datalab | OmniParseBench (site): #1 (#3 in release post) · OmniExtractBench: #1 | Mistral Crawl Multilingual: #2 · ParseArena: #2 · OpenDataLoader Bench (Marker): #4 · BizDocBench (Marker): #8 (last) · DocParsingBench: #10 · Parse Index: #10 · (ParseBench 2 full table: #11 · PulseBench-Tab full table: #13 · ExtractBench Pro full table: #15 · ExtractBench full table: #31 · Real5-OmniDocBench full table: #35, last) |
| Pulse | PulseBench-Tab: #1 | ExtractBench: #4 · LongArray-Extract: #5 · DocuBench: #9 · ParseBench 2: #9 · (ExtractBench Pro full table: #11) |
| Mistral | Crawl Multilingual: #1 · Human preference: #1 | Docsumo: #3 (of 3) · OmniParseBench (site): #6 · PulseBench-Tab: #6 · ParseArena: #6 · Parse Index: #7 · NuExtract 3 evals (Ministral 3B): #7 (last) · OmniExtractBench: #9 · (RealDoc-Bench QA updated table: #12 · IDP Leaderboard full table: #15 · ParseBench 2 full table: #33) |
Across every leaderboard captured here, each of these six vendors ranks #1 on at least one of its own benchmarks, and none of them ranks #1 on a benchmark published by one of the others.
A few direct swaps illustrate how much rankings shift from board to board:
- Pulse vs. LlamaIndex: Pulse Ultra 2 is #1 on PulseBench-Tab (93.5%) and #9 on LlamaIndex’s ParseBench 2 (82.7). Conversely, LlamaParse (Agentic Plus) is #1 on ParseBench 2 (90.1) and #9 on PulseBench-Tab (82.1%).
- Extend vs. LlamaIndex and Pulse: Extend Parse 2.0 is #1 on RealDoc-Bench (96.0%) and #8 on both ParseBench 2 and PulseBench-Tab.
- Datalab vs. LlamaIndex: LlamaExtract is #1 on ExtractBench (96.2) and #6 on Datalab’s OmniExtractBench (87.94), while Datalab is #1 on OmniExtractBench (93.85) and row #31 on ExtractBench (85.70).
- Parseur vs. Airparser: Parseur is #3 on Airparser’s Five-hard-document benchmark and #1 on its own Handwriting OCR benchmark.
Of course, these are not strict apples-to-apples comparisons. Every benchmark tests different documents, schemas, and scoring rules. Even when the same vendor appears on two boards, the tested entry can differ in important ways:
- Mode and effort settings: Many parsers offer multiple tiers (such as fast, balanced, accurate, agentic, or max effort). A vendor will naturally showcase its highest-accuracy tier on its own benchmark, whereas a competitor might test its default or standard tier.
- Model versions and run dates: Benchmarks are run at different times. A leaderboard published or last refreshed months ago may test an older release of a competitor’s API alongside a newly released version of the publisher’s own parser.
Wrapping up
Out of 33 vendor-published benchmarks in this collection, 29 rank the publisher #1 outright on the headline or default metric, and 2 more place the publisher right behind a heavyweight frontier model while framing the vendor’s own product as the practical, cost-effective choice for production. Only 2 do not.
Again, this is completely natural. Benchmarks take real effort to build, and companies publish the evaluations that showcase what their products do best. Many of these benchmarks are genuinely valuable contributions to the ecosystem, offering open datasets, evaluation harnesses, and challenging test cases that anyone can inspect and reuse.
When reading any vendor benchmark, a few quick checks go a long way:
- Check who published or commissioned the benchmark.
- Look beyond the default headline sort to see if sub-metrics, coverage columns, or cost bands tell a more nuanced story.
- Check the roster, modes, and run dates to see whether competitors were tested in their latest versions and strongest configurations.
- Cross-check the same parsers on independent boards (and, above all, on a small sample of your own documents).
Amendments and additions
Leaderboards evolve quickly, new benchmarks launch regularly, and vendors update their scores. If you would like to amend an existing benchmark entry here, point out an update, or add a new benchmark to this collection, please reach out at contact@hashiromer.com.