Independent benchmark of extraction models
Extraction models read text and images and fill typed fields in one pass, with a confidence for each answer. We test whether the values are right, whether the model says nothing when the answer isn't there, and what it costs.
First results comingRead the benchmark design
Extracting Index
0 to 100, higher is better
Planned lineup. Bars appear when the first run is published.
Speed
median seconds per document
Planned lineup. Bars appear when the first run is published.
Cost
list price, US dollars
Hosted providers only. Planned lineup. Bars appear when the first run is published.
How one document is tested the same for every model and provider
Extracting Index
75% accuracy on fields that are in the document plus 25% null honesty on fields that aren't, on the private test set, text input.
Extracting Index vs cost
Extracting Index against cost per 1,000 documents. The dotted line joins the models nothing else beats on both. The shaded corner is better and cheaper than the median.
Measurements each has its own page, method and precision
- AccuracyIs the value right?Planned. First results coming →
- Honesty about missing fieldsDoes it say nothing when the answer isn't there?Planned. First results coming →
- Confidence you can trustWhen it says 90% sure, is it right 90% of the time?Planned. First results coming →
- Messy-input robustnessHow much worse is it on bad photos and scans?Planned. First results coming →
- Long and large documentsDoes it hold up on long documents and long field lists?Planned. First results coming →
- Formats and languagesDoes it read 03/04 and 1.234,56 the right way, and non-English documents?Planned. First results coming →
- ConsistencyDoes it give the same answer every time it should?Planned. First results coming →
- MathCan it compute values that aren't printed?Planned. First results coming →
- PointingDoes the position it returns point at the value?Planned. First results coming →
- SafetyCan text inside a document hijack it?Planned. First results coming →
- Speed and costHow fast is it, and what does it cost, on each provider?Planned. First results coming →
- Extracting IndexHow the one headline number is built, and why it is small.Read the formula →
Lineup every model on every provider that serves it
| Model | Input | Weights | Provider | List price | Status |
|---|---|---|---|---|---|
| ClairicCloudflare | Text and images | Closed | Cloudflare Workers AIserverless | $0.10 per million input tokens, no output charge | Not run yet |
| NuExtract3NuMind | Text and images | Open, Apache 2.0 | NuMind APIserverless | $1 per million input tokens, $5 per million output tokens | Not tested yet: needs an account |
| Hugging Face Inference Endpointsdedicated | per GPU hour | Not tested yet: needs an account | |||
| Our Mac Mini (run it yourself)self-run | no per-document price | Optional, after hosted results | |||
| GLiNER2Fastino | Text only | Open, Apache 2.0 | Fastino APIserverless | $0.03 per million input tokens, no output charge | Not tested yet: needs an account |
| Hugging Face Inference Endpointsdedicated | per CPU hour | Not tested yet: needs an account | |||
| Our Mac Mini (run it yourself)self-run | no per-document price | Optional, after hosted results |
As of 20 Oct 2026, extractingbench has not published results. The planned lineup is Clairic, NuExtract3 and GLiNER2. Before this benchmark, every published accuracy figure we could find for these extraction models came from the vendors themselves.