Benchmarks
How accurate is Stamp AI?
A fact-checker's accuracy is itself a factual claim. So we apply our own standard to it: we publish the numbers, the datasets, and the method, including where it does badly.
The headline
When Stamp AI says high confidence, it is right 97.2% of the time.
That is 2,154 high-confidence verdicts across AVeriTeC and LIAR2: 4,402 real-world claims, measured on the ones professional fact-checkers had settled as true or false. High-confidence verdicts were 58% of those; the rest came with a moderate or low band, which is the point of the bands. Across all of the settled claims, whatever the band, Stamp AI called 89.8% right and 7.1% backwards.
| Confidence band | Verdicts | Right |
|---|---|---|
| High | 2,154 | 97.2% |
| Moderate | 1,285 | 88.3% |
| Low | 166 | 68.1% |
Read each band as an instruction. High: act on it. Moderate: read the evidence first. Low: treat it as flagged for a person.
The method
- The production pipeline, end to end. Each claim runs exactly as it does in a check: live-web gather, screening, the ruling, the citation audit and the confidence band. Nothing is tuned to a dataset.
- The context a document would give. Each claim is sent with its date, speaker and where it was said, the way the surrounding text of a real document would carry it.
- Scoring. A verdict counts as right when it lands on the label's side of true and false; each dataset below lists its exact mapping. Band accuracy is measured where both the fact-checkers and Stamp AI came down on a side.
AVeriTeC
AVeriTeC: Real-world claims from 2020 checked by 50 fact-checking organizations (Schlichtkrull et al., NeurIPS 2023): viral posts, politicians, news. We ran the full dev split (500 claims) plus a random 1,623-claim sample of the train split, 2,123 claims in all, on 2026-10-16. 0.1% of evaluations returned no verdict at all; they are reported, not hidden.
On the 1,808 claims the fact-checkers settled as true or false, Stamp AI called 89% right and got 7.5% backwards.
| Fact-checkers' label | Claims | Stamp AI agreed | Counted as agreeing |
|---|---|---|---|
| Supported | 583 | 84.4% | ruled true or mostly true |
| Refuted | 1,225 | 91.3% | ruled false, mostly false or misleading |
| Conflicting / cherry-picked | 132 | 12.9% | ruled mixed |
| Not enough evidence | 181 | 6.1% | ruled unverifiable |
| Confidence band | Verdicts | Right |
|---|---|---|
| High | 1,063 | 96.6% |
| Moderate | 609 | 87.5% |
| Low | 74 | 67.6% |
A published fact-check reached the evidence for 89.3% of the settled claims. With one, Stamp AI was right 90.8% of the time; without one, 74.7%. Median 8.8 s a claim, 13.2 s at the 90th percentile, with ten claims in flight.
Scoring: Supported counts as right when ruled true or mostly true; Refuted when ruled false, mostly false or misleading; Conflicting evidence when ruled mixed; Not enough evidence when ruled unverifiable.
LIAR2
LIAR2: PolitiFact statements from 2007 to 2023, each rated on PolitiFact's six-point Truth-O-Meter (Xu & Kechadi, 2024): politicians, pundits, viral posts. We ran 2,279 claims of the 2,296-claim test split (the last 17 were cut off by our own API spending cap, not by Stamp AI), 2,279 claims in all, on 2026-10-16. 0.04% of evaluations returned no verdict at all; they are reported, not hidden.
On the 1,910 claims the fact-checkers settled as true or false, Stamp AI called 90.6% right and got 6.7% backwards.
| Fact-checkers' label | Claims | Stamp AI agreed | Counted as agreeing |
|---|---|---|---|
| True | 259 | 96.1% | ruled true or mostly true |
| Mostly true | 340 | 90.9% | ruled true or mostly true |
| Half true | 368 | 25.3% | ruled mixed |
| Barely true | 356 | 77.8% | ruled false, mostly false or misleading |
| False | 655 | 93.1% | ruled false, mostly false or misleading |
| Pants on fire | 300 | 95.3% | ruled false, mostly false or misleading |
| Confidence band | Verdicts | Right |
|---|---|---|
| High | 1,091 | 97.7% |
| Moderate | 676 | 89.1% |
| Low | 92 | 68.5% |
A published fact-check reached the evidence for 99% of the settled claims. With one, Stamp AI was right 90.7% of the time; without one, 85%. Median 7.8 s a claim, 12.3 s at the 90th percentile, with ten claims in flight.
Scoring: True and Mostly true count as right when ruled true or mostly true; False, Pants on fire and Barely true (PolitiFact's mostly false) when ruled false, mostly false or misleading; Half true when ruled mixed.
Where it is weak
- It rarely says "unverifiable". When fact-checkers judged a claim cherry-picked or too thin to settle, Stamp AI usually still picked a side, sometimes with high confidence. For thin or contested claims, read the evidence or run the Research Panel.
- Half-true claims get rounded. On claims PolitiFact rated half true, Stamp AI usually came down on one side instead of ruling them mixed.
- Published fact-checks carry weight. One of Stamp AI's two searches looks for an existing fact-check, correction or denial, because a newsroom would too. Every claim in these datasets has been fact-checked, so the results lean on that: on LIAR2 a PolitiFact rating reached the evidence for 99% of claims. For a claim nobody has checked yet, the better guide is AVeriTeC without a published fact-check in the evidence: 74.7% right.
What's next
The Research Panel runs on the same datasets next, along with claims from after the models' training cutoffs and run-to-run stability. Each result lands here with its date and configuration, and we re-run Stamp AI whenever its pipeline changes. The methodology page describes how verdicts and confidence bands are computed.