Fact-Checking with LLMs
Free while signed in. Answers cite the passages they came from.
Investigates the fact-checking capabilities of frontier LLMs across multiple languages and claim types.
Contextual information helps: LLMs perform significantly better at fact-checking when equipped with retrieved evidence, validating the RAG pattern for claim verification.
GPT-4 > GPT-3: GPT-4 shows meaningful accuracy gains over GPT-3 for fact-checking, but both struggle without supporting context.
Multilingual variance: Accuracy varies substantially by query language and claim veracity, exposing persistent language-equity gaps in fact-checking.
Inconsistent reliability: While LLMs show real fact-checking promise, their accuracy is inconsistent enough that they can't replace human fact-checkers - useful as assistants, not arbiters.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack