Back
PNAS study from December 2024
webAuthors
Farshad Soleimani Sardoo·Nir krakauer
Credibility Rating
5/5
Gold(5)Gold standard. Rigorous peer review, high editorial standards, and strong institutional reputation.
Rating inherited from publication venue: PNAS
Empirical RCT evidence directly relevant to AI deployment risks in high-stakes information environments; important for policymakers and researchers evaluating real-world impacts of AI systems on public epistemics and misinformation.
Paper Details
Citations
0
Year
2024
Metadata
Importance: 72/100journal articleprimary source
Summary
A preregistered randomized controlled experiment published in PNAS (2024) finds that despite 90% accuracy in identifying false headlines, LLM-generated fact checks do not improve users' ability to discern headline accuracy or promote accurate news sharing—unlike human fact checks. Critically, LLM fact checks caused harm in specific failure modes: reducing belief in true headlines mislabeled as false and increasing belief in false headlines when the AI expressed uncertainty.
Key Points
- •LLMs achieved 90% accuracy identifying false headlines, but this did not translate into improved user discernment or accurate news-sharing behavior.
- •Human-generated fact checks significantly improved discernment; LLM fact checks did not, revealing a human-AI trust gap.
- •AI uncertainty expressions backfired: users were more likely to believe false headlines when the LLM expressed doubt about their veracity.
- •When users chose to view LLM fact checks, they became more likely to share both true and false news, and more likely to believe false headlines.
- •Findings highlight systemic risks of deploying AI fact-checking at scale without policies to mitigate unintended harmful effects on information integrity.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| AI-Era Epistemic Infrastructure | Approach | 59.0 |
Cached Content Preview
HTTP 200Fetched Mar 15, 202698 KB
Contents
## Significance
This study explores how large language models (LLMs) used for fact-checking affect the perception and dissemination of political news headlines. Despite the growing adoption of AI and tests of its ability to counter online misinformation, little is known about how people respond to LLM-driven fact-checking. This experiment reveals that even LLMs that accurately identify false headlines do not necessarily enhance users’ abilities to discern headline accuracy or promote accurate news sharing. LLM fact checks can actually reduce belief in true news wrongly labeled as false and increase belief in dubious headlines when the AI is unsure about an article’s veracity. These findings underscore the need for research on AI fact-checking’s unintended consequences, informing policies to enhance information integrity in the digital age.
## Abstract
Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent AI language models have shown impressive ability in fact-checking tasks, but how humans interact with fact-checking information provided by these models is unclear. Here, we investigate the impact of fact-checking information generated by a popular large language model (LLM) on belief in, and sharing intent of, political news headlines in a preregistered randomized control experiment. Although the LLM accurately identifies most false headlines (90%), we find that this information does not significantly improve participants’ ability to discern headline accuracy or share accurate news. In contrast, viewing human-generated fact checks enhances discernment in both cases. Subsequent analysis reveals that the AI fact-checker is harmful in specific cases: It decreases beliefs in true headlines that it mislabels as false and increases beliefs in false headlines that it is unsure about. On the positive side, AI fact-checking information increases the sharing intent for correctly labeled true headlines. When participants are given the option to view LLM fact checks and choose to do so, they are significantly more likely to share both true and false news but only more likely to believe false headlines. Our findings highlight an important source of potential harm stemming from AI applications and underscore the critical need for policies to prevent or mitigate such unintended consequences.
### Sign up for PNAS alerts.
Get alerts for new articles, or get an alert when an article is cited.
[Manage alerts](https://www.pnas.org/action/showPreferences?menuTab=Alerts)
Digital misinformation has rapidly become a critical issue of modern society ( [1](https://www.pnas.org/doi/10.1073/pnas.2322823121#core-collateral-r1), [2](https://www.pnas.org/doi/10.1073/pnas.2322823121#core-collateral-r2)). Recent work suggests that misinformation can erode support for climate change ( [3](https://www.pnas.org/doi/10.1073/pnas.2322823121#core-collatera
... (truncated, 98 KB total)Resource ID:
246011d53fc7e0d5 | Stable ID: YTE3ZTcxMz