35.5% fewer retweets and 33.2% fewer likes

Abstract

Developing interventions that successfully reduce engagement with misinformation on social media is challenging. One intervention that has recently gained great attention is X/Twitter's Community Notes (previously known as "Birdwatch"). Community Notes is a crowdsourced fact-checking approach that allows users to write textual notes to inform others about potentially misleading posts on X/Twitter. Yet, empirical evidence regarding its effectiveness in reducing engagement with misinformation on social media is missing. In this paper, we perform a large-scale empirical study to analyze whether the introduction of the Community Notes feature and its roll-out to users in the U.S. and around the world have reduced engagement with misinformation on X/Twitter in terms of retweet volume and likes. We employ Difference-in-Differences (DiD) models and Regression Discontinuity Design (RDD) to analyze a comprehensive dataset consisting of all fact-checking notes and corresponding source tweets since the launch of Community Notes in early 2021. Although we observe a significant increase in the volume of fact-checks carried out via Community Notes, particularly for tweets from verified users with many followers, we find no evidence that the introduction of Community Notes significantly reduced engagement with misleading tweets on X/Twitter. Rather, our findings suggest that Community Notes might be too slow to effectively reduce engagement with misinformation in the early (and most viral) stage of diffusion. Our work emphasizes the importance of evaluating fact-checking interventions in the field and offers important implications to enhance crowdsourced fact-checking strategies on social media.

Summary

This large-scale empirical study uses Difference-in-Differences and Regression Discontinuity Design to evaluate whether X/Twitter's Community Notes reduces engagement with misinformation. Despite increased fact-checking volume, the study finds no significant reduction in retweets or likes on misleading tweets, attributing this failure to the slow response time of crowdsourced fact-checking relative to misinformation's early viral phase.

Key Points

• •Community Notes showed no statistically significant reduction in retweets or likes on misleading tweets despite increased fact-checking activity since its 2021 launch.
• •The intervention's primary weakness is timing: crowdsourced fact-checks arrive too slowly to interrupt misinformation during its most viral early diffusion window.
• •Fact-checking via Community Notes was disproportionately applied to tweets from verified, high-follower accounts, suggesting triage bias in coverage.
• •The study used comprehensive data covering all Community Notes since launch, with robust quasi-experimental methods (DiD and RDD) to establish causal inference.
• •Findings highlight the broader challenge of designing real-world fact-checking interventions that are both timely and scalable on large social media platforms.