Yeah, this is not a great endorsement of X’s Community Notes crowd-sourced moderation approach.
A day after announcing a significant update to the back end architecture of Community Notes, which will ensure that approved Notes are displayed faster than ever, The Washington Post has published a new report which suggests that Community Notes are largely failing to address misinformation in the app.
The report is based on the Post’s own research, as well as a new report from The Center for Countering Digital Hate (CCDH), both of which suggest that Community Notes’ requirement of securing agreement from reviewers of opposing political viewpoints is hampering the project.
To clarify, the current process for Community Notes is:
- X user taps the “Request Community Note” option from the three dots menu on a post
- The Community Notes contributor group is alerted to the post for review
- An approved Notes contributor then reviews the post, checking for accuracy and relevant info, and submits a proposed Note where valid
- The proposed Note is then reviewed (by another Notes contributor of varying political perspective), before finally being shown on the post, or not, depending on the process
That last step is causing problems in the flow of Notes, according to the CCDH, with the majority of Notes never actually achieving cross-political agreement. Meaning that most are never displayed in the app.
As per The CCDH:
“We found that 209 out of the 283 misleading posts in our sample [related to the U.S. election] had accurate Community Notes that are not being shown to all X users, equivalent to 74%. We rated notes as “accurate” where they align with independent fact-checks, cite reputable sources and explain why their attached post is misleading.”
So in 74% of cases where a Note was proposed, and the CCDH found it to be an accurate request for amendment, the Note was never displayed to users in the app.
The CCDH also notes that posts with misleading claims about the upcoming U.S. election identified within its data set have been viewed in the app more than 2.9 billion times.
So why aren’t these Notes achieving the required consensus?
This chart probably explains it best:
Of the various false claims being amplified across X, and not being Community Noted, the majority relate to the 2020 election being “stolen”, which many Republican voters maintain as being true, despite various investigations finding no evidence of such. Republican candidate Donald Trump also continues to stand by this claim, so it’s no surprise that Republican Community Notes contributors disagree on this being queried.
Second on the list is claims of illegal voter importation, which X owner Elon Musk has been one of the biggest amplifiers of, while third are questions about the safety of voting systems.
Looking at the list of topics, it seems fairly obvious why these are not getting Community Noted in the app, despite there being factual sources to refute such claims. Because on some topics, political opponents are never going to agree, which also means that X is helping to amplify these false claims in the app.
But then again, despite said evidence, many will maintain that these things are true, and that it’s all part of a broader cover-up. Which, really, is why Community Notes holds such appeal to Musk, because his view is that some things reported as truth by the media are not correct, and the people should be the ones who decide what’s actually accurate.
But that clearly highlights a flaw in the Community Notes system. Despite contribitors refuting these claims, with actual evidence, those from the other side of the political aisle can simply shoot them down, because they don’t agree. And no note is shown as a result.
Furthering the CCDH’s findings, WaPo’s own analysis also found that only 79,000 of the more than 900,000 Community Notes written in 2024 have been shown publicly, less than 9%, while the success rate of a Note being displayed is declining over time.
So despite more contributors signing up to the program, and more Notes being created, fewer, on balance, are actually being displayed to users.
The key issue here is that X has tilted the Community Notes approval process from a logical approach to an ideological one, where the actual facts matter less than contributor consensus. And since, in many cases, that will never be reached, the majority of Notes are not displayed.
But again, Musk would see this as a success, because it’s the people who are deciding what’s accurate, not “the mainstream media”, which he often presents as an evil specter on the information landscape.
The best fact-checkers, in Musks’ view, are the people themselves, but if those people are turning their back on actual evidence, that seems like a flawed approach.
