AI-driven disinformation running rampant on social media before US midterms, experts warn

Social media feeds are awash with an unprecedented flood of AI-driven disinformation ahead of November’s US midterm elections, according to experts, who warn many posts are likely “slipping through the cracks”.

As tens of millions of Americans prepare to cast their vote, social media companies have scaled back their efforts to protect voters from lies meant to manipulate elections.

While the backlash against Facebook after the Cambridge Analytica scandal prompted social media giants to bolster their trust and safety operations, a counter-backlash from the right over fears of speech restrictions – which intensified after Elon Musk’s takeover of X, formerly Twitter, and the election of Donald Trump in 2024 – has led to significant layoffs from relevant teams.

Artificial intelligence has made it easier than ever to launch a disinformation campaign. And platforms are increasingly relying on AI, rather than human intervention, to respond.

“You can run an information operation now with quality that was impossible in 2016 for super, super cheap,” said Frances Haugen, a former Facebook data engineer turned whistleblower and advocate for transparency and accountability in Big Tech. Operatives playing dirty tricks can get around content moderation rules “because you can just keep tweaking the post a little bit each time you create them, and some of them are going to get through”, she said.

Campaigns from Pennsylvania to Texas have been scrambling to respond to lies that have surfaced, and spread rapidly, online. Unlike previous elections, they find themselves in a quandary: who, exactly, do they turn to when fabrications and falsehoods propagate?

Meta Platforms, the Facebook and Instagram owner, and X did not respond to requests for comment. YouTube, owned by Google, said all its users are subject to the same guidelines.

James Talarico, the high-flying Democrat running for US Senate in Texas, took time off from campaigning earlier this month to recover from the flu. But many social media feeds hosted another – false – reason for his absence: that he was supposedly being treated at a mental health facility in Ohio.

The claim was a complete fabrication, pushed by a rightwing “provocateur” who published a deceptively edited phone call with a hospital staffer. Talarico’s campaign released security-camera video showing him at home in Texasand the candidate publicly clarified he had never even been to Ohio.

But the lie circulated – and was stoked – for several days.

James Talarico, a Senate candidate, speaks to attendees at the Voto Latino ‘Ready, Go, Vote,’ rally on Sunday in Weslaco, Texas, before the midterm elections. Photograph: Michael Gonzalez/AP

The Digital Democracy Institute of the Americas (DDIA) tracked its propagation, and how the misinformation mutated as it was picked up by conservative Spanish-language media. What began as a viral lie about Talarico being hospitalized in Ohio became a false claim in Spanish that he had attempted suicide.

The case “illustrates a recurring challenge in countering election disinformation”, DDIA’s researchers wrote. Lies evolve and shift as they spread, “making the original correction insufficient to address every version of the story”.

Social media platforms enforce their terms of service “more congruently” in English than other languages including Spanish, according to Roberta Braga, the founder and executive director of DDIA.

“There’s a lot less guardrails in place from the companies to deal with Spanish language, especially now that they’ve walked back a lot of their own investments in content moderation, and they’ve moved toward doing community-centered content moderation, like Community Notes-style things,” said Braga. “Now that they’re using AI to do a lot of the content moderation, even more is likely slipping through the cracks.”

Ken Paxton, Talarico’s Republican opponent in Texas, also launched a campaign advert earlier this year using AI-generated footage of Talarico – a Presbyterian seminarian – supposedly rewriting the Bible.

“I think Texans are fed up with these kind of lies in our politics,” Talarico told NBC News in September. “These AI deepfake ads don’t add anything to our civic discourse. They don’t do anything to elucidate the choice that’s in front of voters.”

Online hit squads behind election disinformation once operated on the fringes. Today, their work might be amplified by official government sources.

When viral posts on X and rightwing blogs claimed that an undocumented immigrant arrested in central Pennsylvania was an active field worker and canvasser for Democrat Janelle Stelson’s congressional campaign, staffers set the truth sprinting, trying to to catch up with a lie.

Democratic candidate Janelle Stelson speaks at a campaign event on 23 September in York Haven, Pennsylvania. Photograph: Marc Levy/AP

As they did, however, an official statement from the Department of Homeland Security amplified the claim. “It’s unbelievable that Janelle Stelson would employ an illegal alien to work on a congressional campaign, let alone a pervert who exposed himself in public,” declared Charles Wall, the deputy director of Immigration and Customs Enforcement (ICE).

But the campaign released records showing that the immigrant, Chimaobim Adedayo Igbojekwe, had no paid position on the campaign, despite an initial police incident report stating that Igbojekwe claimed to be a campaign worker. They hold the claim provably false.

The campaign of Stelson’s opponent, Republican congressman Scott Perry, continues to run ads with the claim.

The Stelson campaign sent cease-and-desist letters to the media. A Fox News reporter took down a post on X about the indecent exposure arrest. Activists at Citizens for Responsibility and Ethics in Washington launched a complaint accusing DHS of violating the Hatch Act, which limits the political activities of government employees.

“Trump has made clear that he intends to interfere in the elections,” said Nora Benavidez, a civil rights and free speech attorney with Free Press, a non-profit advocating for media independence. Trump “has leveraged agencies across the federal government, whether it’s the US Postal Service or DHS – you name it”, she added.

Stelson’s campaign has yet to complain directly to the social media platforms: Meta’s Facebook, Elon Musk’s X, Google’s YouTube or others. But a spokesperson said the campaign “continues to consider every legal avenue to fight the false claims in Perry’s ads”.

The dominant social media platforms do have clear rules – based on specific, limited definitions – on how to handle misinformation driven by AI during election campaigns.

Meta requires advertisers to disclose the use of synthetic images and AI-generated content with election ads, and enforces automated labeling on AI-generated images identified through Coalition for Content Provenance and Authenticity (C2PA) metadata. It also bans posts and ads that misrepresent voting dates, locations or legal requirements, as well as threats of violence.

But the company does little active moderation of political commentary, beyond the exclusion of unsolicited political content from its recommended posts. An analysis by Free Press noted that Meta had narrowed its historical restrictions in 2023 to allow debate around the legitimacy of past election cycles.

In 2023, YouTube also stopped removing content that falsely claimed the 2020 election or other past US presidential elections were marred by “widespread fraud, errors or glitches”, stating that the removals had curbed misinformation, but also had a chilling effect on political speech.

“All users” of YouTube, including political candidates and campaigns, “are subject to the same community guidelines … and have access to the same public reporting tools,” according to the platform.

It defines election misinformation narrowly, however: content that misleads voters about the time, place, means or eligibility requirements for voting or false claims that could materially discourage voting are banned. Claims that a candidate on the ballot is somehow ineligible, or calls to hack a polling site’s computers or incite violence at election locations, are also forbidden. “We also review government requests closely to determine if content should be removed because it violates a law or our policies,” YouTube’s spokesperson wrote in response to inquiries.

Google has a likeness detection system and allows civic leaders to request removal of a video when an AI-generated image falsely depicts their words or deeds. But a direct lie that falls outside these lines could propagate without intervention.

Instead of relying on takedowns, Google prefers to amplify authoritative news outlets and election boards in search queries and underneath high-stakes civic videos via contextual “information panels”.

X, formerly known as Twitter, abandoned internal safety reviews after its acquisition by Musk, replacing it with the crowdsourced “community notes” program. Its civic integrity policy bars false information about how to vote and suppression schemes. The platform also supposedly restricts synthetic media designed to mislead, though experts argue that the policy amounts to little without enforcement.

“When you stop being able to focus on content because the AI will just slightly tweak it until it gets through, you have to focus on what are the systems of distribution,” Haugen said. “What are the biases? What are the ways we have feedback systems, cycles, that allow the community to provide feedback, that allows it to be damped down?”