San Francisco’s city attorney has hit Meta with a cease-and-desist letter telling the company to stop “allowing” paid advertisements that include AI-generated child sexual abuse content and asked it to explain how the ads repeatedly ran on its platforms.
On Tuesday, WIRED reported that Meta has run more than 350 ads in recent months that transformed still images of minors—some of them confirmed as real individuals, including a member of a European royal family—into short videoclips that can depict them performing sexual acts. When clicked, these ads would direct people to download AI image- and video-generation apps, some of which could be used to digitally undress people or create other nonconsensual intimate imagery.
The ads, which were discovered by researchers at the Tech Transparency Project (TTP), included some identical ads that had been uploaded multiple times, and in total they reached more than 29,000 accounts in countries in the European Union. The ads that were shown across Facebook, Instagram, and Threads also targeted accounts in the US, Australia, and India. WIRED initially reported on a batch of 53 of the ads at the start of August, but more than 250 of the total number of ads ran on Meta’s platforms after this initial report.
“Meta has failed to address a known issue of AI-generated child sexually explicit ads, while profiting from them,” San Francisco city attorney David Chiu says in a statement to WIRED. “These ads are deeply disturbing and cause harm to real children. Meta claims to have zero tolerance for child exploitation, yet the scale and persistence of these issues suggest current efforts are woefully inadequate. This is unacceptable, and Meta must do better.”
“No company—particularly one that claims all advertisements are reviewed and approved prior to distribution and then accepts payment for that distribution—should permit its advertising systems to be used in this manner,” Chiu writes in the four-page letter his office sent to Meta’s lawyers.
The letter raises a host of questions about how Meta’s ad policies and safety systems operate and asks that the company’s legal, child-safety, and advertising staff engage in “prompt discussions” with the city office. “These findings are especially troubling because they do not appear to reflect an isolated failure that Meta corrected when first alerted to the problem,” Chiu writes.
Meta says there is “no indication” that the ads were displayed in San Francisco and this means they are “outside the city attorney’s jurisdiction.” While Meta’s ad library, which is a repository of ads, includes some data about ads published in the US, it does not break down where in the US ads may have run.
Meta previously told WIRED that many of the ads identified by the TTP researchers had already been removed before they were reported—all of them have now been removed for violating its policies—and most had “fewer than 200 impressions” from accounts. “We don’t tolerate nudify apps or any kind of child exploitation, whether real or AI-generated,” a Meta spokesperson previously said. The company said the “total ad spend” for the 300-plus ads found by the researchers was “under $5,000.”
The letter, which says the City Attorney’s Office has reviewed many of the ads found by the TTP researchers, demands that Meta stop the ads from running on its platforms and explain how the ads managed to avoid its moderation systems and how it escalates and reports ads to the National Center for Missing and Exploited Children (NCMEC), and asks the company to detail how it addresses the advertisers behind the content and repeat offenders.