San Francisco Demands Apple and Google Delete AI ‘Nudify’ Apps From App Stores
The City Attorney’s Office sent the tech giants cease-and-desist letters this week telling them to stop profiting from 13 “face-swap” apps that are overwhelmingly used to target women and girls.
Apple and Google have been ordered to take down apps that can “nudify” or “undress” people and told that they must stop profiting from the harmful technology, according to cease-and-desist letters sent to the companies seen by WIRED.
On Thursday, San Francisco city attorney David Chiu sent legal notices to Apple and Google demanding that they remove from their app stores 13 face-swapping apps, which allow users to create AI-generated nonconsensual nude images. The letters say the Silicon Valley giants should stop “aiding and abetting” the sale of explicit deepfake images and “sever” business relationships with the app developers.
“Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,” Chiu tells WIRED. The city attorney, whose office previously took legal action against 16 popular deepfake websites, says Apple and Google have likely “made millions of dollars in fees” from apps that offer nudification, and they should improve their moderation processes to stop them appearing in their stores in the first place.
“These companies have responsibility to ensure that apps on their platforms do not facilitate sexual abuse,” Chiu says. The city’s legal letters say California’s laws prohibit supporting services that create deepfake pornography. The apps use in-app payments, which the tech companies take a cut of, the letters says. “The fact that some of the world’s largest and most established technology companies are facilitating this has to stop.”
Researchers have repeatedly found and reported apps in Apple’s App Store and Google’s Play Store that allow people to generate sexual images using AI—including some apps being rated as suitable for use by children . While new laws and bans aim to tackle the scourge of explicit deepfakes online, technology and social media companies consistently direct millions of people toward the harmful tech.
Both Apple and Google have developer policies that prohibit pornography, abuse, and harassment on their platforms. They have previously removed dozens of nudify and deepfake apps, after reports by researchers and journalists.
