Hidden within the base of popular artificial intelligence image generators are thousands of images of child sexual abuse, according to a new report urging companies to take action to address a harmful flaw in the technology they built.
Those same images have made it easier for AI systems to produce realistic and explicit images of fake children and turn social media photos of fully clothed real teenagers into nude photos, much to the horror of schools and law enforcement around the world.
Until recently, anti-abuse researchers thought the only way some unsupervised AI tools produced abusive images of children was by essentially combining what they learned from two separate buckets of online images: adult porn and benign photos of children.
But the Stanford Internet Observatory found more than 3,200 images of suspected child sexual abuse in the massive AI database LAION, an index of online images and captions that has been used to train leading AI image makers like Stable Diffusion. The watchdog group based at Stanford University worked with the Canadian Center for Child Protection and other anti-abuse charities to identify the illegal material and report the original photo links to law enforcement.
The response was immediate. On the eve of Wednesday’s release of the Stanford Internet Observatory report, LAION told The Associated Press that it was temporarily removing its data sets.
LAION, which stands for the nonprofit Large-scale Artificial Intelligence Open Network, said in a statement that it “has a zero-tolerance policy for illegal content and in an abundance of caution we have removed the LAION datasets to ensure they are safe before republish them.”
Although the images make up just a fraction of LAION’s index of some 5.8 billion images, the Stanford group says they likely impact the ability of AI tools to generate malicious results and reduce past abuse of real victims who have been abused multiple times appear to strengthen.
It’s not an easy problem to solve and goes back to the fact that many generative AI projects are being “effectively brought to market” and made widely accessible because the field is so competitive, says David Thiel, Stanford’s chief technologist Internet Observatory, which wrote the report.
“Scouring the entire Internet and creating that dataset to train models is something that really should have been limited to a research operation, and it’s not something that should have been open source without much more rigorous attention,” Thiel said in an interview.
A prominent LAION user who contributed to the development of the dataset is London-based startup Stability AI, creator of the Stable Diffusion text-to-image models. New versions of Stable Diffusion have made it much harder to create malicious content, but an older version introduced last year – which Stability AI says it has not released – is still baked into other applications and tools and remains “the most popular content generation model”. explicit images,” the Stanford report said.
“We cannot reverse that. That model is in the hands of a lot of people on their local machines,” said Lloyd Richardson, director of information technology at the Canadian Center for Child Protection, which runs Canada’s hotline for reporting online sexual exploitation.
Stability AI said Wednesday that it is only hosting filtered versions of Stable Diffusion and that “Since acquiring exclusive development of Stable Diffusion, Stability AI has taken proactive steps to limit the risk of abuse.”
“These filters ensure that unsafe content does not reach the models,” the company said in a prepared statement. “By removing that content before it ever reaches the model, we can help prevent the model from generating unsafe content.”
LAION was the brainchild of a German researcher and teacher, Christoph Schuhmann, who told the AP earlier this year that part of the reason for making such a massive visual database publicly accessible was to ensure that the future of AI development is not controlled by a handful of powerful companies.
“It will be much safer and fairer if we can democratize it so that the entire research community and the entire general public can benefit from it,” he said.
Much of LAION’s data comes from another source, Common Crawl, a repository of data continuously collected from the open Internet, but Common Crawl Executive Director Rich Skrenta said LAION’s “job” was to to scan and filter what was needed before using it. of it.
LAION said this week that it has developed “rigorous filters” to detect and remove illegal content before releasing its datasets, and is still working to improve those filters. The Stanford report acknowledges that LAION’s developers made some attempts to filter out explicit “minor” content, but that they could have done a better job if they had consulted child safety experts earlier.
Many text-to-image generators are derived from the LAION database in some way, although it is not always clear which ones. OpenAI, maker of DALL-E and ChatGPT, said it does not use LAION and has refined its models to deny requests for sexual content involving minors.
Google built its text-to-image Imagen model based on a LAION dataset, but decided not to make it public in 2022 after an audit of the database “revealed a wide range of inappropriate content, including pornographic images , racist comments and harmful social stereotypes.”
Trying to clean the data retroactively is difficult, so the Stanford Internet Observatory is calling for more drastic measures. One is aimed at anyone who has built training sets on LAION-5B – named for the more than five billion image-text pairs it contains – to “remove them or work with intermediaries to clean up the material.” Another is to effectively wipe out an older version of Stable Diffusion from all but the darkest corners of the Internet.
“Legitimate platforms can stop offering versions of it for download,” especially if they are often used to generate offensive images and have no safeguards to block them, Thiel said.
As an example, Thiel cited CivitAI, a platform favored by people who create AI-generated pornography, but which he said lacks safeguards to balance it against capturing images of children. The report also calls on AI company Hugging Face, which distributes the model training data, to implement better methods for reporting and removing links to offensive material.
Hugging Face said it regularly works with regulators and child safety groups to identify and remove offensive material. CivitAI did not return requests for comment submitted on the webpage.
The Stanford report also questions whether photos of children — even the most benign ones — should be entered into AI systems without their families’ consent because of protections in the federal Children’s Online Privacy Protection Act.
Rebecca Portnoff, director of data science at anti-child abuse charity Thorn, said her organization has conducted research showing that the prevalence of AI-generated images among abusers is small but steadily growing.
Developers can limit this damage by ensuring that the datasets they use to develop AI models are free of exploitative material. Portnoff said there are also opportunities to limit harmful uses in the future after models are already in circulation.
Technology companies and child safety groups currently assign videos and images a “hash” – unique digital signatures – to detect and remove child abuse material. According to Portnoff, the same concept can be applied to AI models that are being misused.
“It’s not happening right now,” she said. “But it is something that I believe can and should be done.”