This article is from the source 'bbc' and was first published or seen on . It last changed over 40 days ago and won't be checked again for changes.
You can find the current article at its original source at https://www.bbc.co.uk/news/uk-65932372
The article has changed 8 times. There is an RSS feed of changes available.
| Version 4 | Version 5 |
|---|---|
| Illegal trade in AI child sex abuse images exposed | Illegal trade in AI child sex abuse images exposed |
| (about 3 hours later) | |
| Paedophiles are using artificial intelligence (AI) technology to create and sell life-like child sexual abuse material, the BBC has found. | Paedophiles are using artificial intelligence (AI) technology to create and sell life-like child sexual abuse material, the BBC has found. |
| Some are accessing the images by paying subscriptions to accounts on mainstream content-sharing sites such as Patreon. | Some are accessing the images by paying subscriptions to accounts on mainstream content-sharing sites such as Patreon. |
| Patreon said it had a "zero tolerance" policy about such imagery on its site. | Patreon said it had a "zero tolerance" policy about such imagery on its site. |
| The National Police Chief's Council said it was "outrageous" that some platforms were making "huge profits" but not taking "moral responsibility". | The National Police Chief's Council said it was "outrageous" that some platforms were making "huge profits" but not taking "moral responsibility". |
| And GCHQ, the government's intelligence, security and cyber agency, has responded to the report, saying: "Child sexual abuse offenders adopt all technologies and some believe the future of child sexual abuse material lies in AI-generated content." | And GCHQ, the government's intelligence, security and cyber agency, has responded to the report, saying: "Child sexual abuse offenders adopt all technologies and some believe the future of child sexual abuse material lies in AI-generated content." |
| The makers of the abuse images are using AI software called Stable Diffusion, which was intended to generate images for use in art or graphic design. | The makers of the abuse images are using AI software called Stable Diffusion, which was intended to generate images for use in art or graphic design. |
| AI enables computers to perform tasks that typically require human intelligence. | AI enables computers to perform tasks that typically require human intelligence. |
| The Stable Diffusion software allows users to describe, using word prompts, any image they want - and the program then creates the image. | The Stable Diffusion software allows users to describe, using word prompts, any image they want - and the program then creates the image. |
| But the BBC has found it is being used to create life-like images of child sexual abuse, including of the rape of babies and toddlers. | But the BBC has found it is being used to create life-like images of child sexual abuse, including of the rape of babies and toddlers. |
| UK police online child abuse investigation teams say they are already encountering such content. | UK police online child abuse investigation teams say they are already encountering such content. |
| Journalist Octavia Sheepshanks says there has been a "huge flood" of AI-generated images | Journalist Octavia Sheepshanks says there has been a "huge flood" of AI-generated images |
| Freelance researcher and journalist Octavia Sheepshanks has been investigating this issue for several months. She contacted the BBC via children's charity the NSPCC in order to highlight her findings. | Freelance researcher and journalist Octavia Sheepshanks has been investigating this issue for several months. She contacted the BBC via children's charity the NSPCC in order to highlight her findings. |
| "Since AI-generated images became possible, there has been this huge flood… it's not just very young girls, they're [paedophiles] talking about toddlers," she said. | "Since AI-generated images became possible, there has been this huge flood… it's not just very young girls, they're [paedophiles] talking about toddlers," she said. |
| What is AI and is it dangerous? | What is AI and is it dangerous? |
| A "pseudo image" generated by a computer which depicts child sexual abuse is treated the same as a real image and is illegal to possess, publish or transfer in the UK. | A "pseudo image" generated by a computer which depicts child sexual abuse is treated the same as a real image and is illegal to possess, publish or transfer in the UK. |
| The National Police Chiefs' Council (NPCC) lead on child safeguarding, Ian Critchley, said it would be wrong to argue that because no real children were depicted in such "synthetic" images - that no-one was harmed. | The National Police Chiefs' Council (NPCC) lead on child safeguarding, Ian Critchley, said it would be wrong to argue that because no real children were depicted in such "synthetic" images - that no-one was harmed. |
| He warned that a paedophile could, "move along that scale of offending from thought, to synthetic, to actually the abuse of a live child". | He warned that a paedophile could, "move along that scale of offending from thought, to synthetic, to actually the abuse of a live child". |
| Abuse images are being shared via a three-stage process: | Abuse images are being shared via a three-stage process: |
| Paedophiles make images using AI software | Paedophiles make images using AI software |
| They promote pictures on platforms such as Japanese picture sharing website called Pixiv | They promote pictures on platforms such as Japanese picture sharing website called Pixiv |
| These accounts have links to direct customers to their more explicit images, which people can pay to view on accounts on sites such as Patreon | These accounts have links to direct customers to their more explicit images, which people can pay to view on accounts on sites such as Patreon |
| Some of the image creators are posting on a popular Japanese social media platform called Pixiv, which is mainly used by artists sharing manga and anime. | Some of the image creators are posting on a popular Japanese social media platform called Pixiv, which is mainly used by artists sharing manga and anime. |
| But because the site is hosted in Japan, where sharing sexualised cartoons and drawings of children is not illegal, the creators use it to promote their work in groups and via hashtags - which indexes topics using key words. | But because the site is hosted in Japan, where sharing sexualised cartoons and drawings of children is not illegal, the creators use it to promote their work in groups and via hashtags - which indexes topics using key words. |
| A spokesman for Pixiv said it placed immense emphasis on addressing this issue. It said on 31 May it had banned all photo-realistic depictions of sexual content involving minors. | A spokesman for Pixiv said it placed immense emphasis on addressing this issue. It said on 31 May it had banned all photo-realistic depictions of sexual content involving minors. |
| The company said it had proactively strengthened its monitoring systems and was allocating substantial resources to counteract problems related to developments in AI. | The company said it had proactively strengthened its monitoring systems and was allocating substantial resources to counteract problems related to developments in AI. |
| Ms Sheepshanks told the BBC her research suggested users appeared to be making child abuse images on an industrial scale. | Ms Sheepshanks told the BBC her research suggested users appeared to be making child abuse images on an industrial scale. |
| "The volume is just huge, so people [creators] will say 'we aim to do at least 1,000 images a month,'" she said. | "The volume is just huge, so people [creators] will say 'we aim to do at least 1,000 images a month,'" she said. |
| Comments by users on individual images in Pixiv make it clear they have a sexual interest in children, with some users even offering to provide images and videos of abuse that were not AI-generated. | Comments by users on individual images in Pixiv make it clear they have a sexual interest in children, with some users even offering to provide images and videos of abuse that were not AI-generated. |
| Ms Sheepshanks has been monitoring some of the groups on the platform. | Ms Sheepshanks has been monitoring some of the groups on the platform. |
| "Within those groups, which will have 100 members, people will be sharing, 'Oh here's a link to real stuff,'" she says. | |
| "The most awful stuff, I didn't even know words [the descriptions] like that existed." | "The most awful stuff, I didn't even know words [the descriptions] like that existed." |
| Different pricing levels | Different pricing levels |
| Many of the accounts on Pixiv include links in their biographies directing people to what they call their "uncensored content" on the US-based content sharing site Patreon. | Many of the accounts on Pixiv include links in their biographies directing people to what they call their "uncensored content" on the US-based content sharing site Patreon. |
| Patreon is valued at approximately $4bn (£3.1bn) and claims to have more than 250,000 creators - most of them legitimate accounts belonging to well-known celebrities, journalists and writers. | Patreon is valued at approximately $4bn (£3.1bn) and claims to have more than 250,000 creators - most of them legitimate accounts belonging to well-known celebrities, journalists and writers. |
| Fans can support creators by taking out monthly subscriptions to access blogs, podcasts, videos and images - paying as little as $3.85 (£3) per month. | Fans can support creators by taking out monthly subscriptions to access blogs, podcasts, videos and images - paying as little as $3.85 (£3) per month. |
| But our investigation found Patreon accounts offering AI-generated, photo-realistic obscene images of children for sale, with different levels of pricing depending on the type of material requested. | But our investigation found Patreon accounts offering AI-generated, photo-realistic obscene images of children for sale, with different levels of pricing depending on the type of material requested. |
| One wrote on his account: "I train my girls on my PC," adding that they show "submission". For $8.30 (£6.50) per month, another user offered "exclusive uncensored art". | One wrote on his account: "I train my girls on my PC," adding that they show "submission". For $8.30 (£6.50) per month, another user offered "exclusive uncensored art". |
| The BBC sent Patreon one example, which the platform confirmed was "semi realistic and violates our policies". It said the account was immediately removed. | The BBC sent Patreon one example, which the platform confirmed was "semi realistic and violates our policies". It said the account was immediately removed. |
| Patreon said it had a "zero-tolerance" policy, insisting: "Creators cannot fund content dedicated to sexual themes involving minors." | Patreon said it had a "zero-tolerance" policy, insisting: "Creators cannot fund content dedicated to sexual themes involving minors." |
| The company said the increase in AI-generated harmful content on the internet was "real and distressing", adding that it had "identified and removed increasing amounts" of this material. | The company said the increase in AI-generated harmful content on the internet was "real and distressing", adding that it had "identified and removed increasing amounts" of this material. |
| "We already ban AI-generated synthetic child exploitation material," it said, describing itself as "very proactive", with dedicated teams, technology and partnerships to "keep teens safe". | "We already ban AI-generated synthetic child exploitation material," it said, describing itself as "very proactive", with dedicated teams, technology and partnerships to "keep teens safe". |
| The NPCC's Ian Critchley said it was a "pivotal moment" for society | The NPCC's Ian Critchley said it was a "pivotal moment" for society |
| AI image generator Stable Diffusion was created as a global collaboration between academics and a number of companies, led by UK company Stability AI. | AI image generator Stable Diffusion was created as a global collaboration between academics and a number of companies, led by UK company Stability AI. |
| Several versions have been released, with restrictions written into the code that control the kind of content that can be made. | Several versions have been released, with restrictions written into the code that control the kind of content that can be made. |
| But last year, an earlier "open source" version was released to the public which allowed users to remove any filters and train it to produce any image - including illegal ones. | But last year, an earlier "open source" version was released to the public which allowed users to remove any filters and train it to produce any image - including illegal ones. |
| Stability AI told the BBC it "prohibits any misuse for illegal or immoral purposes across our platforms, and our policies are clear that this includes CSAM (child sexual abuse material). | Stability AI told the BBC it "prohibits any misuse for illegal or immoral purposes across our platforms, and our policies are clear that this includes CSAM (child sexual abuse material). |
| "We strongly support law enforcement efforts against those who misuse our products for illegal or nefarious purposes". | "We strongly support law enforcement efforts against those who misuse our products for illegal or nefarious purposes". |
| As AI continues developing rapidly, questions have been raised about the future risks it could pose to people's privacy, their human rights or their safety. | As AI continues developing rapidly, questions have been raised about the future risks it could pose to people's privacy, their human rights or their safety. |
| Jo [full name withheld for security reasons], GCHQ's Counter Child Sexual Abuse (CCSA) Mission Lead, told the BBC: "GCHQ supports law enforcement to stay ahead of emerging threats such as AI-generated content and ensure there is no safe space for offenders." | Jo [full name withheld for security reasons], GCHQ's Counter Child Sexual Abuse (CCSA) Mission Lead, told the BBC: "GCHQ supports law enforcement to stay ahead of emerging threats such as AI-generated content and ensure there is no safe space for offenders." |
| The NPCC's Ian Critchley said he was also concerned that the flood of realistic AI or "synthetic" images could slow down the process of identifying real victims of abuse. | The NPCC's Ian Critchley said he was also concerned that the flood of realistic AI or "synthetic" images could slow down the process of identifying real victims of abuse. |
| He explains: "It creates additional demand, in terms of policing and law enforcement to identify where an actual child, wherever it is in the world, is being abused as opposed to an artificial or synthetic child." | He explains: "It creates additional demand, in terms of policing and law enforcement to identify where an actual child, wherever it is in the world, is being abused as opposed to an artificial or synthetic child." |
| Mr Critchley said he believed it was a pivotal moment for society. | Mr Critchley said he believed it was a pivotal moment for society. |
| "We can ensure that the internet and tech allows the fantastic opportunities it creates for young people - or it can become a much more harmful place," he said. | "We can ensure that the internet and tech allows the fantastic opportunities it creates for young people - or it can become a much more harmful place," he said. |
| Children's charity the NSPCC called on Wednesday for tech companies to take notice. | |
| "The speed with which these emerging technologies have been co-opted by abusers is breath-taking but not surprising, as companies who were warned of the dangers have sat on their hands while mouthing empty platitudes about safety," said Anna Edmundson, the charity's head of policy and public affairs. | |
| "Tech companies now know how their products are being used to facilitate child sexual abuse and there can be no more excuses for inaction." | |
| Related Topics | Related Topics |
| Social media | Social media |
| Child abuse | Child abuse |
| Artificial intelligence | Artificial intelligence |
| NSPCC | NSPCC |