Childline: The danger of AI with some tech firms neglecting legal responsibilities to protect children

The NSPCC’s Childline counsellors hear first-hand from children and young people here in our region and across the country about their devastation when nude images of them are shared.

Recently, headlines have been filled with stories about innocent images of young people being doctored using generative AI (Artificial Intelligence), to remove clothing and create illegal sexualised material which could be used to humiliate, bully or extort victims.

Young people are in danger of seeing more harmful content created by AI on their mobile phones. Photo posed by modelplaceholder image
Young people are in danger of seeing more harmful content created by AI on their mobile phones. Photo posed by model

These AI apps are alarmingly easy to exploit, and the technologies are a major risk to children and young people.

Yet once again, the tech firms which create or operate these apps continue to neglect their legal responsibilities to protect children from this horrific form of abuse.

It’s a growing risk too, as AI, chatbots and related terms were mentioned to our Childline counsellors in 367 counselling sessions between April 2025 and September 2025. That’s more than four times as many times as in the same time period the previous year.

Young people have told us they use AI to rate their looks, body and weight, to “diagnose” eating disorders or to plan long fasts or calorie restricted diets. Others have told us they have encountering harmful, inappropriate, or misleading content via an AI app, some have been bullied with AI-generated content like fake images or videos which have been shared on social media or in school.

We have also heard how AI chatbots have dissuaded young people from telling safe adults about abuse, in some cases telling the child their experience was not abuse or that the abuse was their fault.

These are terrifying developments and really highlight the importance of ensuring young people know how to access genuine safeguarding advice and services like Childline. Some young people might prefer to share their concerns in writing rather than phoning 0800 1111, but it’s vital they know they can get through to our specially trained counsellors through the Childline website – www.childline.org.uk – where they will be listened to and supported, no matter what they are going through.

It's small consolation, but consolation nonetheless, that our counsellors have heard that some AI chatbots have directed children to the Childline service.

Our counsellors have been hearing concerns from young people about AI since 2019 and will be there whenever they need us going forward.

News you can trust since 1877
Follow us
© 2026 Iconic Media Group Ltd. All rights reserved.Cookie SettingsTerms and ConditionsPrivacy notice