Calls grow to ban AI ‘nudify’ tools over child abuse and deepfake image fears

Coimisiún na Meán, said it was 'engaging' with the European Commission, which was responsible for the oversight of the large social media platforms, such as X. Picture: Gabby Jones/Bloomberg

The Government needs to ban AI 'nudify' tools if it wants to stop the creation of child sexual abuse imagery and the undressing of women by AI technology, various groups and experts have said.

They said this requires a shift in the criminal laws, from currently focusing only on users of the technology to create the images to include the technology itself and the platforms providing it.

An Garda Síochána is assessing a request by civil and digital rights groups to “urgently investigate” the social media platform X under child sexual abuse material (CSAM) legislation for its Grok AI chatbot over its ability to generate non-consensual deepfake sexualised images of children.

Senior officers are examining the matter and may seek legal opinion, but sources have said that gardaí can only investigate a specific complaint of a criminal offence — in this case the creation of child sex abuse material by a user.

These sources said the technology is not a criminal matter, but a legal or regulatory issue.

A range of organisations have called for a ban on the technology, including Irish Internet Hotline, Rape Crisis Ireland, Children’s Rights Ireland, the Sexual Exploitation Research Programme, and mental health charity Turn2Me.

Irish Internet Hotline, the national reporting centre for online illegal content, said: “It is absolutely critical that the Government acts to make it an offence to possess, create, or distribute AI tools capable of generating deepfake ‘nudification’ images."

The special rapporteur on child protection, Caoilfhinn Gallagher, said the legal and policy focus was on the user generating the images “rather than on the platform or products” facilitating the creation of the images.

She told RTÉ that Ireland “was not alone”, with the international focus on finding a mechanism to “hold platforms to account”, adding she believed they should be regulated more tightly.

Garda sources said that, in this case, the criminal law relates to people who use the Grok technology.

“The technology itself is not illegal, so that's a regulatory matter,” said one source.

The media regulator, Coimisiún na Meán, said it was “engaging” with the European Commission, which was responsible for the oversight of the large social media platforms, such as X.

Coimisiún na Meán declined to say whether or not it has contacted X and sought a meeting. 

Its counterpart in Britain, Ofcom, said last Monday that it had made “urgent contact” with X.

The Department of Communications said the specific use of AI by social media companies was subject to the EU AI Act, but that provisions to enable “surveillance, penalties and enforcement” will not come into effect until August 2026.

Minister of State, Niamh Smyth, who has responsibility for AI, said she has sought a meeting with X management. 

Enterprise minister Peter Burke said the Government would consider strengthening laws, if necessary.

A spokesperson for X said anyone “using or prompting Grok to make illegal content” will suffer the same consequences as if they upload illegal content, including CSAM.

Last month, the British government said it intended to ban "nudification" apps.