Nudification apps have no legitimate reason to exist, says online reporting platform

Mick Moran, chief executive of Hotline.ie, warned that the fact that users of X’s AI chatbot Grok could create sexualised images based on photographs of real women and children is going to keep happening. File photo

So-called 'nudification' apps have no reason to exist and should be banned, an online reporting platform has said.

Mick Moran, chief executive of Hotline.ie, has also called again for it to be illegal for any artificial intelligence (AI) app to produce child sexual abuse material (CSAM).

However, he warned that the fact that users of X’s AI chatbot Grok could create sexualised images based on photographs of real women and children is going to keep happening.

He said: “What has not been addressed is — can content that is illegal in the real world be illegal in the online world? It is all linked into pornography, extreme pornography that is connected to gender-based violence.

“These things do not sit in isolation in a connected world. Nudification apps have no legitimate reason to exist in the world. It is touching on the bodily integrity of humans and it should not exist. There is no reason for it.” 

He was responding to X owner Elon Musk’s statement that its AI chatbot Grok will no longer allow users to edit people in revealing clothing in places where it is illegal.

What's your view on this issue?

You can tell us here

Causing outrage among people in countries all over the world, Grok has allowed users to create sexualised images of women and children by "undressing people" from real photographs.

Mr Musk said in a statement that X has implemented technological measures to prevent Grok users from allowing the editing of images of real people in revealing clothing such as bikinis, just days after laughing off the controversy.

This applies to users who generate nude or near-naked images of real people “in those jurisdictions where it’s illegal” to do so. The block, however, doesn’t apply to Ireland because while it is illegal to share such images, it isn’t illegal to generate images of real people in a state of undress.

The statement was issued the same day gardaí confirmed they are currently investigating 200 child sexual abuse images generated by Grok. Asked on RTÉ'S Morning Ireland if he believed there should be a law against them, Mr Moran replied: “Yes.” 

He added: 

While we are at it, we should make it illegal for AI to produce CSAM (child sexual abuse material) in the first place. 

He said the “race for AI development” had to stop as many AI engines were not moderated. 

“Is this the future? Is this the future for every new functionality and ability for AI to produce, to do something?” he asked.

“Is it to be run out like this with no guidelines and no guardrails in place and no safety built in at design level and then for the public to react and then for regulators and legislators to catch up?” 

He warned: “This is going to happen again and again and again as new functionality is brought out. So the functionality is really the key at the end of the day."