Regulator to recommend ban of AI 'nudification' systems

It comes over a month on from the emergence of the controversy surrounding social media platform X, owned by Elon Musk, and its Grok AI tool. File photo: Yui Mok/PA

The Government should consider banning AI “nudification” systems that can generate sexualised images of real people without their consent or create child sex abuse material, the Irish media regulator will say on Tuesday.

Addressing the Oireachtas Artificial Intelligence Committee, Coimisiún na Meán executive chair Jeremy Godfrey will say that changes to how tech firms operate as a result of regulation “have not yet been sufficient” while new types of online harm have emerged.

“We think it is worth considering whether to supplement these requirements [they already have under the law],” Mr Godfrey will tell TDs and Senators.

“For instance, it could be useful to make it a prohibited practice to deploy AI systems that are capable of producing intimate imagery of real people without their consent, or which are capable of producing child sex abuse material.

“It may also be useful to widen the scope of ‘high-risk systems’ under the AI Act to include a wider range of chatbots and generative AI tools.” 

On Tuesday, the AI Committee will hear representatives from Coimisiún na Meán, the Data Protection Commission and Google on recent issues around images, deepfakes and consent.

Grok and X

It comes over a month on from the emergence of the controversy surrounding social media platform X, owned by Elon Musk, and its Grok AI tool.

A global outcry was sparked after non-consensual intimate images were generated by users prompting Grok to do so, putting adults and children in bikinis, underwear or in sexually suggestive poses, and disseminated on X.

Facing pressure from numerous jurisdictions, X eventually said it would limit the ability to use Grok in this way in countries where this was illegal.

While Coimisiún na Meán is collaborating with the European Commission on an investigation into the affair, the Data Protection Commission also recently launched a “large-scale inquiry” into X on the same issue.

Mr Godfrey will tell the committee that while the investigation into Grok is a live one, he will not comment further about it.

Other risks from AI systems

But, he will say that 'nudification' is not the only way generative AI might produce illegal content that depicts real people without their consent.

“For instance, it can be used to produce deep fakes that incite hatred or violence,” Mr Godfrey will tell politicians. “And it can be used to produce scam ads that include a purported endorsement by a public figure.

“It is important that deployers of generative AI systems should be alive to the risks of misuse and they should implement safeguards to mitigate them.” 

According to Mr Godfrey, in the three years Coimisiún na Méan has been operating, platforms have already made changes that have made the online world safer.

This includes new age assurance measures, platforms prohibiting pornography or violence in their terms and conditions, and one platform withdrawing a proposed product with “damaging addictive design features”.

“However, the changes have not yet been sufficient and new types of harm, especially harms related to AI, have emerged,” he will add.