Can AI image generators be policed to prevent explicit deepfakes of children? 

As one of the largest ‘training’ datasets has been found to contain child sexual abuse material, Alex Hern asks can bans on creating such imagery be feasible?

A manual found on the dark web contained a section encouraging criminals to use “nudifying” tools to remove clothing from underwear shots sent by a child. The manipulated image could then be used against the child to blackmail them into sending more graphic content.

Child abusers are creating AI-generated “deepfakes” of their targets in order to blackmail them into filming their own abuse, beginning a cycle of sextortion that can last for years.

In Britain, the Internet Watch Foundation (IWF) said a manual found on the dark web contained a section encouraging criminals to use “nudifying” tools to remove clothing from underwear shots sent by a child. The manipulated image could then be used against the child to blackmail them into sending more graphic content, the IWF said.