It’s Valentine’s Day, and a man wakes up to a text saying ‘I love you’, followed by a joke that only his partner would know. Half an hour later, his phone buzzes and lights up with a selfie of her. He smiles.
The Black Mirror style twist in this story is that his girlfriend left him two years ago and the messages and the picture are creations of generative AI. The AI was trained on data in the man’s possession: pictures taken on holiday when they were together, text messages and WhatsApp conversations.
Unnerving? Yes. Wrong? Probably. But illegal? Maybe not. And this will soon be the question facing legislators.
An emerging niche is ‘grief tech‘, training an AI to speak in a loved one’s voice. It’s the step beyond reading a loved one’s letters or looking over their photos. With the right tech, you can interact online with a digital footprint of them. But the problem coming down the tracks is that the technological capability will not be limited to expressions of grief. The tech that recreates a dead loved one will work the same way when asked to create a replica of a living ex-girlfriend, or co-worker, or friend.
The canary in the coalmine is deepfake porn, the creation and circulation of fake pornographic images: in 2019 deepfake porn was estimated to be over 90% of online deepfake content. As with revenge porn, it’s primarily used as an intimidation tool against women. But also as with revenge porn, once the law has been updated to cover the method, it slots into criminal legislation. While not perfect, the Online Safety Act in the UK had the stated intent of banning deepfake porn, which in the final drafting became an amendment to the Sexual Offences Act to cover manufactured images. Whatever the faults of the Act, it placed this type of online replication within the boundaries of criminal law.
But what about the fake selfie? That scratches the surface of a legislative minefield almost entirely within civil law. Here, there aren’t existing guardrails because we have very limited rights over something very basic: our likeness. That’s why, when journalist Natalie Pinkham had photographs of her with Prince Harry stolen from her flat in the mid-2000s and published without her consent by The Sun, Pinkham fought the battle over copyright. An individual’s right over a photo of their lunch, taken by them, is absolute under copyright law. But their right over a photograph of them eating it, taken by someone else, is minimal. Historically, this has caused tensions with the right to privacy. The test cases in the early and mid-2000s largely involved celebrities and public figures and had legitimate trade-offs with other rights like freedom of speech and freedom of the press. But now, this question could cover everyone. If someone uses AI to generate videos of their co-worker telling them ‘I love you’ every morning and they have trained the data on photos of the co-worker taken on public property, it is not clear where a definitive legal breach lies. Even catfishing – impersonating someone online – is not illegal in the UK unless you are impersonating a police officer or solicitor.
There are cases that could be made and tools that could be deployed – defamation, trespass to the person, intention of psychiatric harm, protection from harassment, photograph copyright – but it is ‘could’ not ‘would’. Most of these legal remedies lie within civil law, meaning a claimant willing to put the time and money into bringing forward a case. In the case of the messages, while it is arguable that the author retains copyright, messages sent ‘from’ the ex-girlfriend are newly generated based on copyrighted training data and that’s a different emerging issue within AI.
The law evolves, particularly civil law. Rights to privacy evolved with the growth of the media, with several public tensions with freedom of the press. The same is true over rights to personal information and GDPR. Along with bodily autonomy, the right to personality and identity could be considered the most basic legal entitlement there is: but we have never needed it to be defined in law. We will need it soon.
Click here to subscribe to our daily briefing – the best pieces from CapX and across the web.
CapX depends on the generosity of its readers. If you value what we do, please consider making a donation.
