Caitríona Redmond: Why AI means we need to pay social media extra scrutiny

Emotive AI crafted pieces on social media are designed to draw you in, but should be taken with a pinch of salt.

We all like to think that we wouldn’t click on a post or comment on something that may be dodgy but consider images of missing people which can be very emotive and spread like wildfire. 

“I'm literally shaking as I type this”. So starts one of the more viral AI-crafted pieces on Facebook in the past week. It draws the reader in with emotion and entices us to read more.

As I write this week’s column, there are multiple variants of this post which accompanies several images of an intricately carved wooden cradle with a canopy. 

The cradle has a distinctive yellow sticker of a $5,000 sale price. I’m not shaking, unlike the author who has the uncanny ability to write exactly the same prose in several languages under differing Facebook profiles.

The cradle in question first appeared on the Bored Panda website in 2019 in a post about “weird, odd second-hand finds”, which is apt considering its appearance. 

In all likelihood, the cradle exists somewhere on earth and was on sale at some point but not discovered in the past week by several people who seem to be located across the globe but have found the exact same item.

The text on the post goes on to explain how the author found their family heirloom cradle, carved in the Victorian era by artisans. 

It had stayed in the family home until their grandparents had to sell due to medical bills. They describe in detail how, when they found the item, they had to have it at all costs.

Unfortunately, if it looks like a duck and quacks like a duck, that isn’t enough indication that what you see on social media is actually a duck. Everything we know to be true in real life has to be taken with a healthy pinch of salt when it comes to online posts.

Hidden near the end of this wonderfully emotive piece of prose is a personal recommendation for the Tedooo app, explaining how the community on the app would help them restore the family heirloom. Don’t you think that’s a bit odd?

A friend showed me the post on a Facebook group because they have an interest in woodwork, and they thought the cradle was remarkable. They had taken the writing at face value and asked me what I thought of said app.

Swiftly tapping on the profile of the poster who found the cradle told a different story. Yes, they love crafting and DIY, but they also seem to have an unhealthy obsession with the Tedooo app (pronounced To-Do). 

In fact, all of their posts since they joined the group have had a reference to the app helping their craft business or giving them access to experts to help with their renovations.

I skimmed over the posts and also discovered that for somebody who has apparently downsized to a one-bedroom apartment, they also had many children, a lovely large garden, and outsized furniture. 

They also shared images of children and vulnerable people holding wooden items which attracted many comments and likes.

This is a classic example of engagement farming which circumvents the rules of advertising on Facebook and elsewhere. 

It’s a manipulative strategy designed to collect likes, shares, and comments and exploit an algorithm that encourages businesses to advertise to increase their visibility.

Social media is rife with scams like this, designed to draw readers in and persuade them to download an app or buy something they may not need. Even the act of engaging with the post by commenting may attract unwelcome direct messages from scammers who see a vulnerability in a commenter.

Tracking that distinctive cradle across the internet shows me that it has been reshared in French, Russian, and even Chinese language websites by different individuals using exactly the same spiel, right down to endorsing an app. 

Other items that the posters have in common are thousands of friends or followers, but their personal profile is locked down and you cannot even determine their location.

Some of the profiles contain a link to a separate website containing discount codes for many websites, including Tedooo, but also to meal kit boxes and retail websites like Shein and Temu.

These viral posts use images that have been poached from elsewhere on the internet. 

Ask yourself, how would you feel if an innocent image you posted from your family home or an outing was used for this purpose. Review your security settings on your social media and, at the very least, ensure that images you share are set to friends only.

We can protect ourselves from engaging with this type of content on social media by being aware of their clear emotional arc. 

Each post commences with a story of loss, then develops into a discovery, restoration of the item and a moral at the end. The mention of the product is embedded seamlessly within the story, and it’s not just limited to one specific app or website.

We all like to think that we wouldn’t click on a post or comment on something that may be dodgy but consider images of missing people which can be very emotive and spread like wildfire. 

The moral of the story this week is to take everything you read on social media with a pinch of salt.