Colman Noctor: Algorithms targeting teens with disturbing content 

The gateway to misogynistic content often begins with videos devoted to male motivation, money-making, and performance.

The internet is the greatest library we have ever known, but we must also accept that algorithms vet and direct the information we see.

Recent research from Dublin City University’s Anti-Bullying Centre and a recent RTÉ Prime Time investigation highlighted how influential algorithms are targeting young people’s devices with disturbing content.

DCU’s Anti-Bullying Centre found that recommender algorithms used by social media platforms rapidly amplify misogynistic and male supremacist content. Meanwhile, Prime Time illustrated how TikTok’s recommender system delivers a stream of videos almost exclusively related to depression, self-harm, and suicidal thoughts to users it believed to be 13 years old.

The power of propaganda is not new. History has shown us how much it can impact our views, beliefs, and actions. The only difference is smart technologies are more sophisticated and can target individual users.

The DCU study, conducted by Prof Debbie Ging, Dr Catherine Baker, and Dr Maja Andreasen, found that fake accounts registered as teenage boys, resulted in masculinist, anti-feminist, and other extremist content dominating their feeds. This male supremacist content was made available within the first 23 minutes of the beginning of the experiment.

Once the fake user appeared to show interest and engage in watching this content, it rapidly increased. Both platforms were equally culpable, with 76% of TikTok and 78% of YouTube Shorts material falling into the category of ‘manosphere’, which promotes alpha male and anti-feminist content.

The gateway to misogynistic content often begins with videos devoted to male motivation, money-making, and performance. This material strategically taps into boys’ financial and emotional insecurities and offers quick-fix ways of achieving status, happiness, and satisfaction. The creators talk about ‘pushing yourself to succeed’ and having a ‘laser-like focus’ on your goals, promising the viewer quick and impressive outcomes. This content quickly moves from motivational to alpha-male domination and can contain dangerous mental health messaging, for example, that depression is a sign of weakness and therapy is ineffective.

‘Manfluencers’ accounted for most of the recommended videos in the DCU dataset, demonstrating their centrality in the current manosphere ecosystem. By far, the most prevalent was Andrew Tate, who featured 582 times on the YouTube Shorts accounts and 93 times on the TikTok accounts.

Disturbing content

A similar story emerged from Prime Time’s investigation, which involved creating TikTok accounts imitating 13-year-old users. The TikTok algorithm directed these phantom users to disturbing mental health content, and within 20 minutes, they were shown videos referencing self-harm and suicidal thoughts. Within an hour of scrolling, TikTok’s recommender system was showing videos almost exclusively related to depression, self-harm, and suicidal thoughts to users it believed to be 13 years old.

‘Time on-screen’ is the driving metric of social media platforms as it is directly linked to revenue streams. The more people watch your platform, the more data you gather and the more advertising revenue generated. In the race for our attention, content that captivates and commands our attention is promoted, regardless of its ethical persuasion. The online world is not held to account, so content that gains traction will be prioritised, irrespective of its potential danger.

We need to accept that online social media companies are not interested in our wellbeing, and we are responsible for mediating our exposure to their content — no simple task for the teenage brain, which has yet to master self-regulation.

What gains traction or ‘goes viral’ is usually extreme online. The sensationalist view will nearly always attract more attention than the moderate one. Content creators know this, so their content will always veer towards the extreme or sensational side of things. These days it’s unpopular to be moderate; engage in compromise to have a balanced perspective. We are being corralled into polarised and extreme views because it’s what we see online.

While working in in-patient mental health environments, I often encountered young people with conditions such as eating disorders competing to be the sickest. In these instances, young people would compete to eat less and move more than their peers to confirm their disorder was worse than the others in the unit. A similar scenario can be seen in relentless posts on hopelessness, self-harm, and suicidal thoughts on teenagers’ social media. The darker and grimmer your post, the more views it will get.

Tortured philosophers

I have worked with many young ‘persecuted souls’ who have spoken to me at length about their life philosophy, which has resulted in their view that ‘life is futile’ and that anyone who doesn’t realise that is in some way ‘lesser’ or ‘not seeing the full picture’.

As with any 15-year-old philosopher with limited life experience, it tends to be a pretty superficial perspective, and with some gentle probing and critiquing, it can be easily remedied. However, this approach is impossible when the ‘tortured philosophers’ post videos online with nobody to challenge their grim life perspective. It is important to point out that just because adults believe this negative life view to be misguided and inaccurate, this does not reduce the risk of these same young people acting on these beliefs.

It is not surprising that deflated teens who have viewed content on the pointlessness of living are attracted to viewing content from the manosphere or body image content. These manfluencers promise motivation, superior mental health, monetary success, and unlimited popularity. After viewing a grim post about the futility of life, this may seem like an antidote to hopelessness.

There needs to be a villain and so the male supremacist influencers will choose feminism or wokeness as the enemy. Girls might find solace in weight loss and other body image content that simplifies life’s tribulations as just a matter of losing weight or getting fillers or other alterations to your body for instant life improvement.

Never have we had so much information but felt so lost. Never have we been so politically split — right and left, liberalism and conservatism. We have witnessed riots on our streets, school stabbings, anti-immigration protests, and a rise in pervasive frustrations, whether it is retail rage or road rage.

Our tolerance for difference is lowering, leaving us open to more extreme views. The moderate voice is dismissed as either ‘sitting on the fence’ or ‘if you are not with us, then you are against us’, leaving many in silent paralysis. If we want this polarisation of society to change, we need to look at the impact of social media algorithms on our society.

Those who are rights-based say we need to invest in education and awareness, while those who are responsibility-based advocate prohibiting smartphone use. I believe we should do both. The answer to this societal problem is not an either/ or but a combination of the two.

We should spare nothing and throw all our resources at this problem because, without a significant reaction, we risk further fragmentation of our beliefs and values.

As adults, we might view content about radical dieting and male supremacy as over the top and dismiss its influence, but we cannot underestimate the impressionability of youth.

Dr Colman Noctor is a child psychotherapist