COMMENT
DR JULIA SHAW:
CONTENT MODERATORS PAY A PSYCHOLOGICAL TOLL TO KEEP SOCIAL MEDIA CLEAN
We should be offering more support to the people who help to protect us from distressing online content
Content moderators are the unsung heroes of the internet. They work in a growing field which upholds the social media infrastructure of today. But keeping us safer, by constantly seeing and filtering the worst content online, takes its toll. Can people really cope with this constant barrage of horror?
One of the main handbooks for psychologists, the DSM-5, includes ‘indirect exposure to aversive details’ in the category of post-traumatic stress disorder. In extreme circumstances this can result in what is often called ‘secondary trauma’.
Secondary trauma can happen when people, such as first responders, deal with victims in distress. It can also occur after seeing or reading distressing content. Among others, those who work in content moderation for social media companies, in medicine or psychology, and in social work, can all relate to how unhelpfully memorable horrible cases can be.
Personally, I have had flashbacks as the result of my work as a criminal psychologist. The most intrusive were after a case where I had an unusual amount of access to video content, including zoomed-in shots of highly emotional victim statements.
Among the first to study the specific impact of violent videos was a team led by stress and trauma researcher, Arija Birze. In 2022, the team published the results of two studies in which police, lawyers, judges, psychiatrists, law clerks and court reporters were interviewed.
In the first study, they found that in criminal justice settings, violent videos increase ‘novel emotional proximity’. It’s harder to stay unaffected when you can see every second of emotion in hyper-realistic detail. This is because, compared to photos, being able to see dynamic facial expressions leads to more activation of the parts of the brain responsible for empathy and emotional memory.
They also found what they named ‘perpetual visibility’. Before cameras and high-res footage, most of the intimate details of a crime would have been unknowable, imperceptible, or fleeting. Details like the look on a victim’s face while being attacked were previously limited to our imaginations. But now we are seated in the front row, sometimes even closer than the actual witnesses.
The researchers also found that people were often blindsided by the violence. It can be impossible to be fully prepared for graphic images, even if you know they are coming. Added to this is an almost complete lack of training for many people dealing with these shocking videos.
It makes sense that the psychology of victims and perpetrators is prioritised, but the wider psychological fallout of these crimes also needs to be considered. Particularly because those in criminal justice and content moderation often need to watch video evidence over, and over, and over.
“The negative effects of violent content are cumulative, and compassion fatigue can develop”
Unfortunately, in a second study, the researchers found that organisations have failed to keep up with the rise in disturbing video content. Criminal justice professionals are typically only offered help when they are so affected they can no longer do their jobs. Many don’t even get offered help then.
Because of this, people over-rely on colleagues who are themselves struggling, which can create what the researchers call an intense ‘trauma contagion’. This means that some people who are traumatised are amplifying each other’s suffering.
Interventions have been developed for content moderators, with mixed success. For example, in 2022 a group of researchers added positive content to breaks. Between distressing posts, they were shown pictures of cute baby animals and awe-inspiring landscapes. These kinds of pictures have been shown to decrease stress in other research settings, but here it had the opposite of the intended effect. This might be due to ‘affective contrast’, the happy images made the negative seem even worse.
It seems that the negative effects of violent content are cumulative, and compassion fatigue can develop over time. That’s when people have spent so much time helping safeguard others that they get burned out and their emotions become flat.
For professionals who regularly deal with distressing video content, in addition to organisations providing formal support, taking a break from the content can help. It’s also important to regularly zoom out. People who focus on general aspects of crimes and content, compared to specific details, are less likely to have flashbacks.
As technologies develop and new ones emerge, research needs to consider how to ensure people who keep us safe remain psychologically healthy.
DR JULIA SHAW
(@drjuliashaw) Julia is a psychological scientist at University College London, the author of multiple best-selling books, and the co-host of the hit podcast Bad People on BBC Sounds.
Listen to Dr Julia Shaw on the Bad People podcast, available on BBC Sounds.