Science

Internal Facebook documents showed that the platform once weighted every reaction emoji five times more heavily than Likes — a design choice that quietly amplified provocative content because strong reactions like outrage counted for more


Beginning in 2017, Facebook’s ranking algorithm treated a reaction emoji as five times more valuable than a Like. Not the angry face specifically. All of them. Love, haha, wow, sad and angry each counted for five, while the old thumbs-up counted for one. This is the detail that got flattened almost everywhere the story travelled, and it is worth holding onto.

The reporting comes from a single source: the Washington Post’s October 2021 account, by Jeremy Merrill and Will Oremus, of the internal documents that former employee Frances Haugen provided to Congress and the Securities and Exchange Commission. You can read the full article syndicated at the Seattle Times. It is careful reporting drawn from company memos, but it is one outlet’s read of one document set. This is one study of a company’s internal life, not settled consensus about how the platform behaved.

The logic behind the five was mundane. In 2017 Facebook was watching a decline in how much people posted and talked to each other, and the reaction emoji gave it new signals to work with. As the company reasoned in the documents, reacting with an emoji took an extra step beyond a single tap, so it was read as a stronger sign that a post had landed. More effort in, more weight out. The intention was reasonable enough. What it selected for was the problem.

The harm was downstream of the weighting

Here the story separates into two claims that often get welded together, and the difference matters.

The first claim is the weighting itself: reactions counted five times a Like. That is a design choice, made in 2017, applied across all five emoji. The second claim is what Facebook’s own data scientists found later. By 2019, per the Post, the company’s researchers had concluded that posts drawing a heavy share of angry reactions were disproportionately likely to carry misinformation, toxicity and low-quality news. An internal study that November, reported by NBC News, found that angry, haha and wow reactions turned up more often on low-quality and misleading civic and health content than on other material. The angry face carried no more weight than love or sad. The trouble was that angry-heavy content turned out to be worse content, and the algorithm was amplifying whatever drew reactions.

So the mechanism was cruder than deliberate outrage-farming. A blunt setting boosted all strong reactions, and ran straight into the fact that the posts most reliable at generating strong reactions were often the ones designed to provoke. A staffer had flagged this early. As Nieman Lab noted in its summary of the documents, one employee asked directly whether weighting reactions five times stronger than Likes would tilt the feed toward controversial content over agreeable content. Another acknowledged the concern was possible. The warning sat in the record for three years before the weighting was meaningfully changed.

Why the correction is more interesting than the myth

The compressed version of this story, the one that says Facebook cranked up anger specifically because anger keeps people scrolling, is emotionally satisfying and slightly wrong. The technology writer Mike Masnick argued at Techdirt that the coverage buried the fact that all five reactions started equal, and that the framing made a deliberate outrage machine out of what the documents describe as a cruder mistake.

He has a point worth taking, though it does not exonerate the outcome. The distinction is between malice and a certain kind of institutional carelessness. Nobody in the documents appears to have sat down to engineer rage. What happened instead is that a company optimised for a proxy, engagement measured through reactions, and the proxy quietly favoured the content least worth favouring. That is a more common failure than deliberate manipulation, and a harder one to legislate against, because everyone involved can honestly say they were only trying to measure interest.

The pattern under the pattern

This is where the Facebook case stops being about Facebook. Any system that measures a thing in order to reward it eventually rewards whatever games the measure. Charles Goodhart’s observation, later sharpened into the line that a measure which becomes a target stops being a good measure, was made about monetary policy in 1975. It keeps reappearing wherever institutions try to run themselves on a single number.

What the reaction emoji added was speed and reach. A newsroom that chases clicks distorts itself slowly, over editorial meetings and reworked headlines. A ranking algorithm applies the same distortion to billions of feeds in an afternoon, invisibly, with no one in the loop deciding that this particular divisive post about abortion or guns should go to a wider audience today. The Post reported that the angry reaction skewed heavily toward divisive topics, with one outlet, Fox News, drawing nearly double the angry reactions of any other page. No editor chose that. The weighting chose it.

The uncomfortable part is that the emotional impression the emoji were meant to capture was real. People genuinely did react more strongly to those posts. The signal was real enough. What had come apart was the link between strength of feeling and worth of content, and the system had no way to tell the difference between a post that moved someone and a post that inflamed them.

What the record does and does not settle

Facebook walked the weighting back in stages. It cut the reactions to four times a Like in 2018, built a mechanism to demote content drawing disproportionate angry reactions in 2019, dropped all reactions to one and a half times a Like in 2020, and eventually set the angry reaction to zero. A company spokesperson told the Post it continued working to understand and reduce content that produced negative experiences, including posts with a disproportionate share of angry reactions.

What the documents show is a specific sequence of internal choices at one company over roughly three years, as characterised by one newspaper reading a leaked cache. What they do not show, and cannot, is the counterfactual. We do not know how much of the outrage in public life over those years was manufactured by a ranking weight and how much was already there, waiting for any amplifier at all. The honest position is that the weighting made a bad tendency worse, at scale, and that the people who built it were warned and were slow.

The thing to watch is not whether a single reaction gets set to zero. It is whether the underlying habit changes: running enormous social systems on a proxy for attention, then discovering three years later what the proxy was actually selecting for.



Source link