The Hidden Engine Behind Myth Spread

Introduction

The spread of myths and misconceptions online is not determined solely by whether a post is removed or fact-checked. Just as important is whether a platform’s recommendation system decides to show that post to ten people, ten thousand people, or millions. Recommender systems are the hidden engines that rank feeds, suggest videos, recommend accounts, and determine what receives attention. When these systems optimise primarily for engagement—clicks, comments, shares, watch time, and reactions—they can unintentionally favour emotionally charged and controversial material. When they incorporate signals related to reliability, user wellbeing, or harm reduction, they can reduce the visibility of misleading claims and slow the spread of myths. The central governance question is therefore not only what content is allowed online, but what content algorithms choose to amplify. [PMC]pmc.ncbi.nlm.nih.govPMCby S Milli · 2025 · Cited by 164 — Social media ranking algorithms typically optimize for users' revealed preferences, i.e. user engag…

Why engagement ranking favours emotional claims

Most major social platforms use recommendation systems that predict what users are likely to engage with. Engagement is attractive as a ranking signal because it is measurable and closely tied to platform growth. Likes, comments, shares, replies, and viewing time all provide immediate feedback about user behaviour. Yet emotional content often performs unusually well on those measures. Anger, outrage, fear, and conflict can provoke stronger reactions than careful explanations or corrections. [PMC]pmc.ncbi.nlm.nih.govPMCby S Milli · 2025 · Cited by 164 — Social media ranking algorithms typically optimize for users' revealed preferences, i.e. user engag…

This creates a structural advantage for some myths and misconceptions. A sensational false claim may generate more comments and shares than a nuanced correction, even if many of those comments are critical. From the perspective of a system optimised for interaction, controversy can look like success.

Research auditing social-media ranking systems has found that engagement-based algorithms can amplify angry, partisan, and hostile content relative to chronological feeds. One large study found that engagement-driven ranking promoted emotionally charged political content that users themselves often reported liking less than alternative ranking approaches. The same research suggested that systems incorporating users’ stated preferences rather than only their behavioural reactions could reduce the prominence of angry and divisive content. [arXiv]arxiv.orgarXivEngagement, User Satisfaction, and the Amplification of Divisive Content on Social MediaMay 26, 2023…Published: May 26, 2023

The mechanism is important. Recommendation systems do not need to “prefer” misinformation in order to spread it. They only need to reward the behaviours that misinformation frequently triggers.

The feedback loop between users and algorithms

Recommendation systems and users influence one another. Users react to content, algorithms learn from those reactions, and future recommendations are adjusted accordingly. Over time, the system can reinforce patterns that maximise engagement even when those patterns increase exposure to misleading claims.

Recent theoretical and empirical research describes this as a feedback loop in which greater weight placed on likes, shares, and other popularity signals can increase engagement while simultaneously increasing misinformation and political polarisation. The more heavily a system rewards social reactions, the more likely highly reactive content is to dominate attention. ScienceDirect [SSRN]papers.ssrn.comSocial Media Algorithms Fuel Misinformation and…To connect this prediction to observational data, we exploit Facebook's “Meaningful So…

How recommender goals affect myth reach

The spread of myths depends heavily on what objective a recommendation system is trying to achieve.

An engagement-maximising system asks a question such as: “Which content is most likely to generate interaction?” A reliability-aware system asks a different question: “Which content is most likely to inform users without causing harm?” The difference may seem subtle, but it changes what rises to the top of a feed.

[Several approaches have emerged:]arxiv.orgsed recommender systems - which form the majority of recommendation engines…Read more…

• Reducing the weight of pure engagement signals. Platforms can lessen the influence of shares, reaction counts, or other popularity metrics that disproportionately reward outrage. [Panoptykon]panoptykon.orgFixing Recommender SystemsPanoptykonFixing Recommender SystemsAugust 22, 2023 — 25 Aug 2023 — In 2018 Facebook introduced a new metric for its News Feed algorithm…Published: August 22, 2023
• Incorporating source quality signals. Recommendations can favour content from sources with stronger records of accuracy and transparency rather than treating every engagement event equally.
• Demoting repeatedly debunked claims. Even when content remains visible, platforms can reduce its algorithmic reach.
• Optimising for user satisfaction rather than immediate interaction. Research suggests that what users engage with in the moment is not always what they later say they wanted to see. Ranking systems based partly on expressed preferences may produce healthier information environments. [PMC]pmc.ncbi.nlm.nih.govPMCby S Milli · 2025 · Cited by 164 — Social media ranking algorithms typically optimize for users' revealed preferences, i.e. user engag…

These design choices matter because recommendation systems determine exposure. A misleading claim with limited reach may remain a fringe belief. The same claim repeatedly recommended to large audiences can become widely accepted.

When platform changes produce unexpected results

Attempts to improve online discourse can sometimes have unintended consequences. One frequently discussed example is Facebook’s 2018 emphasis on “Meaningful Social Interactions”, which increased the importance of comments and interactions between users. The intention was to promote meaningful engagement, but later analyses and reporting suggested that content provoking outrage and division often benefited from the new ranking incentives because it generated strong reactions and discussion. [Knight First Amendment Institute]knightcolumbia.orgIt's the Algorithm: A large-scale comparative…Read more… [panoptykon]panoptykon.orgFixing Recommender SystemsPanoptykonFixing Recommender SystemsAugust 22, 2023 — 25 Aug 2023 — In 2018 Facebook introduced a new metric for its News Feed algorithm…Published: August 22, 2023 This episode illustrates a broader governance lesson: ranking objectives can have consequences that are difficult to predict. A metric designed to measure healthy interaction may end up rewarding controversy if the underlying incentives are not carefully evaluated.

Transparency and accountability trade-offs

Because recommendation systems operate largely behind the scenes, outsiders often struggle to determine why particular content spreads. This opacity creates challenges for researchers, regulators, journalists, and users attempting to understand how myths gain visibility.

Algorithmic amplification is now recognised as a distinct governance issue. The concern is not simply whether harmful content exists, but whether recommendation systems actively increase its visibility beyond what would occur through ordinary user choice. Yet measuring amplification remains difficult because platforms rarely disclose the full details of their ranking systems, and those systems change frequently. EU DisinfoLab [wikipedia]WikipediaAlgorithmic amplificationAlgorithmic amplification is the process by which automated ranking and recommendation systems on digital pla… Transparency advocates argue that platforms should provide clearer information about:

Greater transparency can improve accountability, but it also creates trade-offs. Detailed disclosure may expose systems to manipulation by spammers, propagandists, or coordinated influence campaigns seeking to exploit ranking rules.

Regulatory responses

Governments have increasingly shifted attention from individual pieces of content toward the design of recommender systems themselves. The European Union’s Digital Services Act is one prominent example. It requires large platforms to assess and mitigate systemic risks linked to recommender systems and to provide users with greater transparency about how recommendations operate. The legislation reflects a growing policy view that amplification mechanisms deserve scrutiny alongside moderation policies. [Mozilla Foundation]mozillafoundation.orgMozilla FoundationHow the Digital Services Act Addresses Platform…Feb 27, 2023 — While the DSA pays close attention to the algorithmic… [DSA Observatory]dsa-observatory.euThe Regulation of Recommender Systems Under the DSANov 22, 2024 — The DSA is the first supranational regulation that aims to address the…

This represents an important change in thinking. Rather than treating misinformation solely as a content problem, regulators are increasingly examining the architecture that determines which content receives attention.

The governance challenge: reducing myths without controlling opinion

A common misconception is that limiting the spread of myths requires extensive censorship. Recommendation systems demonstrate that there are other options. Platforms can alter ranking incentives, reduce rewards for outrage, elevate reliable information, and provide users with more control over how recommendations work without necessarily removing large amounts of content.

The challenge is that no ranking system is neutral. Every recommendation algorithm reflects choices about what counts as valuable: engagement, satisfaction, reliability, diversity, safety, or some combination of them. Prioritising one goal inevitably changes who receives attention and which ideas gain reach. [Knight First Amendment Institute]knightcolumbia.orgIt's the Algorithm: A large-scale comparative…Read more… [Academic Commons]academiccommons.columbia.eduAcademic CommonsUnderstanding Social Media Recommendation Algorithmsby A Narayanan · 2023 · Cited by 228 — In computer science, the algor…

For myths and misconceptions, this means the architecture of recommendation often matters as much as moderation itself. A false claim does not become influential merely because it exists. It becomes influential when systems designed to allocate attention repeatedly place it in front of new audiences. Conversely, recommender systems that value reliability and harm reduction can make the same claim far less likely to dominate public discussion. [dl.acm.org]dl.acm.orgmpacted by misinformation.Read more… [Mozilla Foundation]mozillafoundation.orgMozilla FoundationHow the Digital Services Act Addresses Platform…Feb 27, 2023 — While the DSA pays close attention to the algorithmic…

Amazon book picks

Further Reading

Books and field guides related to The Hidden Engine Behind Myth Spread. Use these as the next step if you want deeper reading beyond the article.

Book

The Chaos Machine

By Max Fisher

Directly examines recommendation systems and amplification effects.

See on Amazon

Book

The Age of Surveillance Capitalism

By Shoshana Zuboff

Explores incentives that shape platform design and engagement optimisation.

See on Amazon

Book

Network Propaganda

By Yochai Benkler, Robert Faris et al.

First published 2018. Subjects: Politics & government, Presidents, united states, election, 2016, Communication in politics, Political ca...

See on Amazon

Book

Chaos Machine

By Max Fisher

First published 2022.

See on Amazon

Browse more on Amazon: The Chaos Machine The Age of Surveillance Capitalism Network Propaganda

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Example marketplace items related to this page. Use the search link to explore similar finds on eBay.

Shop location

Using USA USA USAUKCanadaAustraliaIreland

Example eBay listing

Poster: the Mode of Information (Paper) - Poster

Search eBay.co.uk: science communication poster

Browse similar on eBay.co.uk

Example eBay listing

Military Phonetic Alphabet Poster – Tactical Survival Communication Chart

Search eBay.co.uk: science communication poster

Browse similar on eBay.co.uk

Browse more on eBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: pmc.ncbi.nlm.nih.gov
   Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC11894805/
   Source snippet

   PMCby S Milli · 2025 · Cited by 164 — Social media ranking algorithms typically optimize for users' revealed preferences, i.e. user engag...

2. Source: arxiv.org
   Link: https://arxiv.org/abs/2305.16941
   Source snippet

   arXivEngagement, User Satisfaction, and the Amplification of Divisive Content on Social MediaMay 26, 2023...

   Published: May 26, 2023

3. Source: sciencedirect.com
   Link: https://www.sciencedirect.com/science/article/pii/S0047272726000253
   Source snippet

   ScienceDirectRanking for engagement: How social media algorithms fuel...by F Germano · 2026 · Cited by 16 — This paper investigates the...

4. Source: papers.ssrn.com
   Link: https://papers.ssrn.com/sol3/Delivery.cfm/4f79d5ca-867d-49db-a4ba-ebfdcaf0ac9b-MECA.pdf?abstractid=5316506&mirid=1
   Source snippet

   Social Media Algorithms Fuel Misinformation and...To connect this prediction to observational data, we exploit Facebook's “Meaningful So...

5. Source: panoptykon.org
   Title: Fixing Recommender Systems
   Link: https://panoptykon.org/sites/default/files/2023-08/Panoptykon_ICCL_PvsBT_Fixing-recommender-systems_Aug%202023.pdf
   Source snippet

   PanoptykonFixing Recommender SystemsAugust 22, 2023 — 25 Aug 2023 — In 2018 Facebook introduced a new metric for its News Feed algorithm...

   Published: August 22, 2023

6. Source: Wikipedia
   Link: https://en.wikipedia.org/wiki/Algorithmic_amplification
   Source snippet

   Algorithmic amplificationAlgorithmic amplification is the process by which automated ranking and recommendation systems on digital pla...

7. Source: arxiv.org
   Link: https://arxiv.org/html/2505.11577v3
   Source snippet

   How Platform API Restrictions Undermine AI Transparency...Mar 27, 2026 — This study documents an accountability paradox: as platforms em...

8. Source: dsa-observatory.eu
   Link: https://dsa-observatory.eu/2024/11/22/the-regulation-of-recommender-systems-under-the-dsa-a-transition-from-default-to-multiple-and-dynamic-controls/
   Source snippet

   The Regulation of Recommender Systems Under the DSANov 22, 2024 — The DSA is the first supranational regulation that aims to address the...

9. Source: dl.acm.org
   Link: https://dl.acm.org/doi/10.1145/3616088
   Source snippet

   mpacted by misinformation.Read more...

10. Source: sciencedirect.com
    Link: https://www.sciencedirect.com/science/article/pii/S0268401226000381
    Source snippet

    AI-based digital disinformation: A theory-informed and...by BW Wirtz · 2026 — Truth, lies, and automation: How language models could cha...

11. Source: sciencedirect.com
    Link: https://www.sciencedirect.com/org/science/article/pii/S1548367326000062
    Source snippet

    Integrating user behavior, content...Read more...

12. Source: arxiv.org
    Link: https://arxiv.org/html/2305.06125v3
    Source snippet

    sed recommender systems - which form the majority of recommendation engines...Read more...

13. Source: academiccommons.columbia.edu
    Link: https://academiccommons.columbia.edu/doi/10.7916/1h2v-pn50/download
    Source snippet

    Academic CommonsUnderstanding Social Media Recommendation Algorithmsby A Narayanan · 2023 · Cited by 228 — In computer science, the algor...

14. Source: knightcolumbia.org
    Link: https://knightcolumbia.org/research/algorithmic-amplification-and-society
    Source snippet

    It's the Algorithm: A large-scale comparative...Read more...

15. Source: knightcolumbia.org
    Title: understanding social media recommendation algorithms
    Link: https://knightcolumbia.org/content/understanding-social-media-recommendation-algorithms
    Source snippet

    misinformation is also consistent with some research.) 61 61... MSI = Meaningful Social Interactions, Facebook's [engagement metric]({{ 'metric-gap/' | relative_url }}). Read...

16. Source: mozillafoundation.org
    Link: https://www.mozillafoundation.org/en/blog/action-recommended-how-the-digital-services-act-addresses-platform-recommender-systems/
    Source snippet

    Mozilla FoundationHow the Digital Services Act Addresses Platform...Feb 27, 2023 — While the DSA pays close attention to the algorithmic...

Additional References

1. Source: researchgate.net
   Link: https://www.researchgate.net/publication/401405020_Ranking_for_engagement_How_social_media_algorithms_fuel_misinformation_and_polarization
   Source snippet

   Ranking for engagement: How social media algorithms fuel...4 Mar 2026 — varying the weight on engagement metrics can raise platform acti...

2. Source: knightcolumbia.org
   Link: https://knightcolumbia.org/tags/algorithmic-amplification
   Source snippet

   Algorithmic AmplificationEngagement-based algorithms often amplify divisive content and fail to meet user preferences. What's the alterna...

3. Source: isdglobal.org
   Link: https://www.isdglobal.org/our-expertise/tech-accountability-and-safety/recommender-systems-and-algorithms/page/2/
   Source snippet

   Recommender Systems and Algorithms- Page 2 of 5This research project explores the topics and content YouTube's algorithm recommends to yo...

4. Source: disinfo.eu
   Title: mapping algorithmic amplification transparency challenges lessons germany
   Link: https://www.disinfo.eu/publications/mapping-algorithmic-amplification-transparency-challenges-lessons-germany/
   Source snippet

   EU DisinfoLabMapping algorithmic amplification: transparency...Nov 24, 2025 — The report maps how algorithmic amplification operates acr...

5. Source: cambridge.org
   Title: Addressing Misinformation and Disinformationby J Meese · Cited by 1 —
   Link: https://www.cambridge.org/core/elements/addressing-misinformation-and-disinformation/66BE72E9F1FC74DE2CD6286B8383C146
   Source snippet

   system could miss falsehoods and amplify hateful content (see... Post-Truth, [Fake News]({{ 'fake-news/' | relative_url }}) and Democracy: Mapping the Politics of Falsehood...

6. Source: researchgate.net
   Link: https://www.researchgate.net/publication/393407319_Research_on_the_Impact_of_Social_Media_Algorithmic_on_User_Decision-making_Focus_on_Algorithmic_Transparent_and_Ethical_Design
   Source snippet

   These recommendation systems and their users form a feedback loop, wherein the former aims to...Read more...

7. Source: freshfields.com
   Title: dsa decoded 10 algorithmic transparency under the dsa 102mgg8
   Link: https://www.freshfields.com/en/our-thinking/blogs/technology-quotient/dsa-decoded-10-algorithmic-transparency-under-the-dsa-102mgg8
   Source snippet

   DSA decoded # 10: Algorithmic transparency under the DSAFeb 6, 2026 — Most online platforms use recommender systems to enhance the user e...

8. Source: ifo.de
   Link: https://www.ifo.de/en/cesifo/publications/2026/working-paper/ranking-engagement-how-social-media-algorithms-fuel-misinformation
   Source snippet

   s and user behavior, and develops a theoretical framework to assess the impact...Read more...

9. Source: aph.gov.au
   Link: [https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Information_Integrity_on_Climate_Change_and_Energy/ClimateIntegrity/Report/Chapter_7-_Improving_digital_platform_transparency_and_accountability](https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Information_Integrity_on_Climate_Change_and_Energy/ClimateIntegrity/Report/Chapter_7-_Improving_digital_platform_transparency_and_accountability)
   Source snippet

   transparency, as well as specific actions in relation to bots and inauthentic...Read more...

10. Source: ifo.de
    Title: Institut CESifo Working Paper No
    Link: https://www.ifo.de/DocDL/cesifo1_wp10011.pdf
    Source snippet

    10011by F Germano · Cited by 16 — Finally, empirical evidence from survey data in Italy and the United States indicates that Facebook's 2...