Within Anecdotes
When a handful of cases looks like proof
A few memorable examples can look like a rule when small samples are expected to represent the wider world too neatly.
On this page
- Why small samples feel more stable than they are
- Selection bias and the cases nobody mentions
- How larger samples change the conclusion
Page outline Jump by section
Introduction
Many misconceptions begin with a handful of convincing examples. A person meets three rude tourists from the same country, knows two entrepreneurs who became wealthy without university degrees, or hears four stories about a medical treatment that seemed to work. The examples are real, but the conclusion drawn from them is often not. The mistake is assuming that a small sample behaves like the wider population.
Psychologists Daniel Kahneman and Amos Tversky described this tendency as a belief in the “law of small numbers”: the expectation that even a small sample will closely resemble the larger group from which it comes. In reality, small samples are unstable. They produce extreme results, unusual clusters and apparent patterns far more often than people intuitively expect. [ScienceDirect]sciencedirect.comScienceDirectSubjective probability: A judgment of representativenessby D Kahneman · 1972 · Cited by 9598 — In particular, since sample s…
Within the broader problem of overgeneralising from personal experience, this mechanism helps explain why a few memorable cases can seem to prove a rule that disappears when more evidence is collected.
Why small samples feel more stable than they are
The illusion begins with a reasonable intuition. If a sample comes from a larger population, people expect it to look like that population. The problem is that they often expect this resemblance to appear immediately, even when only a few observations are available.
Research on judgement under uncertainty found that people frequently neglect sample size when estimating probabilities and drawing conclusions. Small groups are treated as though they should mirror the broader population, despite the fact that random variation is much larger in small samples. [ScienceDirect]sciencedirect.comScienceDirectSubjective probability: A judgment of representativenessby D Kahneman · 1972 · Cited by 9598 — In particular, since sample s… [Sage Journals]journals.sagepub.comSage JournalsRevisiting representativeness heuristic classic paradigmsKT proposed that participants neglect sample size when determining…
Consider a simple example. If a fair coin is flipped ten times, a run of seven heads is unusual but entirely possible. If it is flipped ten thousand times, the overall proportion of heads will be much closer to fifty per cent. Small samples naturally produce more dramatic swings than large ones. Yet people often treat those swings as meaningful signals rather than ordinary variation. [Wikipedia]WikipediaInsensitivity to sample sizeInsensitivity to sample size is a cognitive bias where people estimate the probability of obtaining a samp…
This misunderstanding creates an illusion of order. A few observations line up in a way that feels too neat to be accidental, and the mind quickly converts the coincidence into a pattern.
The historical idea of the “law of small numbers”
Kahneman and Tversky used the phrase “law of small numbers” partly as an ironic contrast to the genuine law of large numbers in probability theory. The real law of large numbers states that averages become more reliable as observations accumulate. The imagined law of small numbers assumes the same reliability exists even when evidence is scarce. [effectiviology.com]effectiviology.comepresentative of the populations they come from (similarly to…Read more… Wikipedia Their work showed that people often believe small samples should be highly representative [Wikipedia]WikipediaInsensitivity to sample sizeInsensitivity to sample size is a cognitive bias where people estimate the probability of obtaining a samp…, leading them to overinterpret limited evidence and underestimate the role of chance. Cambridge University Press & Assessment [ScienceDirect]sciencedirect.comScienceDirectSubjective probability: A judgment of representativenessby D Kahneman · 1972 · Cited by 9598 — In particular, since sample s…
Selection bias and the cases nobody mentions
Small samples become even more misleading when they are not randomly selected.
In everyday life, people rarely encounter a neutral collection of cases. They encounter memorable ones. Successful investors write books. Patients who believe a treatment changed their lives tell their stories. Exceptional experiences spread through conversation and social media. Ordinary experiences often remain invisible.
This creates selection bias: the sample that reaches public attention differs from the larger population. A few visible examples can therefore seem representative even when they are not. [arXiv]arxiv.orgarXiv A Bayesian algorithm for sample selection bias correctionarXivA Bayesian algorithm for sample selection bias correctionDecember 19, 2022…
Imagine hearing five stories from people who followed a particular diet and lost substantial weight. The stories may be genuine. What is missing are the hundreds or thousands of people who tried the same approach without unusual results and never became part of the conversation. The visible sample creates a pattern because the invisible cases have been filtered out.
This is one reason myths often persist. People remember striking successes, failures and coincidences while overlooking the silent majority of uneventful outcomes.
Clusters look meaningful even when they are random
Another consequence of small samples is the tendency to see significance in clusters.
Random events do not spread themselves evenly. They naturally produce streaks, bunching and concentrations. Yet many people expect randomness to look orderly. When they encounter a cluster, they assume some hidden cause must exist. [Conversion Uplift]conversion-uplift.co.ukConversion UpliftClustering IllusionDefinition. The clustering illusion (or the Law of small numbers) is the tendency to perceive pattern…
Historically, this has fuelled beliefs ranging from lucky locations and cursed places to supposed demographic trends inferred from a handful of observations. A cluster of cases may be real, but a cluster alone does not prove a mechanism. Random processes often generate patterns that look intentional.
The key misunderstanding is not that people see a pattern where none exists; it is that they underestimate how often random variation creates patterns on its own.
How larger samples change the conclusion
Many apparent rules weaken or disappear when more evidence is gathered.
A small town may appear to have an unusually high rate of a particular outcome. A few early customers may seem to demonstrate that a product is exceptionally good or exceptionally bad. An investor may appear brilliant after a short run of successful trades. As the sample grows, many of these striking differences shrink toward the broader average. [Investopedia]investopedia.comInvestopediaUnderstanding the Law of Large Numbers: Key Concepts…The law of large numbers states that the larger a sample size is, the… [PMC]pmc.ncbi.nlm.nih.govPMCby ID Dinov · 2009 · Cited by 47 — The LLN implies that if we were to increase the number of experiments (N), say from 10 to 100, and…
This does not mean small samples are useless. They can reveal possibilities, generate hypotheses and identify questions worth investigating. The problem arises when they are treated as proof.
Larger samples help in three important ways:
- They reduce the influence of random fluctuations.
- They make unusual observations less dominant.
- They provide a better estimate of what is typical rather than exceptional. [Investopedia]investopedia.comSample size neglectSmaller sample sizes have more volatile and less reliable…Read more… [Wikipedia]WikipediaLaw of large numbersLaw of large numbers
The difference can be dramatic. A school with twenty pupils may swing from outstanding results to poor results from year to year simply because a few students have an unusually strong or weak cohort. A school with several thousand pupils is generally more stable because individual variation has less impact on the overall picture. This principle applies across medicine, economics, sports, polling and everyday judgement.
Why the misconception survives
The illusion of a pattern is difficult to overcome because small samples often produce compelling stories.
A vivid set of examples feels concrete. A large dataset feels abstract. Human judgement evolved to learn from direct experience, not to perform statistical analysis. As a result, a handful of memorable cases can seem more persuasive than a much larger body of less dramatic evidence. [The Decision Lab]thedecisionlab.comThe Decision LabRepresentativeness HeuristicThe representativeness heuristic is a mental shortcut that we use when estimating probabiliti… [Verywell Mind]verywellmind.comrepresentativeness heuristic 2795805How the Representativeness Heuristic Affects Decisions…6 Apr 2026 — The representativeness heuristic involves estimating the likelihoo…
The representativeness heuristic strengthens this effect. People judge how likely something is by how much it resembles a familiar pattern or stereotype rather than by considering how much evidence is available. When a small sample appears representative, it feels convincing even when statistically it is not. [Sage Journals]journals.sagepub.comSage JournalsRevisiting representativeness heuristic classic paradigmsKT proposed that participants neglect sample size when determining… [The Decision Lab]thedecisionlab.comThe Decision LabRepresentativeness HeuristicThe representativeness heuristic is a mental shortcut that we use when estimating probabiliti…
The result is a recurring source of myths and misconceptions: a few cases are mistaken for a rule, a cluster is mistaken for a cause, and limited experience is mistaken for broad knowledge.
Recognising the instability of small samples does not require abandoning personal experience. It requires placing that experience in context. A handful of observations can suggest a possibility, but they rarely tell the whole story. The larger and more representative the sample becomes, the more likely it is that the apparent pattern reflects reality rather than chance. [effectiviology.com]effectiviology.comepresentative of the populations they come from (similarly to…Read more… [PMC]pmc.ncbi.nlm.nih.govPMCCognitive biases resulting from the representativenessPMCby M AlKhars · 2019 · Cited by 63 — This paper aims to investigate the six cognitive biases resulting from the use of the representati…
Amazon book picks
Further Reading
Books and field guides related to When a handful of cases looks like proof. Use these as the next step if you want deeper reading beyond the article.
The Drunkard's Walk
First published 2008. Subjects: Probabilities, Chance, Random variables, Mathematics, Nonfiction.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: sciencedirect.com
Link: https://www.sciencedirect.com/science/article/pii/0010028572900163Source snippet
ScienceDirectSubjective probability: A judgment of representativenessby D Kahneman · 1972 · Cited by 9598 — In particular, since sample s...
-
Source: cambridge.org
Title: Only 9 of our 84 respondents gave answers between
Link: https://www.cambridge.org/core/books/judgment-under-uncertainty/belief-in-the-law-of-small-numbers/BC22F08226B4BC58280BCCBA5DC9C312Source snippet
Cambridge University Press & AssessmentBelief in the law of small numbers (Chapter 2)May 5, 2013 — On the other hand, if you feel that th...
Published: May 5, 2013
-
Source: Wikipedia
Link: https://en.wikipedia.org/wiki/Insensitivity_to_sample_sizeSource snippet
Insensitivity to sample sizeInsensitivity to sample size is a cognitive bias where people estimate the probability of obtaining a samp...
-
Source: Wikipedia
Title: Law of large numbers
Link: https://en.wikipedia.org/wiki/Law_of_large_numbers -
Source: effectiviology.com
Link: https://effectiviology.com/law-of-small-numbers/Source snippet
epresentative of the populations they come from (similarly to...Read more...
-
Source: investopedia.com
Link: https://www.investopedia.com/terms/l/lawoflargenumbers.aspSource snippet
InvestopediaUnderstanding the Law of Large Numbers: Key Concepts...The law of large numbers states that the larger a sample size is, the...
-
Source: arxiv.org
Title: arXiv A Bayesian algorithm for sample selection bias [correction]({{ ‘correction/’ | relative_url }})
Link: https://arxiv.org/abs/2212.09813Source snippet
arXivA Bayesian algorithm for sample selection bias correctionDecember 19, 2022...
Published: December 19, 2022
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC3095954/Source snippet
PMCby ID Dinov · 2009 · Cited by 47 — The LLN implies that if we were to increase the number of experiments (N), say from 10 to 100, and...
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCCognitive biases resulting from the representativeness
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC6462158/Source snippet
PMCby M AlKhars · 2019 · Cited by 63 — This paper aims to investigate the six cognitive biases resulting from the use of the representati...
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCCan we shift belief in the ‘Law of Small Numbers’?
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8889191/Source snippet
PMCby DVM Bishop · 2022 · Cited by 8 — 'Sample size neglect' is a tendency to underestimate how the variability of mean estimates changes...
-
Source: investopedia.com
Title: Sample size neglect
Link: https://www.investopedia.com/terms/s/sample-size-neglect.aspSource snippet
Smaller sample sizes have more volatile and less reliable...Read more...
-
Source: resolve.cambridge.org
Link: https://resolve.cambridge.org/core/services/aop-cambridge-core/content/view/BC22F08226B4BC58280BCCBA5DC9C312/9780511809477c2_p23-31_CBO.pdf/belief_in_the_law_of_small_numbers.pdfSource snippet
Belief in the law of small numbers Amos Tversky and..."Suppose you have run an experiment on 20 subjects, and have obtained a significan...
-
Source: journals.sagepub.com
Link: https://journals.sagepub.com/doi/10.1177/17470218241255916Source snippet
Sage JournalsRevisiting representativeness heuristic classic paradigmsKT proposed that participants neglect sample size when determining...
-
Source: conversion-uplift.co.uk
Link: https://conversion-uplift.co.uk/glossary-of-conversion-marketing/clustering-illusion/Source snippet
Conversion UpliftClustering IllusionDefinition. The clustering illusion (or the Law of small numbers) is the tendency to perceive pattern...
-
Source: datacamp.com
Title: law of large numbers
Link: https://www.datacamp.com/tutorial/law-of-large-numbersSource snippet
Definition, Examples, and Use Cases5 Jun 2025 — The law of large numbers is a statistical theorem stating that as the sample size grows...
-
Source: thedecisionlab.com
Link: https://thedecisionlab.com/biases/representativeness-heuristicSource snippet
The Decision LabRepresentativeness HeuristicThe representativeness heuristic is a mental shortcut that we use when estimating probabiliti...
-
Source: verywellmind.com
Title: representativeness heuristic 2795805
Link: https://www.verywellmind.com/representativeness-heuristic-2795805Source snippet
How the Representativeness Heuristic Affects Decisions...6 Apr 2026 — The representativeness heuristic involves estimating the likelihoo...
-
Source: scribbr.co.uk
Title: Scribbr Representativeness Heuristic | Example & Definition
Link: https://www.scribbr.co.uk/bias-in-research/the-representativeness-heuristic/Source snippet
Representativeness Heuristic | Example & Definition - Scribbr28 Dec 2022 — The representativeness heuristic occurs when we estimate the p...
-
Source: conversion-uplift.co.uk
Link: https://conversion-uplift.co.uk/post/myths-and-the-law-of-small-numbers/Source snippet
over doubt.Read more...
-
Source: blog.udemy.com
Title: representativeness heuristic
Link: https://blog.udemy.com/representativeness-heuristic/Source snippet
Heuristic: Understanding Decision...16 Feb 2020 — The representativeness heuristic is a mental shortcut where people judge situations by...
Additional References
-
Source: probabilitycourse.com
Link: https://www.probabilitycourse.com/chapter7/7_1_1_law_of_large_numbers.phpSource snippet
7.1.1 Law of Large NumbersIt states that if you repeat an experiment independently a large number of times and average the result, what y...
-
Source: fs.blog
Link: https://fs.blog/mental-model-bias-from-insensitivity-to-sample-size/ -
Source: aaea.org
Link: https://www.aaea.org/UserFiles/file/SessionII-Coble.pdf -
Source: quizlet.com
Title: tversky a kahneman d 1974 ce642ded 2f2d 41ff bcd3 0b12f0df6fe4
Link: https://quizlet.com/study-guides/tversky-a-kahneman-d-1974-ce642ded-2f2d-41ff-bcd3-0b12f0df6fe4Source snippet
Tversky, A., & Kahneman, D. (1974) Study Guide7 Oct 2025 — A study revealed that even experienced researchers hold onto the 'law of small...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/332324419_Cognitive_biases_resulting_from_the_representativeness_heuristic_in_operations_management_an_experimental_investigationSource snippet
of the representativeness heuristic, namely, insensitivity to prior probability...Read more...
-
Source: marketingjournal.org
Title: law of small numbers redux mark blessington karl hellman
Link: https://www.marketingjournal.org/law-of-small-numbers-redux-mark-blessington-karl-hellman/Source snippet
“Law of Small Numbers Redux” – Mark Blessington & Karl...2 Jan 2017 — Their research indicated that scientists were strongly inclined to...
-
Source: datamantra.medium.com
Title: law of large numbers statistical stability 22bf0f66823b
Link: https://datamantra.medium.com/law-of-large-numbers-statistical-stability-22bf0f66823bSource snippet
of Large Numbers — Statistical StabilityThe law of large numbers is a fundamental principle in statistics that describes the behaviour of...
-
Source: onlinelibrary.wiley.com
Link: https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291099-0771%28200001/03%2913%3A1%3C125%3A%3AAID-BDM346%3E3.0.CO%3B2-1Source snippet
Bernoulli might have been wrong: a comment on...by G Keren · 2000 · Cited by 9 — The size of a sample is an essential concept of inferen...
-
Source: researchgate.net
Title: The Role of Sample Size in Sample Evaluation D
Link: https://www.researchgate.net/publication/232338408_The_Role_of_Sample_Size_in_Sample_EvaluationSource snippet
Kahneman and A. Tversky (Cognitive Psychology, 1972, 3, 430–454) claimed that “the notion that sampling variance decreases in proportion...
-
Source: medium.com
Link: https://medium.com/%40bpn.bruno/the-dueling-laws-of-large-and-small-numbers-5-40da8c3ec54cSource snippet
bilities, necessarily represents the probabilities of...Read more...
Topic Tree