When evidence based becomes a sales phrase

Introduction

“Evidence-based” is one of the most persuasive phrases in education, training, coaching and professional development. Used carefully, it signals that decisions are informed by research rather than intuition alone. Used loosely, it can become part of the marketing. A common misconception is that a programme is proven simply because its website cites studies about feedback, motivation, memory, practice or other broad learning principles. In reality, evidence supporting a general mechanism is not the same as evidence showing that a specific product, training package or intervention produces the outcomes it claims. The distinction matters because buyers often invest money, time and organisational trust in programmes that have borrowed the authority of research without demonstrating their own effectiveness. [media.nesta.org.uk]media.nesta.org.ukWhat counts as good evidence?February 22, 2013 — by SM Nutley · 2013 · Cited by 306 — Research–based – the programme or practice is based…Published: February 22, 2013

Why broad research does not prove a product

A programme may be built around an idea that has substantial research support. For example, there is strong evidence that feedback can improve learning under the right conditions, and there is extensive research on retrieval practice, spaced learning and other instructional approaches. Yet evidence for a principle does not automatically transfer to every branded system that claims to apply it. [EEF]educationendowmentfoundation.org.ukeef blog using research evidence navigating the maze of evidence claimsUsing Research Evidence — navigating the maze of… | EEF11 Jan 2024 — The EEF's concise guide on using research evidence includes informat…

The key question is whether the programme itself has been tested. A company might cite dozens of studies showing that feedback matters, but those studies do not prove that its training package, software platform or consultancy process improves outcomes. Between a general principle and a commercial product lie many design choices: implementation methods, user training, context, dosage, measurement and fidelity. Any of these can influence whether a programme succeeds, fails or produces no meaningful effect. [EEF]educationendowmentfoundation.org.ukEEFTeacher Feedback to Improve Pupil Learning11 Jun 2021 — This guidance report focuses on teacher-delivered feedback and is relevant to…

This is sometimes called citation borrowing. Research findings about one thing are used to imply support for another. The cited studies may be accurate and relevant at a broad level, but they do not necessarily validate the specific intervention being sold.

A useful comparison is medicine. Research showing that exercise improves health does not prove that a particular branded fitness programme is effective. The programme still requires evaluation. The same logic applies to educational and professional interventions.

What fair programme evidence should compare

When organisations evaluate a programme properly, they do more than demonstrate that participants enjoyed it or felt motivated by it. They ask whether the programme performs better than realistic alternatives.

Independent evidence frameworks increasingly distinguish between a programme that is merely research-based and one that is evidence-based. In these frameworks, a research-based approach is grounded in theory and prior findings, whereas an evidence-based programme has been rigorously evaluated and shown to produce consistent benefits. [media.nesta.org.uk]media.nesta.org.ukWhat counts as good evidence?February 22, 2013 — by SM Nutley · 2013 · Cited by 306 — Research–based – the programme or practice is based…Published: February 22, 2013

Good programme evidence usually includes several features:

• A clearly defined intervention rather than a vague collection of ideas.
• A comparison group, allowing evaluators to estimate what would have happened without the programme.
• Relevant outcome measures, such as learning, performance, behaviour or attainment rather than simple satisfaction ratings.
• Replication, showing that results are not dependent on a single setting or unusually favourable conditions.
• Transparency, including publication of methods, limitations and findings. [ies.ed.gov]ies.ed.govWW C | Find What Works!WWC | Find What Works! - Institute of Education SciencesIntervention Reports: summaries of findings of the highest-quality research on a… [Sage Journals]journals.sagepub.comSage JournalsHow Consistent Are Meanings of “Evidence-Based”? A…21 Feb 2023 — Clearinghouses set standards of scientific quality to ve…

This is why organisations such as the Education Endowment Foundation (EEF) and the What Works Clearinghouse place substantial emphasis on evaluation quality. Their focus is not simply whether an intervention sounds plausible but whether credible evidence indicates that it improves outcomes in practice. [ies.ed.gov]ies.ed.govWW C | Find What Works!WWC | Find What Works! - Institute of Education SciencesIntervention Reports: summaries of findings of the highest-quality research on a… [EEF]educationendowmentfoundation.org.ukTeaching and Learning ToolkitThe EEF is an independent charity dedicated to breaking the link between family income and educational achie…

Plausible is not the same as effective

One reason citation borrowing is persuasive is that many programmes are built around ideas that genuinely sound sensible.

The learning-styles movement illustrates the problem. The underlying observation that people have preferences is uncontroversial. However, the stronger claim—that learners should be classified and taught according to their preferred style—requires direct evidence. Major reviews found that the evidence needed to support this matching approach was lacking, despite widespread promotion and commercial adoption. Many learning-styles products therefore rested on a plausible-sounding mechanism rather than demonstrated programme effectiveness. [PubMed]pubmed.ncbi.nlm.nih.govPubMedLearning Styles: Concepts and Evidenceby H Pashler · 2008 · Cited by 4949 — Learning styles refers to the concept that individuals… [PMC]pmc.ncbi.nlm.nih.govUsing these standards, some clearinghouses rate interventions as “evidence-based”…Read more…

The lesson extends beyond learning styles. Any programme can cite valid research while still lacking proof that its own implementation works.

How buyers can spot citation borrowing

Governance decisions become easier when buyers learn to separate evidence about a principle from evidence about a programme.

Several warning signs are worth examining.

Research is cited, but the programme is not evaluated

A provider may present a long bibliography yet offer no independent studies of the actual intervention. The references discuss broad topics such as motivation, neuroscience, feedback or behaviour change, but none directly test the programme being sold.

In such cases, the research may support the theory behind the programme without supporting the programme itself. [media.nesta.org.uk]media.nesta.org.ukWhat counts as good evidence?February 22, 2013 — by SM Nutley · 2013 · Cited by 306 — Research–based – the programme or practice is based…Published: February 22, 2013

Testimonials replace outcome data

Case studies and testimonials can be useful sources of practical insight, but they cannot establish causation. Positive experiences may reflect enthusiasm, selection effects, novelty or other influences unrelated to programme effectiveness.

Evidence claims become stronger when accompanied by independently collected outcome data rather than anecdotes alone. [eppi.ioe.ac.uk]eppi.ioe.ac.ukCFHI EVIDENCE STANDARDS REPORT V15 PRINTThe research on the…Read more…

The phrase “evidence-based” is undefined

Different organisations use the term differently. Research on evidence standards has shown substantial variation in how clearinghouses and evaluators classify interventions as evidence-based. A credible provider should be able to explain exactly what evidence standard is being used and what studies support the claim. [Sage Journals]journals.sagepub.comSage JournalsHow Consistent Are Meanings of “Evidence-Based”? A…21 Feb 2023 — Clearinghouses set standards of scientific quality to ve…

Evidence of implementation is confused with evidence of impact

A programme may demonstrate that participants completed training, used materials correctly or liked the process. These findings can be valuable, but they do not automatically show improved outcomes.

Implementation evidence answers the question, “Was the programme delivered as intended?” Impact evidence answers the question, “Did it make a meaningful difference?” Both matter, but they are not interchangeable. [EEF]educationendowmentfoundation.org.ukare designed to give you practical guidance and…Read more…

A better way to use evidence-based claims

The phrase “evidence-based” becomes most useful when treated as the start of a conversation rather than the end of one.

A responsible buyer asks three separate questions:

These questions recognise that good ideas, effective implementation and demonstrated impact are related but distinct. Research can justify trying an approach. Only programme-level evidence can justify confidence that the specific product delivers what its marketing suggests.

In governance terms, the goal is not to reject evidence-informed innovation. It is to avoid confusing the authority of research with proof of effectiveness. A programme may be inspired by evidence, informed by evidence or compatible with evidence. Those are worthwhile attributes. They are not the same thing as showing that the programme itself has been tested and shown to work. [media.nesta.org.uk]media.nesta.org.ukWhat counts as good evidence?February 22, 2013 — by SM Nutley · 2013 · Cited by 306 — Research–based – the programme or practice is based…Published: February 22, 2013 [EEF]educationendowmentfoundation.org.uknew guide to help education professionals to make best use of research evidenceNew guide to help education professionals to make best…11 Jan 2024 — A new guide, published by the Education Endowment Foundation (EEF…

Amazon book picks

Further Reading

Books and field guides related to When evidence based becomes a sales phrase. Use these as the next step if you want deeper reading beyond the article.

Book

Bad Science

By Ben Goldacre

Shows how evidence can be misused in marketing.

See on Amazon

Book

Calling Bullshit

By Carl T. Bergstrom, Jevin D. West

Teaches readers to spot unsupported evidence claims.

See on Amazon

Book

The Skeptics' Guide to the Universe

By Dr. Steven Novella

Provides frameworks for evaluating extraordinary claims.

See on Amazon

Book

Thinking, Fast and Slow

By Daniel Kahneman

Explains why people accept evidence-sounding arguments too readily.

See on Amazon

Browse more on Amazon: Bad Science Calling Bullshit The Skeptics' to the Universe books

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Example marketplace items related to this page. Use the search link to explore similar finds on eBay.

Shop location

Using USA USA USAUKCanadaAustraliaIreland

Example eBay listing

16 Biology Posters for Classroom - 11x17in Matte-Laminated Educational Science D

Search eBay.co.uk: science classroom poster

Browse similar on eBay.co.uk

Example eBay listing

jackinthebox 12 Science Posters for Classroom - 12 x 17 inches Laminated

Search eBay.co.uk: science classroom poster

Browse similar on eBay.co.uk

Example eBay listing

Large Science Posters for Classrooms - Science Classroom 10pcs - 11x14in

Search eBay.co.uk: science classroom poster

Browse similar on eBay.co.uk

Example eBay listing

Mathematics Formula Chalkboard Poster Science Classroom Wall Art

Search eBay.co.uk: science classroom poster

Browse similar on eBay.co.uk

Browse more on eBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: media.nesta.org.uk
   Link: https://media.nesta.org.uk/documents/What-Counts-as-Good-Evidence-WEB.pdf
   Source snippet

   What counts as good evidence?February 22, 2013 — by SM Nutley · 2013 · Cited by 306 — Research–based – the programme or practice is based...

   Published: February 22, 2013

2. Source: educationendowmentfoundation.org.uk
   Title: eef blog using research evidence navigating the maze of evidence claims
   Link: https://educationendowmentfoundation.org.uk/news/eef-blog-using-research-evidence-navigating-the-maze-of-evidence-claims
   Source snippet

   Using Research Evidence — navigating the maze of… | EEF11 Jan 2024 — The EEF's concise guide on using research evidence includes informat...

3. Source: educationendowmentfoundation.org.uk
   Link: https://educationendowmentfoundation.org.uk/education-evidence/guidance-reports/feedback
   Source snippet

   EEFTeacher Feedback to Improve Pupil Learning11 Jun 2021 — This guidance report focuses on teacher-delivered feedback and is relevant to...

4. Source: educationendowmentfoundation.org.uk
   Link: https://educationendowmentfoundation.org.uk/education-evidence/teaching-learning-toolkit
   Source snippet

   Teaching and Learning ToolkitThe EEF is an independent charity dedicated to breaking the link between family income and educational achie...

5. Source: eppi.ioe.ac.uk
   Title: CFHI EVIDENCE STANDARDS REPORT V15 PRINT
   Link: https://eppi.ioe.ac.uk/cms/Portals/0/PDF%20reviews%20and%20summaries/CFHI_EVIDENCE_STANDARDS_REPORT_V15_PRINT.pdf
   Source snippet

   The research on the...Read more...

6. Source: pmc.ncbi.nlm.nih.gov
   Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC9283145/
   Source snippet

   Using these standards, some clearinghouses rate interventions as “evidence-based”...Read more...

7. Source: ies.ed.gov
   Title: WW C | Find What Works!
   Link: https://ies.ed.gov/ncee/wwc/
   Source snippet

   WWC | Find What Works! - Institute of Education SciencesIntervention Reports: summaries of findings of the highest-quality research on a...

8. Source: educationendowmentfoundation.org.uk
   Link: https://educationendowmentfoundation.org.uk/education-evidence
   Source snippet

   are designed to give you practical guidance and...Read more...

9. Source: pmc.ncbi.nlm.nih.gov
   Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC5366351/
   Source snippet

   evidence has been found (Pashler et al., 2008). The empirical evidence is clear that there is currently no evidence to support the use of...

10. Source: educationendowmentfoundation.org.uk
    Title: new guide to help education professionals to make best use of research evidence
    Link: https://educationendowmentfoundation.org.uk/news/new-guide-to-help-education-professionals-to-make-best-use-of-research-evidence
    Source snippet

    New guide to help education professionals to make best...11 Jan 2024 — A new guide, published by the Education Endowment Foundation (EEF...

11. Source: educationendowmentfoundation.org.uk
    Link: https://educationendowmentfoundation.org.uk/education-evidence/more-resources-and-support/using-research-evidence
    Source snippet

    Using research evidenceUsing research evidence well can therefore support teachers and leaders to refine their practice or make strategic...

12. Source: schools.oxfordshire.gov.uk
    Link: [https://schools.oxfordshire.gov.uk/sites/default/files/2023-09/EEF_Guide_for_School_Governors_and_Trustees_2019-_print_version.pdf](https://schools.oxfordshire.gov.uk/sites/default/files/2023-09/EEF_Guide_for_School_Governors_and_Trustees_2019-_print_version.pdf)
    Source snippet

    EEF guide to becoming an evidence-informed school...Many of these have been developed by EEF Research Schools, a national network of sch...

13. Source: assets.publishing.service.gov.uk
    Title: Evidence informed teaching an evaluation of progress in England
    Link: [https://assets.publishing.service.gov.uk/media/5a749aca40f0b61938c7ece0/Evidence-informed_teaching_-an_evaluation_of_progress_in_England.pdf](https://assets.publishing.service.gov.uk/media/5a749aca40f0b61938c7ece0/Evidence-informed_teaching-_an_evaluation_of_progress_in_England.pdf)
    Source snippet

    publishing.service.gov.ukEvidence-informed teaching: an evaluation of progress in...by M Coldwell · 2017 · Cited by 198 — In August 2014...

    Published: August 2014

14. Source: educationendowmentfoundation.org.uk
    Link: https://educationendowmentfoundation.org.uk/education-evidence/guidance-reports/implementation
    Source snippet

    EEFA School's Guide to Implementation24 Apr 2024 — Our guidance on effective implementation - and its accompanying resources – help educa...

15. Source: journals.sagepub.com
    Link: https://journals.sagepub.com/doi/10.3102/00346543231152262
    Source snippet

    Sage JournalsHow Consistent Are Meanings of “Evidence-Based”? A...21 Feb 2023 — Clearinghouses set standards of scientific quality to ve...

16. Source: pubmed.ncbi.nlm.nih.gov
    Link: https://pubmed.ncbi.nlm.nih.gov/26162104/
    Source snippet

    PubMedLearning Styles: Concepts and Evidenceby H Pashler · 2008 · Cited by 4949 — Learning styles refers to the concept that individuals...

Additional References

1. Source: dera.ioe.ac.uk
   Link: https://dera.ioe.ac.uk/id/eprint/31088/1/EEF-Implementation-Guidance-Report.pdf
   Source snippet

   Digital Education Resource ArchiveEEF-Implementation-Guidance-Report.pdfThe guide can be used to help implement any school improvement de...

2. Source: gc-bs.org
   Link: [https://gc-bs.org/articles/from-styles-to-science-debunking
   Source snippet

   Debunking the Learning Styles Myth and Embracing...12 Jan 2026 — In 2008, a team of prominent cognitive psychologists led by Harold Pash...

3. Source: my.chartered.college
   Link: https://my.chartered.college/impact_article/the-problem-with-learning-styles-debunking-the-meshing-hypothesis-in-english-language-teaching/
   Source snippet

   problem with learning styles: debunking the meshing...22 Feb 2018 — The idea that catering to a learner's favoured sensory preference, o...

4. Source: educationendowmentfoundation.org.uk
   Link: https://educationendowmentfoundation.org.uk/news/government-confirms-funding-for-eef-to-support-evidence-informed-practice-in-the-post-16-space
   Source snippet

   Government confirms funding for EEF to support…4 Oct 2023 — The Government announced that the Education Endowment Foundation (EEF) would...

5. Source: educationendowmentfoundation.org.uk
   Link: https://educationendowmentfoundation.org.uk/news/new-report-finds-pilot-to-support-evidence-informed-school-improvement-shows-promise
   Source snippet

   New report finds pilot to support evidence-informed school…7 Dec 2023 — - Many schools made progress in aligning their use of teaching as...

6. Source: educationnext.org
   Link: https://www.educationnext.org/stubborn-myth-learning-styles-state-teacher-license-prep-materials-debunked-theory/
   Source snippet

   The Stubborn Myth of “Learning Styles”7 Apr 2020 — The idea of “learning styles” is persistent and popular in the field, in part because...

7. Source: educationendowmentfoundation.org.uk
   Link: https://educationendowmentfoundation.org.uk/news/eef-blog-the-eefs-new-and-updated-guide-to-effective-implementation-whats-changed
   Source snippet

   EEF blog: The EEF's new and updated guide to effective…24 Apr 2024 — Since it was launched in 2018, our guidance report on effective impl...

8. Source: educationendowmentfoundation.org.uk
   Title: new eef report 6 recommendations for improving behaviour in schools
   Link: https://educationendowmentfoundation.org.uk/news/new-eef-report-6-recommendations-for-improving-behaviour-in-schools
   Source snippet

   Improving behaviour in schools: 6 recommendations...6 Jun 2019 — Today's guidance report reviews the best available evidence to offer sc...

9. Source: lrfoundation.org.uk
   Title: final evidence review and synthesis methods guide oct 25
   Link: https://www.lrfoundation.org.uk/sites/default/files/2025-10/final-evidence-review-and-synthesis-methods-guide-oct-25.pdf
   Source snippet

   EVIDENCE REVIEW AND SYNTHESIS METHODS GUIDEby IA Scherer · 2025 — It helps reviewers to identify, extract, synthesise, and report evidenc...

10. Source: journals.lwbooks.co.uk
    Link: https://journals.lwbooks.co.uk/forum/vol-58-issue-2/article-6288/
    Source snippet

    At surface value, there may appear much to commend such an approach.Read more...