“Backed by Science”: The Three Words That Mean Almost Nothing
A consumer’s guide to what scientific evidence actually proves
“Backed by science” appears on thousands of products, websites, and practitioner recommendations. It is also entirely useless, it tells you nothing about the specific findings to justify the claims made.
A $69 Billion Market
The US dietary supplement market reached $69 billion in 2024 and is growing at approximately 7–8% annually, with projections toward $130 billion by 2033 (Nutrition Business Journal, via Nutraceuticals World, April 2025). The broader US wellness economy is valued at $2 trillion (Global Wellness Institute, March 2025).
The Federal Trade Commission has primary jurisdiction over supplement advertising claims. Over the past decade, the FTC brought approximately 120 enforcement actions challenging health claims for dietary supplements while explicitly acknowledging it lacks the staffing to police a $70 billion industry (FTC Health Claims page; FTC Health Products Compliance Guidance, 2022). Enforcement activity has measurably declined since 2025 under the current administration (KFF Health News, August 2025).
A lawsuit currently pending (Xlear v. FTC, filed 2025) seeks to shift the burden of proof for supplement health claims, moving the obligation from manufacturers proving their claims, to the government proving claims are false instead (KFF Health News, August 2025; PolitiFact, August 2025). If successful, the framework that produced what you are about to read about Prevagen would be substantially dismantled.
The Four Levels of Evidence
Level 1: Cell Studies
A cell study exposes isolated human or animal cells in a laboratory dish to a compound and observes what happens. These studies establish whether a biological interaction is chemically possible under controlled laboratory conditions. They cannot establish whether that interaction occurs in a living human body at concentrations a product can actually deliver.
When a product claims it “supports cellular repair” and cites cell study data, you are reading a Level 1 finding dressed as a clinical outcome.
Level 2: Animal Studies
Animal studies establish biological plausibility in intact living organisms. They move beyond the dish into a living system, but cannot establish human efficacy. The majority of compounds showing promise in animal models fail in human trials, a pattern documented across decades of drug development (Hay M et al., Nature Biotechnology, 2014).
Thalidomide a striking illustration: effective and well-tolerated in animal models, catastrophic in humans when used during pregnancy, causing severe limb malformations in thousands of children.
When a product’s scientific bases is limited to animal research, human efficacy remains unestablished by definition.
Level 3: Observational Human Studies
Observational studies generate findings in actual human beings, but they cannot establish causation.
The main limitation is confounding: hidden factors alongside the thing being studied that can make it look responsible for an outcome it didn’t actually cause.
In supplement and wellness research three confounders are common:
Healthy user bias: supplement users differ from non-users in exercise habits, diet quality, smoking rates, and health literacy.
Socioeconomic confounding: wellness products are usually purchased by higher-income populations with better baseline health outcomes for reasons unrelated to supplementation.
Recall bias: self-reported supplement use is unreliable.
No statistical adjustment fully eliminates confounding from observational data.
Level 4: Double Blind Placebo Controlled Randomized Trial
A double-blind placebo-controlled randomized trial (DB-RCT) randomly assigns participants to either an active intervention (the study subject) or an indistinguishable inert comparator (placebo), while keeping both participants and investigators unaware of group allocation until data collection is complete. Randomization distributes known and unknown confounders between groups; blinding eliminates expectation and assessment bias on both sides.
Together these features make the DB-RCT the only study design capable of establishing causation rather than association which is why it sits at the top of the evidence hierarchy. While is the best research design available, is not free of occasional design problems.
If you enjoy evidence-based medical information, subscribe to receive these articles delivered to your mailbox every week.
Open-Label Studies and Testimonials
Open-label studies are conducted without blinding: participants know what they are receiving, and investigators know what they are administering. Testimonials are self-reports from people who used a product and noticed something. Both are the most frequently cited evidence in wellness marketing and carry the least scientific weight.
Three biases operate in the absence of blinding:
Prescriber bias: an investigator who believes in the intervention unconsciously interprets ambiguous findings favorably.
User bias: a participant who knows they received an active treatment attributes any positive findings to the treatment itself.
And the placebo effect: when a person believes they are receiving something beneficial, reported physiological and subjective changes can occur independent of any actual pharmacological activity. Without a control to compare, is not possible to reliably determine cause and effect.
The placebo effect is real, reproducible, and well-documented across clinical trial literature (Hróbjartsson A, Gøtzsche PC, New England Journal of Medicine, 2001). The effect is particularly large for subjective outcomes: energy, mood, pain, and wellbeing. This are precisely the outcomes wellness products most commonly claim to improve.
Where the Marketing Lives
Each level of evidence is legitimate for what it shows. The problem is the claim made from it. Backed by science, clinically proven or any other such phrases may not mean what you think they do.
Two additional problems compound the design issue:
The dose gap: a compound effective at laboratory concentrations may be pharmacologically inactive at the concentration delivered by a consumer product.
The population gap: a finding in patients with a diagnosed deficiency, a specific age group, or a particular clinical condition does not automatically generalize to a healthy adult purchasing a supplement.
The Prevagen Case
Quincy Bioscience, the manufacturer, conducted an actual clinical trial, the Madison Memory Study. The trial showed no statistically significant improvement over placebo in its primary analysis. The company then conducted 30 post-hoc subgroup analyses, identified three results that favored the product, and built a “clinically shown to improve memory” marketing claim on that foundation.
Post-hoc subgroup analysis, running statistical tests on subsets of data after the primary analysis has failed, is a well-recognized form of data mining that produces findings by chance rather than by design.
The FTC and the New York Attorney General filed suit in 2017. After seven years of litigation and more than $165 million in sales, a federal court issued an injunction in December 2024 ordering removal of all memory-improvement claims (FTC press release, December 10, 2024; NY Attorney General press release, May 2024).
Three Questions Worth Asking
The entire framework collapses into three questions you can apply in under sixty seconds to any health claim from any source.
What kind of study is this? Cell study, animal study, observational data, open-label report, or randomized controlled trial. The answer tells you immediately what the evidence can and cannot establish.
Does the claim match what the study actually measured? A finding in isolated cells does not support a claim about human outcomes and showing an association does not support a claim of causation. The question is whether the study supports the claim being made.
At what dose, and in what population? A compound studied at 10 grams per day in elderly patients with confirmed cognitive impairment tells you absolutely nothing about a 500-milligram daily capsule marketed to healthy adults in their forties.
What Can We Do
Cell studies and animal research are essential early steps in understanding how biology works. Observational data has driven some of the most important public health advances of the past century. The problem is the claim made from it; the gap between what a specific piece of evidence was designed to establish and what a manufacturer asserts it proves.
The main question is: does the level of evidence cited as proof, supports the claim made?
Found this article useful? Share your thoughts. Join the conversation below.
Educational content on The Metabolic Archives is free, because medical information should be accessible to everyone. If you find value and want to support the work, a paid subscription is available and genuinely appreciated. Visit the About Page for additional information.
The Metabolic Archives is for educational and informational purposes only, and is not intended as medical advice, diagnosis, or treatment, and does not constitute a doctor-patient relationship. Do not adopt any recommendation discussed in any article or guides published here, make changes or abandon any prescribed medical treatment without prior consultation with your physician. Always seek the advice of your physician or other qualified health provider for any questions regarding your medical condition and recommended treatment options.
By reading this post, you acknowledge that you have read and agree to the Terms of Service of The Metabolic Archives, which govern all use of this content including restrictions on reproduction.
© 2026 The Metabolic Archives. All rights reserved.
Another well written article which makes clear how product prejudice can be and is built into ‘the science’.