The replication crisis and marketing psychology

What is replication, and what is the crisis about?

Replication is testing whether a research finding recurs with new observations under conditions relevant to the original claim. Reproducibility can also refer to obtaining the same analysis from the same data and code; terminology varies, so reports should explain what was actually done.

The crisis concerns the reliability and interpretation of parts of the published literature. Small samples, selective publication, flexible analysis and incomplete reporting can make an apparent result look more certain than it is. A compelling story can then spread much further than its evidence warrants.

For marketers, there is an additional leap. A finding about a laboratory judgement may become a claim about online sales, then a universal rule in a presentation. Each step adds assumptions about audience, context and outcome. Even a well-replicated laboratory effect does not remove the need to assess those assumptions.

When using the marketing psychology field guide, the right response is neither unquestioning acceptance nor blanket rejection. It is to ask a more precise question: what evidence supports this claim, under which conditions, and how closely does our proposed use match those conditions?

Why does replication matter to marketing decisions?

Marketing decisions commit money and shape customer experience, so an overstated psychological claim can justify an ineffective or misleading intervention.

The Open Science Collaboration’s 2015 project repeated a selected set of psychology studies and found that many results were weaker or did not meet the same significance criterion in replication. The project was not a census of all psychology and should not be described as proving a fixed proportion of the entire discipline false.

Replication also changes what a responsible brief should say. “This mechanism may be relevant, but the commercial application needs testing” is different from “science proves this will increase conversion.” The first statement preserves a useful hypothesis. The second invents certainty.

A failed replication is not automatically evidence of fraud. Differences in methods, context, random variation or the original estimate can contribute. Those possibilities need investigation; they should not become an unlimited excuse that makes a theory impossible to challenge.

What happened to some famous findings?

Several influential claims became more qualified when larger or more coordinated tests examined the relevant procedures.

Ego depletion became a contested explanation

Baumeister and colleagues’ 1998 paper proposed that prior self-control exertion could impair later self-control, interpreted through a limited-resource account. The popular marketing version often turns this into a simple rule about tired customers becoming easier to persuade.

Hagger and colleagues’ 2016 preregistered multilab project found a very small estimated effect, with uncertainty including no effect, for the specific standardised procedure tested. It was not a literal rerun of every original task. The result challenged a strong general account and should make marketers cautious about treating depleted willpower as a dependable sales mechanism.

Facial feedback requires procedural distinctions

Strack, Martin and Stepper’s 1988 study investigated facial feedback using a pen-holding procedure. A 2016 registered replication report did not reproduce the original result in that procedure.

The story did not end with a universal rejection of facial feedback. The Many Smiles Collaboration’s 2022 study found support for some facial-action tasks, while evidence for the pen-in-mouth task was less conclusive. The responsible conclusion distinguishes methods rather than declaring that smiling either always changes emotion or never does.

None of this directly proves that making customers smile will improve a brand’s sales. That commercial leap requires its own evidence even when an emotional effect is supported.

Decoy effects have important boundaries

Huber, Payne and Puto’s 1982 paper introduced an influential attraction-effect finding. Frederick, Lee and Baskin’s 2014 research demonstrated limits to its generality under different attribute representations.

This is a boundary-condition lesson rather than a clean declaration that every attraction effect disappeared. As the guide to cognitive biases in buying explains, a pricing team should not assume that adding an inferior option will reliably increase useful demand, particularly when the real choice is more complex than the experimental display.

How should a team assess a psychology claim?

Trace the claim to the original study, inspect later evidence and distinguish the observed result from the proposed marketing application.

1. Write the claim precisely

Replace a broad label with a statement about an intervention, audience and outcome. “This display may improve recall of the product name” is inspectable. “The brain loves contrast” is too vague to evaluate.

Ask what would count as a failure. A claim that can explain every possible result after the fact is not a useful guide to a test. Precision makes disagreement productive rather than merely rhetorical.

2. Read beyond the abstract or quotation

Check the participants, task, comparison, exclusions and measured outcome. Determine whether the result concerns behaviour, intention, self-report or an indirect measure. Do not replace the original outcome with sales because that is what the business wants.

Look for the original paper’s own limitations. A cautious research conclusion often becomes much more confident in a secondary summary. The presentation should restore the uncertainty rather than amplify the slogan.

3. Search for later tests and criticism

Look for direct replications, registered reports, meta-analyses, corrections and serious methodological responses. A large citation count shows attention, not necessarily confirmation. Several papers citing the same original experiment are not several independent replications.

Distinguish a dispute about effect size from a dispute about existence, mechanism or generality. The practical implications differ. A smaller effect may still matter at low cost, while a poorly understood effect may be unsuitable for a consequential decision.

4. Design the local test before seeing results

Specify the hypothesis, comparison, main outcome and stopping rule. Choose a sample and observation period suited to the decision with appropriate analytical help. Keep unsuccessful results and unexpected harms in the record.

If the intervention changes several elements, describe the result as evidence about that package. Do not claim to have validated a particular psychological mechanism unless the design separates it from plausible alternatives.

Where is transfer especially uncertain in India?

Transfer is especially uncertain when the original research differs from the intended audience in language, category familiarity, stakes or the way choices are made.

A student laboratory task in one country is not a complete model of a household purchase or a regional B2B sale. That does not make the study irrelevant. It identifies the additional assumptions a local application must test.

Avoid replacing one overgeneralisation with another. “Indian consumers are different” is not a sufficient explanation either. Describe the relevant difference and how it could change the mechanism. Then seek evidence from the actual decision context.

When should a familiar claim be retired?

Retire or qualify a claim when the evidence no longer supports the confidence with which the team uses it, or when repeated local tests fail to justify its practical role.

In week one of an evidence audit, inspect the psychology claims in current briefs and training materials. Prioritise those used to justify spending, pressure tactics or consequential customer decisions. Update the wording and record the sources checked.

Do not wait for a definitive verdict on an entire theory before correcting an exaggerated marketing statement. A claim can be too strong for the evidence even while the underlying research question remains open.

Who owns evidence quality?

The strategy or research owner should maintain the evidence record, while the campaign owner remains responsible for the decision made from it.

Analysts help evaluate uncertainty. Creative teams should be told which ideas are hypotheses. Leaders should reward accurate reporting of null results rather than encouraging teams to find a winning narrative in every test. That incentive matters as much as the checklist.

My view

I would keep behavioural science in the toolkit and remove the promise of effortless control. Its most useful contribution is often a better question about how people understand a decision.

The trade-off is that careful evidence rarely fits a dramatic slide headline. It may require a narrower claim and a local experiment. That is worthwhile when the alternative is building a strategy on a story that sounds scientific but has lost its conditions.

A useful psychological claim becomes more precise when the evidence changes.

Start with one famous claim

Take a behavioural rule your team repeats confidently. Find its original study and a serious later test. Rewrite the rule to state what is supported, what remains uncertain and what your own campaign would need to demonstrate.