TL;DR is the part in bold below.

The field of psychology had a big crisis in the 2010s, when many widely accepted results turned out to be much less solid than previously thought. It's called the replication crisis, because labs around the world tried and failed to replicate, in new experiments, previous results published by their original "discoverers". In other words, many reported psychological effects were either non-existent—artifacts of the experimenter's flawed setup—or so much weaker than originally claimed that they lost most of their intellectual sparkle.

(The crisis spanned other fields as well, but I mostly care about psychology here, especially the cognitive kind.)

This is very old news, and I've been vaguely aware of several of the biggest disgraced results for years, but I keep on forgetting which are (still probably) real and which aren't. This is not good. Most results in the field do actually replicate and are robust^{[citation needed]}, so it would be a pity to lose confidence in the whole field just because of a few bad apples.

This post is a compact reference list of the most (in)famous cognitive science results that failed to replicate and should, for the time being, be considered false. The only goal is to offset the trust-undermining effects of my poor memory—and perhaps yours, too?—with a bookmarkable page.

This can't be a comprehensive list: if a study is not on this page, it's not guaranteed to be fully replicated. Still, this should cover most of the high-profile debunked theories that laypeople like me may have heard of.

Credit: I enlisted the help of Kimi K2, o3, and Sonnet 4 to gather and fact-check this list. I also checked, pruned, and de-hallucinated all the results.

Ego Depletion Effect

• Claimed result: We have a "willpower battery" that gradually depletes during the day as we exercise self-control. (I remember reading Baumeister's pop-science book and being awed by the implications of their findings; I might have known it sounded too good to be true.)
• Representative paper: Baumeister et al. 1998
• Replication status: did not replicate
• Source: Hagger et (63!) al. 2016

Power Posing Effect

• Claimed result: Adopting expansive body postures for 2 minutes (like standing with hands on hips or arms raised) increases testosterone, decreases cortisol, and makes people feel more powerful and take more risks.
• Representative paper: Carney, Cuddy, & Yap (2010)
• Replication status: did not replicate
• Source: Ranehill et al. (2015)

Social Priming: Elderly Words Effect

• Claimed result: People walk more slowly after being exposed to words related to elderly stereotypes.
• Representative paper: Bargh, Chen, & Burrows (1996)
• Replication status: did not replicate
• Source: Doyen et al. (2012) (I like how they prove that the psychological effect was actually in the experimenters, rather than the subjects!)

Money Priming Effect

• Claimed result: Simply thinking about money makes you more selfish and more likely to endorse free market values.
• Representative paper: Vohs, Mead, & Goode (2006)
• Replication status: did not replicate
• Source: Rohrer, Pashler, & Harris (2015)

ESP Precognition Effect

• Claimed result: In some cases, people can predict future events "that could not otherwise be anticipated through any known inferential process".
• Representative paper: Bem (2011)
• Replication status: did not replicate
• Source: Galak et al. (2012), Ritchie, Wiseman, & French (2012)

Cleanliness and Morality Effect

• Claimed result: Being clean or thinking about cleanliness makes people more morally lax.
• Representative paper: Schnall, Benton, & Harvey (2008)
• Replication status: did not replicate
• Source: Johnson, Cheung, & Donnellan (2014)

Glucose and Ego Depletion Effect

• Claimed result: Connected to the debunked ego-depletion effect, this one claims that adding glucose to your blood "recharges" the willpower battery. (For a while, I may have drunk more orange juice than usual after reading Baumeister's book. At least it's healthy-ish.)
• Representative paper: Gailliot & Baumeister (2007)
• Replication status: did not replicate
• Source: Lange & Eggert (2014)

Hunger and Risk-Taking Effect

• Claimed result: People exposed to the scent of freshly baked cookies become less sensitive to risk and take more risks to obtain food.
• Representative paper: Ditto et al. 2006
• Replication status: did not replicate
• Source: Festjens, Bruyneel, & Dewitte (2018)

Psychological Distance & Construal Level Theory

• Claimed result: "Psychologically distant" events are processed more abstractly, while "psychologically near" events are processed more concretely. E.g., you worry about the difficulty of a task if you have to do it tomorrow, but you see the same task's attractive side if it is planned far in the future.
• Representative paper: Trope & Liberman (2010), building on Liberman & Trope (1998)
• Replication status: serious credibility problems
• Source: A collaboration between 73 labs around the world is vetting this theory right now because of many doubts about its validity.

Ovulation & Mate Preferences Effect

• Claimed result: Women are more attracted to hot guys during high-fertility days of their cycles.
• Representative paper: Gildersleeve, Haselton, & Fales (2014)
• Replication status: did not replicate
• Source: Stern, Gerlach, & Penke (2020)

Marshmallow Test & Long-Term Success Effect

• Claimed result: Children's ability to resist eating a marshmallow when left alone in a room at age 4-5 strongly predicts adolescent achievement, with those who waited longer showing better life outcomes.
• Representative paper: Shoda, Mischel, & Peake (1990)
• Replication status: did not replicate significantly
• Source: Watts, Duncan, & Quan (2018)

Stereotype Threat (Women's Math Performance) Effect

• Claimed result: Women risk being judged by the negative stereotype that women have weaker math ability, and this apprehension disrupts their math performance on difficult tests.
• Representative paper: Spencer, Steele, & Quinn (1999)
• Replication status: did not replicate
• Source: Flore & Wicherts (2015)

Smile to Feel Better Effect

• Claimed result: Holding a pen in your teeth (forcing a smile-like expression) makes you rate cartoons as funnier compared to holding a pen with your lips (preventing smiling). More broadly, facial expressions can influence emotional experiences: "fake it till you make it."
• Representative paper: Strack, Martin, & Stepper (1988)
• Replication status: did not replicate
• Source: Wagenmakers et (54!) al. (2016)

Objective Measurement of Biases

• Claimed result: You can predict if someone is racist by how quickly they answer certain trick questions.
• Representative paper: Greenwald, McGhee, & Schwartz (1998)
• Replication status: mixed evidence with small effects
• Source: Oswald et al. (2013) shows that the prediction power is small at best.

Mozart Effect

• Claimed result: Listening to Mozart temporarily makes you smarter.
• Representative paper: Rauscher, Shaw, & Ky (1993)
• Replication status: did not replicate
• Source: Pietschnig, Voracek, & Formann (2010) (What a title!)

Growth Mindset Interventions

• Claimed result: Teaching students that intelligence is malleable (not fixed) dramatically improves academic performance.
• Representative paper: Dweck, & Leggett (1988)
• Replication status: mixed results - many failed replications but also some successful replications
• Failed replication source: Li & Bates 2019
• Notable successful replication: Yeager et al. 2019 in Nature

Bilinguals Are Smarter

• Claimed result: Being bilingual provides substantial cognitive advantages in attention, task-switching, and executive control.
• Representative paper: Bialystok, Craik, & Luk (2012)
• Replication status: did not replicate
• Source: Lehtonen et al. 2018

Did I miss any famous debunked studies? Let me know by replying to this newsletter, and I'll add it to the list. ●

A while ago I treated you to a dissertation entitled “Does The Sun Set On The British Empire?”, and concluded that it doesn't. The UK's widely scattered overseas territories, sparse though they are, mean that the sun is still always shining, somewhere in the world, over British territory.

Real understanding, argues thought leader Jeff DeGraff, doesn’t come from outputs — it comes from practice.

Change comes at you fast.

In late May 2025, Google announced its Veo 3 video generation model at its I/O conference. Early testers were stunned by the photorealistic video, audio, and dialogue that the model produced.

Just a month later, a single director used Veo 3 to create a hyper-realistic AI-generated ad for betting platform Kalshi. The ad took three days to create and cost $400 in Veo 3 credits. It aired during the NBA Finals beside ads that took months to create and cost hundreds of thousands of dollars.

Veo 3 isn’t the first AI video generation platform. But its power has given rise to a sobering fact: in the right (or wrong) hands, generative AI tools can now produce images and videos that are indistinguishable from reality.

These tools are cheap and are usable by anyone who can type a prompt into a chat window. That may lead to an unprecedented wave of AI-generated content across social media platforms and digital channels that is so realistic it’s hard to tell if it’s real or not. In fact, AI slop already has a strong presence on platforms like YouTube, Pinterest, and Instagram, as John Oliver explained on a recent episode of Last Week Tonight.

This begs the question: What, if anything, can be done about this?

A Big (Synthetic) Problem

How big of a problem is the proliferation of hyper-realistic synthetic images and video generated by AI? According to experts, it’s enormous.

“It is now almost impossible to tell, in the digital world, what is real and what is artificial,” said Paolo Giudici, a professor of statistics at Italy’s University of Pavia.

AI image and video models can now produce content for distribution on social media that most users “would not question as fake,” said Mike Perkins, a researcher at the British University Vietnam who has done work on synthetic content.

In a sense, Pandora’s box has been opened. The AI tools that generate hyper-realistic synthetic content are not going away. In fact, as Veo 3 proves, they’re getting better, faster.

So, the first, and biggest, effort to address what is digitally real and what is not starts at the tool level. And the primary way that is being tackled right now is through watermarking.

Some AI labs are using, or participating in, digital watermarking efforts to indicate that an image is AI-generated by adding data to the image file itself the moment it is generated by an AI tool. The image essentially carries a digital “badge” that indicates it is AI-generated.

One of the top lab-run watermarking initiatives is SynthID from Google. SynthID embeds into content machine-readable watermarks that can’t be seen by the human eye. It is now automatically added to all content produced by Google’s generative AI models, including Veo. Google is reported to have already watermarked more than 10 billion pieces of content with SynthID.

Other labs, like OpenAI and Microsoft, participate in C2PA, the Coalition for Content Provenance and Authority, an open standard for watermarking AI-generated content. The C2PA initiative seeks to create a standardized way to track the origin of digital content. It allows cryptographically signed metadata to be attached to digital assets identifying the tools that created it.

Watermarking shows plenty of promise, said Giudici. Efforts like SynthID and C2PA are becoming more sophisticated and cost-effective.

But there’s still an obvious problem.

Watermarking works, but it requires universal, consistent application at scale to fully address the problem. And we are nowhere near that.

Policing Falls Short

To fill the gaps in watermarking’s coverage, some social media platforms are also taking steps to combat AI-generated content.

Instagram automatically attaches a “Made with AI” tag to AI-generated content flagged by its systems or by the user uploading it. YouTube requires creators to disclose when a video they upload contains AI-generated content. TikTok requires users to label AI-generated content.

But a familiar problem quickly rears its ugly head: for platform-led policing to be effective, every platform needs to do it consistently. Based on the flood of AI-generated content already available across these platforms, this is something that is decidedly not happening today.

That leaves a burden on users to address the issue. The problem, said Perkins, is that many users are unable to identify AI-generated content.

“I believe we are now at the tipping point where the signs of fakes are becoming so small that we need to rely on critical thinking of viewers rather than any visible artifacts or problems with the content,” he said. That raises the importance of verifying the source of consumed information, a skill in short supply if the proliferation of online misinformation is an indicator.

The best bet, Perkins said, may be more education at the user level. When social media users know how good AI-generated content has become, they can be better prepared to handle their consumption.

“Having some form of AI literacy is important so that users realize what is possible, and then know what to watch out for,” he said.

Then there’s the final, perhaps largest, elephant in the room.

Watermarking is inconsistently applied. Platform- and user-led policing is inconsistently applied. But even if both somehow worked perfectly, that wouldn’t eliminate hyper-realistic AI-generated content that’s designed to mislead, due in part to open-source AI.

Open-source AI allow users to run models that generate images and video locally on a robust machine without restrictions or guardrails imposed by the model creator.

That means, said Perkins, that malicious actors can find open-source models that don’t have watermarking and use them to generate non-watermarked content. Even if an open-source model creator adds watermarking later, bad actors can simply run a version of the model—or another one—without any safeguards.

While Perkins recommends that platforms implement more robust policies and technical checks, and that users become more AI-aware and literate, at the end of the day open-source models challenge a perfect solution to distinguishing what’s real and what’s not.

And that’s a future for which users are not ready.

“It’s my opinion,” said Perkins, “that the current methods for detection of AI-generated image and video content is entirely insufficient to prevent the average user from being fooled by the new generation of text-to-video diffusion models.”

Logan Kugler is a freelance technology writer based in Tampa, FL. He is a regular contributor to Communications and has written for nearly 100 major publications.

Given that AI is here to stay, it behooves those of us attempting to adapt universities to the realities of the mid-21st century to pay attention to insights gleaned from past wisdom as we develop our curricula. Will we turn our backs on AI, pretending that device-free exam rooms will somehow prepare students for an AI-saturated world? Will we simply allow our students to sell their figurative souls to the AI devil, allowing it to do much of their thinking for them? Or will we recognize that today’s most important skill is to think alongside machines and start building institutions bold enough to teach students how to do this?

The most up-to-date AI models are more knowledgeable not only than any individual human mind but are in certain respects able to rival entire expert communities. They provide an interactive, pattern-based generator enabling them to do enormous labor. Given their novelty, one might suppose that ethical issues surrounding their adoption would be a purely contemporary concern. Nevertheless, works of literature frequently debate significant issues long before humans create the technology that allows them to be instantiated. So it is with AI, one of whose leading purveyors in world literature turns out to be not Sam Altman or Elon Musk but a rather more complex figure—Satan.

The earliest major literary work to develop the idea of a deal with the devil in the context of AI is Christopher Marlowe’s The Tragical History of Doctor Faustus (early 1590s). Today’s reader is struck by Faustus’ situation before he sells his soul. He is no desperate second-rater, but a respected scholar. Describing his accomplishments, Faustus points to his skill as a physician: “Are not thy bills hung up as monuments / Whereby whole cities have escaped the plague / And thousand desperate maladies been eased?” But what can be achieved through purely human effort is insufficient. He turns to magic and signs over his soul to the devil in exchange for access to artificial intelligence for twenty-four years. Unfortunately, Faustus does not employ his gift to make the world better but instead fritters it away in frivolous adventures and dies miserably.

Marlowe pinpointed a central issue surrounding the adoption of AI. The AI user can know everything, but there is a danger that in accepting this tool he may give up his duty to do the work (intellectual/moral) that would give meaning to this knowledge. Merely ask Mephistopheles or ChatGPT and nothing further is needed. In the end, however, sloth kills, and the devil will have his due. Marlowe asks: If humans no longer need to work to achieve knowledge, will they still value it or will they spend their lives playing the equivalent of elaborate video games?

Subscribe now

Goethe’s version of Faust—manifesting in literature two hundred years after Marlowe’s—is well respected by his scholarly colleagues, but after a life of study he despairs, “All I see is that we cannot know!” He craves endless knowledge without work:

Therefore I have turned to magic,
so that by the spirit’s might and main
I might yet learn some secret lore;
that I need no longer sweat and toil
and dress my ignorance in empty words;
that I might behold the warp and woof
of the world’s inmost fabric,
of its essential strength and fount
and no longer dig about in words.

Mephistopheles promises precisely the ability to gain insight without toil. “My friend, in this one hour you will gain / far more for all your senses / than in a year’s indifferent course.”

Goethe’s hero is willing to sign over his soul but fails to make good use of his newfound powers. Instead, like many of our students, he decides to apply his newfound knowledge to a field in which he has no expertise and one for which the tool is singularly unfit—in this case, love. The result is disastrous, for Faust himself and for the object of his seduction and her family.

However, the Faust story is quite elastic, and 20th-century authors, including Thomas Mann in his 1947 novel Doktor Faustus, would revisit it to illustrate complexities that could provide insights for today. Mann’s hero, Adrian Leverkühn, is a talented composer who dreams of achieving transcendental greatness. The novel’s key chapter records a conversation between the composer and the devil, who materializes suddenly and offers Leverkühn twenty-four years of time. But, unlike earlier devils who purvey knowledge without the need for work, and in a twist on the usual facile Faust-narrative of soul-selling, Mann’s incarnation promotes AI as a tool to be deployed alongside human ingenuity to allow for the production of compositions that would otherwise be unrealizable. As the devil puts it in his baroque idiom:

Time? Simple time? No my dear fere, that is not the devyll’s ware. For that we should not earn the reward, namely that the end belongs to us. What manner of time, that is the heart of the matter! Great time, mad time, quite bedivelled time, in which the fun waxes fast and furious with heaven-high leaping and springing.

Leverkühn accepts the devil’s bargain and, if we are to believe the novel’s narrator, during the twenty-four years at his disposal he creates a series of brilliant compositions that set the agenda for the development of music for generations. He is able to accomplish this, however, because years of training had given him enormous expertise and because he remained willing to spend endless hours refining his gifts with the help of Satan’s AI boost. True to form, at the end of the allotted period Leverkühn burns out and dies, but we can suppose he found his deal with the devil to have been fair and fulfilling.

As educators develop our university curricula in the AI age, our wisest choice out of the trinity offered to us by classical literature is to attempt to follow Mann’s lead, recognizing that in the mid-21st century the challenge will be not to choose between human or artificial intelligence but to discover a synthesis through systematic experimentation—one that might allow our students to produce transcendental work like those of Leverkühn, though hopefully without the need to burn out in the end.

Andrew Wachtel is president of inVision U, a university project piloting in Kazakhstan and featuring a curriculum allowing students to synthesize human and artificial intelligence.

Follow Persuasion on Twitter, LinkedIn, and YouTube to keep up with our latest articles, podcasts, and events, as well as updates from excellent writers across our network.

And, to receive pieces like this in your inbox and support our work, subscribe below:

Subscribe now

In our everyday lives, humans interact with lots of other humans. Most of the time in these interactions, we need to keep track of approximately one other human’s thoughts and feelings. That’s not easy! It’s not easy to live your life while also thinking about the perspective of another human sitting or standing across from you.

Teaching is an order of magnitude more difficult. The task is to keep track of 25 humans in front of you. Oh, and three of them have to pee, one is mad at the kid he’s sitting next to, two forgot their pencils, one is glaring at you for some reason, and another is kicking the table in front of her every time she thinks you’re not looking.

One of the hardest intellectual tasks as a teacher is keeping all that complexity in your mind. Teachers often say, “oh, they know that,” or “they’re confused about X.” Well not every student knows that. You mean most students know that. But that’s a hard idea to hold in your head, so we end up with this shorthand as if a class of students isn’t 25 individuals, but some weird monolithic many-headed behemoth.

Here are three ways that’s bad.

Moving On

You teach something. You realize most students understand it. So you say, “Great, looks like we understand this, let’s move on.” But actually, only 22 of the 25 students understand it. This sends an unmistakable message: there is an “us” in the room, but those last three students aren’t part of it.

There are practical considerations in teaching. Sometimes we have to move on. But the framing matters. Will you circle back to this topic in the future? Will you follow up with students individually? Is there a time students can get extra help? Don’t just say, “Great, we understand this.” Say, “Thanks everyone. We’ll keep working on this topic later in the week,” or make a note and follow up with those last few students individually. Anything that doesn’t render students invisible.

Using the Restroom

I have taught a lot of students who have made poor choices in the restroom. Fights, vandalism, vaping, wandering the school, and more. It’s a challenging problem to solve. But here is a hill I will die on: student A does not lose their right to go to the bathroom because student B (or students B, C, D, E, F, and whoever else) made poor choices in the restroom.

Whether it’s an administrator responding to bathroom vandalism or a teacher saying “no more bathroom visits this class,” after a trash can gets toppled in the hallway, we often respond with consequences for the many in response to the transgressions of a few. In many cases punishing everyone else just adds to the humor for the culprits. In the long run, it creates exactly the negative feelings toward school that cause students to tear toilets out of walls. Students are not some hive-mind organism that makes collective decisions. One or a few students made a decision, and we can separate those students from everyone else — even if we don’t know who the specific culprits are.1

Math Identity

Here’s an activity that has become more common in math classes in the last few years (on my mind because I got a marketing email suggesting something like it yesterday): some version of a mathematical autobiography. Early in the school year, the teacher asks students to describe their experience with math. There are lots of variations on this activity — some narrative, some artistic, with different prompts or structure. The goal is to surface how students feel about math, and to start conversations that will (hopefully) lead to a positive change in those feelings. Disclaimer: I’ve done lots of these in the past!

Whenever I’ve done some variation on this activity, I’m always struck by how many students feel negatively about math. There’s plenty more to learn as well, but that’s always the headline takeaway. So what do I do about it? Well, here’s the truth. In a given class I’ll manage to glean a few solid takeaways. Start an interesting conversation, connect with a student about something they wrote. But for every win and every relationship built, there are a bunch more students who say something negative about math and never feel like it goes anywhere. It’s easy for me to mistake a few anecdotes for broad success. I think about that one student I had a great conversation with, and let that conversation stand in for the entire class. There are a bunch of other students in the room who write about how much they don’t like math, and they just go on…continuing to not like math. My job is to hold in my head that I can have some good conversations with a few students, but the goal is to teach everyone. How does the rest of the class feel?

Double Vision

I think this is one of the most challenging intellectual tasks of teaching. I could give more examples — calling on a single student and assuming the rest of the class is on the same page, generalizing about how long students can pay attention, responding to classroom behavior.

Teaching requires a kind of double vision. Seeing the class as a whole and responding to broad patterns, while also recognizing each student as an individual and seeing the range of experiences within any given class. Getting that right is an important part of teaching, and also an important part of helping every student feel like they are a valued member of the classroom community.