Well! Last week was an unexpectedly exciting one here at Cognitive Resonance HQ (aka my house) after The Verge published my essay explaining why large-language models are not on the path to general intelligence, because language is not the same as thought. Forgive the crowing, but if I may: Yann LeCunn, the outgoing chief AI scientist at Meta, posted about it favorably on LinkedIn; Alison Gopnik, a cognitive scientist I’ve long admired, described it as “excellent and very clear”; and I picked up more than 200 new followers to this newsletter—welcome, y’all! (Yes, I live in Texas. Yee-haw.)

Given this interest, I thought this week I’d address two of the main critiques of my argument. Before continuing, if you need to build some background knowledge on this topic, the entire essay is re-published below (with The Verge’s permission)—and regular readers may note that it borrows liberally from several previous essays published here, including my interview of neuroscientist Ev Fedorenko (more from her later).

But now let’s get into the pushback. Basically, we’ve got two groups of people who think I’m very mistaken about the future of AI, although interestingly their critiques are diametrically opposed to each other. I’m calling the first group the “wing flappers” and the second the “cognitive pluralists.”

The wing flappers believe that my argument fails because I mistakenly assume that the only way to get to artificial general intelligence is by emulating human intelligence. “Cutting-edge research shows airplanes do not work the same as birds, [so] the entire airline industry is a bubble ignoring this,” said someone on Reddit, thinking they zinged me but good. The theory here seems to be that because there’s so much data that’s used to train LLMs, perhaps that’s sufficient in and of itself to achive general intelligence. “Written language stores thought,” wrote one particularly aggravated commentor on The Verge, “so LLMs are learning from the knowledge stored by language.”

On its face, this is a plausible hypothesis about how we might achieve artificial general intelligence. But the number of plausible predictive theories will always exceed the number that actually are true, and I think this perspective suffers from at least three flaws.

The first problem with this counterargument is one I briefly allude to in my essay, which is that there’s a non-trivial amount of human knowledge that sits outside of our language. The philosopher of science Michael Polanyi described this as “tacit knowledge,” the sort of things we know how to do but cannot easily explain. Riding a bike is one common example, but also applies to activities such as playing a musical instrument, or solving a jigsaw puzzle, the sort of things we typically learn how to do from interactive experience. There’s even a growing scientific effort around “embodied cognition” that builds off this notion, and researchers in this area are actively investigating how animals (including humans) develop certain cognitive capacities that may not rely exclusively on mental representations in the brain. Whether one buys into that or not, the point remains that linguistic information codified in writing provides an expansive yet ultimately limited database of knowledge.

As computer scientist Deb Raji observes, “there is no dataset that will be able to capture the full complexity of the details of existence, in the same way that there can be no museum to contain the full catalog of everything in the whole wide world”—more on this here in an essay featuring Grover from Sesame Street.

The second problem the wing flappers face is that “intelligence” as a concept is much fuzzier than something like, say, “flying.” When it comes to the latter, either the thing we care about is successfully locomoting through the air, or not—there’s an objective definition of success. The same simply cannot be said whether something is manifesting “intelligence”—consider, for example, that there’s been a long-running debate in cognitive science over whether a household thermostat manifests intelligent behavior. If we abandon using human cognitive processes as our baseline for developing and evaluating the behavior of artificial systems, it’s not at all clear what should take its place. The Wright Brothers at least had aerodynamic data to work with.

Similarly, the third flaw with this counterargument is that generative AI as it exists today arises from explicit efforts to model human cognitive processes, in our brains! For several decades, cognitive scientists operating in the “connectionist” school have argued that the process that we use to process information through neurons could be simulated artificially, and they’ve diligently worked to build such models digitally. We call them “artificial neural networks” for a reason, they are explicitly premised on a theory of how our brains function. And it turns out that with enough computing power—and especially the capacity to process information in parallel—you can indeed develop an AI system that can, with a lot of human tuning, fluidly process an incredibly diverse array of input. Geoffrey Hinton deserves his Nobel Prize! But it is profoundly odd to argue that, having used the human brain as the model to get us where we are today with generative AI, we should now chuck it aside.

That’s enough on wing flapping, now let’s turn to the second group of critics, the “cognitive pluralists. ” Unlike the flappers, the pluralists believe that we should be modeling AI systems with human intelligence as our north star. And they largely agree with me that there is a broad range of cognitive capabilities that go beyond what’s captured in linguistic data. But the cognitive pluralists believe we can, and likely will, achieve something akin to AGI by stitching together digital emulations of these various capacities in a multi-modal AI system. Frankenstein, but for intelligence.

Look, I also anticipated this objection, and pointed to a new definition of AGI that many AI luminaries recently signed on to as begging many questions about what exactly should be included or excluded in their definition of intelligence—a construct validity question. But even if we hand-wave that away and assume the pluralists have the right goal in mind—a very big assumption indeed—they are all over the map in what it’ll take to get there. Consider the following theories from various AI bigwigs:

• Sundar Pichai says that AI models can surmount the limitations of language by being trained on “video, text, images and code,” he seems to think multimodality is literally all we need;

• Yann LeCun is leaving Meta to found an AI startup that will focus on building AI with “world models”;

• Yoshua Bengio has been working on capturing the “flow state” of the brain as a dynamical system;

• Gary Marcus maintains that AI will need to be able to manipulate symbols (“neurosymbolic AI”) to ever match or surpass human intelligence; and

• Ilya Sutskever hints that AGI will only arrive if we discover a new way to simulate human generalization using sparse data, seemingly using some evolutionary-like method.

So many men, so many different dreams of superintelligence. Whether any of them will be realized, however, is unknown. No one knows! My point remains that scaling up linguistic data will not be sufficient to deliver omniscient robots that will bring forth our glorious abundant future. That’s it, that’s my stake in the ground.

In the meantime, we should continue to pursue scientific research on how we humans think, and compare and contrast this to how AI models do what they do. To that end, in a bit of fortuitous timing, just last week a group of researchers—including Ev Fedorenko yet again—published an interesting paper that offers a conceptual model for how humans “export” linguistic information to interact with our other cognitive capacities. They posit a model that looks something like this:

Note that under this model, our core language system is limited to “shallow” understanding of linguistic forms, whereas other cognitive capacities support “deep understanding.” These researchers then ask an important question that speaks to the “known unknowns” of the AI scientific frontier: “Do any AI systems understand language deeply? If so, then we might expect them to show signatures of exportation. Several recent studies have attempted to explicitly build exportation into AI systems by augmenting LLMs with diverse extralinguistic systems: vision models, physics engines, formal logic provers, theory of mind engines, memory augmentations, and many more.”

Note the word “attempted”—there’s no breakthroughs yet. Again, we simply do not know how any of this will shake out. In the meantime, I will repeat my call to adopt an adversarial mindset when it comes all things AI related, including whether we’re on the path to AGI or superintelligence. Be skeptical. Be critical. Be thoughtful!

Now onto the essay…

Large Language Mistake

Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it.

“Developing superintelligence is now in sight,” says Mark Zuckerberg, heralding the “creation and discovery of new things that aren’t imaginable today.” Powerful AI “may come as soon as 2026 [and will be] smarter than a Nobel Prize winner across most relevant fields,” says Dario Amodei, offering the doubling of human lifespans or even “escape velocity” from death itself. “We are now confident we know how to build AGI,” says Sam Altman, referring to the industry’s holy grail of artificial general intelligence — and soon superintelligent AI “could massively accelerate scientific discovery and innovation well beyond what we are capable of doing on our own.”

Should we believe them? Not if we trust the science of human intelligence, and simply look at the AI systems these companies have produced so far.

The common feature cutting across chatbots such as OpenAI’s ChatGPT, Anthropic’s Claude, Google’s Gemini, and whatever Meta is calling its AI product this week are that they are all primarily “large language models.” Fundamentally, they are based on gathering an extraordinary amount of linguistic data (much of it codified on the internet), finding correlations between words (more accurately, sub-words called “tokens”), and then predicting what output should follow given a particular prompt as input. For all the alleged complexity of generative AI, at their core they really are models of language.

The problem is that according to current neuroscience, human thinking is largely independent of human language — and we have little reason to believe ever more sophisticated modeling of language will create a form of intelligence that meets or surpasses our own. Humans use language to communicate the results of our capacity to reason, form abstractions, and make generalizations, or what we might call our intelligence. We use language to think, but that does not make language the same as thought. Understanding this distinction is the key to separating scientific fact from the speculative science fiction of AI-exuberant CEOs.

The AI hype machine relentlessly promotes the idea that we’re on the verge of creating something as intelligent as humans, or even “superintelligence” that will dwarf our own cognitive capacities. If we gather tons of data about the world, and combine this with ever more powerful computing power (read: Nvidia chips) to improve our statistical correlations, then presto, we’ll have AGI. Scaling is all we need.

But this theory is seriously scientifically flawed. LLMs are simply tools that emulate the communicative function of language, not the separate and distinct cognitive process of thinking and reasoning, no matter how many data centers we build.

Last year, three scientists published a commentary in the journal Nature titled, with admirable clarity, “Language is primarily a tool for communication rather than thought.” Co-authored by Evelina Fedorenko (MIT), Steven T. Piantadosi (UC Berkeley) and Edward A.F. Gibson (MIT), the article is a tour de force summary of decades of scientific research regarding the relationship between language and thought, and has two purposes: one, to tear down the notion that language gives rise to our ability to think and reason, and two, to build up the idea that language evolved as a cultural tool we use to share our thoughts with one another.

Let’s take each of these claims in turn.

When we contemplate our own thinking, it often feels as if we are thinking in a particular language, and therefore because of our language. But if it were true that language is essential to thought, then taking away language should likewise take away our ability to think. This does not happen. I repeat: Taking away language does not take away our ability to think. And we know this for a couple of empirical reasons.

First, using advanced functional magnetic resonance imaging (fMRI), we can see different parts of the human brain activating when we engage in different mental activities. As it turns out, when we engage in various cognitive activities — solving a math problem, say, or trying understand what is happening in the mind of another human — different parts of our brains “light up” as part of networks that are distinct from our linguistic ability:

Second, studies of humans who have lost their language abilities due to brain damage or other disorders demonstrate conclusively that this loss does not fundamentally impair the general ability to think. “The evidence is unequivocal,” Fedorenko et al. state, that “there are many cases of individuals with severe linguistic impairments … who nevertheless exhibit intact abilities to engage in many forms of thought.” These people can solve math problems, follow nonverbal instructions, understand the motivation of others, and engage in reasoning — including formal logical reasoning and causal reasoning about the world.

If you’d like to independently investigate this for yourself, here’s one simple way: Find a baby and watch them (when they’re not napping). What you will no doubt observe is a tiny human curiously exploring the world around them, playing with objects, making noises, imitating faces, and otherwise learning from interactions and experiences. “Studies suggest that children learn about the world in much the same way that scientists do—by conducting experiments, analyzing statistics, and forming intuitive theories of the physical, biological and psychological realms,” the cognitive scientist Alison Gopnik notes, all before learning how to talk. Babies may not yet be able to use language, but of course they are thinking! And every parent knows the joy of watching their child’s cognition emerge over time, at least until the teen years.

So, scientifically speaking, language is only one aspect of human thinking, and much of our intelligence involves our non-linguistic capacities. Why then do so many of us intuitively feel otherwise?

This brings us to the second major claim in the Nature article by Fedorenko et al., that language is primarily a tool we use to share our thoughts with one another — an “efficient communication code,” in their words. This is evidenced by the fact that, across the wide diversity of human languages, they share certain common features that make them “easy to produce, easy to learn and understand, concise and efficient for use, and robust to noise.”

Without diving too deep into the linguistic weeds here, the upshot is that human beings, as a species, benefit tremendously from using language to share our knowledge, both in the present and across generations. Understood this way, language is what the cognitive scientist Cecilia Heyes calls a “cognitive gadget” that “enables humans to learn from others with extraordinary efficiency, fidelity, and precision.”

Our cognition improves because of language — but it’s not created or defined by it.

Take away our ability to speak, and we can still think, reason, form beliefs, fall in love, and move about the world; our range of what we can experience and think about remains vast.

But take away language from a large language model, and you are left with literally nothing at all.

An AI enthusiast might argue that human-level intelligence doesn’t need to necessarily function in the same way as human cognition. AI models have surpassed human performance in activities like chess using processes that differ from what we do, so perhaps they could become superintelligent through some unique method based on drawing correlations from training data.

Maybe! But there’s no obvious reason to think we can get to general intelligence — not improving narrowly defined tasks —through text-based training. After all, humans possess all sorts of knowledge that is not easily encapsulated in linguistic data — and if you doubt this, think about how you know how to ride a bike.

In fact, within the AI research community there is growing awareness that LLMs are, in and of themselves, insufficient models of human intelligence. For example, Yann LeCun, a Turing Award winner for his AI research and a prominent skeptic of LLMs, left his role at Meta last week to found an AI startup developing what are dubbed world models: “​​systems that understand the physical world, have persistent memory, can reason, and can plan complex action sequences.” And recently, a group of prominent AI scientists and “thought leaders” — including Yoshua Bengio (another Turing Award winner), former Google CEO Eric Schmidt, and noted AI skeptic Gary Marcus — coalesced around a working definition of AGI as “AI that can match or exceed the cognitive versatility and proficiency of a well-educated adult” (emphasis added). Rather than treating intelligence as a “monolithic capacity,” they propose instead we embrace a model of both human and artificial cognition that reflects “a complex architecture composed of many distinct abilities.”

They argue intelligence looks something like this:

Is this progress? Perhaps, insofar as this moves us past the silly quest for more training data to feed into server racks. But there are still some problems. Can we really aggregate individual cognitive capabilities and deem the resulting sum to be general intelligence? How do we define what weights they should be given, and what capabilities to include and exclude? What exactly do we mean by “knowledge” or “speed,” and in what contexts? And while these experts agree simply scaling language models won’t get us there, their proposed paths forward are all over the place — they’re offering a better goalpost, not a roadmap for reaching it.

Whatever the method, let’s assume that in the not-too-distant future, we succeed in building an AI system that performs admirably well across the broad range of cognitive challenging tasks reflected in this spiderweb graphic. Will we have achieved building an AI system that possesses the sort of intelligence that will lead to transformative scientific discoveries, as the Big Tech CEOs are promising? Not necessarily. Because there’s one final hurdle: Even replicating the way humans currently think doesn’t guarantee AI systems can make the cognitive leaps humanity achieves.

We can credit Thomas Kuhn and his book The Structure of Scientific Revolutions for our notion of “scientific paradigms,” the basic frameworks for how we understand our world at any given time. He argued these paradigms “shift” not as the result of iterative experimentation, but rather when new questions and ideas emerge that no longer fit within our existing scientific descriptions of the world. Einstein, for example, conceived of relativity before any empirical evidence confirmed it. Building off this notion, the philosopher Richard Rorty contended that it is when scientists and artists become dissatisfied with existing paradigms (or vocabularies, as he called them) that they create new metaphors that give rise to new descriptions of the world — and if these new ideas are useful, they then become our common understanding of what is true. As such, he argued, “common sense is a collection of dead metaphors.”

As currently conceived, an AI system that spans multiple cognitive domains could, supposedly, predict and replicate what a generally intelligent human would do or say in response to a given prompt. These predictions will be made based on electronically aggregating and modeling whatever existing data they have been fed. They could even incorporate new paradigms into their models in a way that appears human-like. But they have no apparent reason to become dissatisfied with the data they’re being fed — and by extension, to make great scientific and creative leaps.

Instead, the most obvious outcome is nothing more than a common-sense repository. Yes, an AI system might remix and recycle our knowledge in interesting ways. But that’s all it will be able to do. It will be forever trapped in the vocabulary we’ve encoded in our data and trained it upon — a dead-metaphor machine. And actual humans — thinking and reasoning and using language to communicate our thoughts to one another — will remain at the forefront of transforming our understanding of the world.

My thanks to the terrific editing team at The Verge who helped polish my prose (and push my thinking), and to Dr. Adam Dubé at McGill University for making the introduction.

Subscribe now

Lately I’ve been thinking about how much pressure we put on ourselves to only do things we’re already good at. Somewhere along the way, “trying something new” stopped sounding fun and started feeling like a reputational risk. Like if you can’t perform at a certain level instantly, you shouldn’t bother.

That mindset looks harmless from the outside. It feels responsible. Mature, even. But it quietly chokes off the part of you that’s curious, playful, and willing to be surprised. And it keeps you stuck in the same narrow corners of your life.

What I’ve been noticing in myself is this instinct to avoid anything where I can’t show up as the polished version of who I used to be. The competent one. The quick learner. The person who could pick something up and immediately “get it.”

(We’ve been unpacking this in therapy.)

Burnout has a way of shrinking your world like that. Even long after the exhaustion fades, the pressure to not mess up can linger. You don’t want to see yourself fall short again, so you avoid situations where you might.

I’ve accepted that I’ve been putting pressure on myself that nobody else is even giving a second thought to. Nobody is asking me to figure it out on the first try, to be immediately good at something. It’s a self-imposed requirement that sets me up to fail.

But recently, I’ve been letting myself be bad at things again. Truly bad. And there’s something surprisingly freeing about it.

Being bad lowers the stakes

Take learning French. I have all the apps. All the books. All the good intentions. And yet: my pronunciation? Rough. My verb conjugations? In progress. My confidence? Depends on the hour. Generally low. I’m still terrified to speak French in public, but I did say one sentence in French out loud today. Please clap.

For a while, that bothered me. I felt slow. Like I was failing. Like my brain should just “try harder.”

But then I realized the moment you admit you’re bad at something, the pressure to be perfect disappears. There’s nothing to protect. No expectations to uphold. You’re just a person learning a new thing, which is one of the most human experiences you can have.

It’s honestly kind of fun.

Being bad opens the door to creativity

There’s also this unexpected creativity that shows up when you stop needing to be good right away. When you’re allowed to make mistakes, you naturally experiment more. You explore. You follow impulses without evaluating whether they’re “right.”

That messy middle is where the good stuff hides.
It’s where play comes back.

And play is a muscle you lose when everything in your life has revolved around productivity, competence, or survival.

Being bad rebuilds your trust in yourself

The more I let myself be terrible at something, the more I notice how quickly “terrible” turns into “okay,” and how “okay” eventually turns into “not bad actually.”

It’s not about the skill. It’s about proving to yourself that you can start small, stay with it, and watch the world open a little because you were brave enough to begin.

That’s the quiet part of burnout recovery no one talks about. Your life expands again through the small, low-stakes attempts.

Not through a grand reinvention, but through tiny, unimpressive experiments that remind you you’re still capable of learning and creating.

Letting yourself be a beginner again

So here’s where I’ve landed:

You don’t need confidence before you try something. Confidence shows up after you’ve tried it enough times that you stop caring how you look doing it.

Letting yourself be bad at something isn’t a failure. It’s an act of self-trust. It’s choosing curiosity over ego. It’s giving yourself permission to exist outside the narrow confines of what you already know how to do.

Honestly, it feels good to grow again, even if the growth looks awkward at first.

If you’re in a season of rebuilding, or rediscovering parts of yourself, or letting your world widen after it’s felt small for a while, maybe give yourself the gift of being bad at something. You might enjoy who you become on the other side of it.

Source

Gradual Release of Responsibility

I’m not a fan of I/we/you lessons.

I/we/you is the default lesson structure for lots of math teachers. Start by modeling a skill. Then work through the skill as a class. Then give students some independent practice.

The tagline for this type of teaching is “gradual release of responsibility.” A graph of the lesson might look like this:

Take the task you want students to be able to do. Start by offering a lot of support, typically by modeling or looking at examples, then gradually have the students do more of the work themselves until they’re doing the task on their own.

I don’t think this is a good model. Instead of I/we/you with gradual release of responsibility, I think it‘s more helpful to think of a gradual increase in difficulty.

Gradual Increase in Difficulty

Gradual increase in difficulty might look like this:

But that’s actually not realistic. The stuff we want kids to learn doesn’t break down into nice neat steps that are all the same size. Maybe a lesson looks more like this:

I’m not putting teacher support on here, because it’s not some predetermined gradual release in responsibility. Instead it’s adaptive: some steps are small and students can make the jump without much support. Others are trickier and require more support.

Here’s what this might look like in terms of I/we/you:

You/I/you/you/I/you (this is a bunch of review questions about relevant prior knowledge, with some quick reteaches as necessary)

I/we/you (there’s a jump in difficulty that students need some support with)

I/you/I/you (this is two quick cycles for small, manageable chunks of new learning building on the last step)

You/you (this is a few more jumps in difficulty, but smaller jumps that students can do on their own)

I/you (one final model and then some more practice)

The key to making all this work is to break learning down into small, manageable steps, and help students connect each new step to what came before it. This is a big mental shift from teaching one objective at a time. It requires knowing your content well. It also means tossing out some of the curricular resources you might’ve been given.

I’m sure some people are saying, “Hey, this is how I teach, and I call this I/we/you.” Great! Glad we agree. My experience is that what I’m describing is very different from how most teachers conceptualize I/we/you. This is not a “gradual release” model — the support varies across the lesson, with students doing plenty on their own very early on. There’s some I/we/you within the lesson, but it doesn’t start with the “I” chunk and I don’t think I/we/you is a good way to describe this type of teaching. I/we/you is a tool. It’s useful at some specific places in a lesson, but not a structure for entire lessons.

Some Benefits

Here are some benefits of the gradual increase in difficulty approach:

• Review is so important. Review to improve retention of what was taught yesterday, to activate prior knowledge, to check prerequisite knowledge and reteach if necessary. When done well, review can also build confidence early in a lesson. I realize there is no rule saying “no review is allowed in I/we/you lessons.” But if the name of the teaching strategy begins with the “I” section, that will inevitably push teachers away from starting with review.

• There are no sudden jumps in difficulty. I’ve seen — and taught — plenty of I/we/you lessons that are a bit of a mess. The jump in difficulty is just too big. The teacher knows it’s a bit of a mess, so the “I” and “we” stages drag on. There’s not much time left for independent practice, half the class is confused, and it all ends in a muddle. I can’t promise you that a gradual-increase-in-difficulty lesson will go smoothly. That’s not how teaching works. But by approaching learning one small step at a time it’s much less likely to fall apart completely. Maybe students are shaky on some of the early steps, and they take longer than you would like. Maybe you don’t get through everything you planned. Much better to teach the first few steps well than to bite off more than you can chew and have to start from scratch the next day.

• There are fewer artificial divisions between objectives. I/we/you lessons break content down into lesson-size chunks. Here is today’s objective, then tomorrow’s, then the next day’s. Unfortunately, learning doesn’t always fit neatly into those chunks. Some days will be too much, some will be too easy. Focusing on a gradual increase in difficulty throws out those distinctions. It’s a gradual ramp, and you get through what you get through in each lesson.

• There are more opportunities to check for understanding. It is possible to check for understanding in the “I” and “we” stages of an I/we/you lesson. But it’s not easy. The entire premise is a gradual release of responsibility, so students aren’t doing the task independently until the “you” stage. A gradual increase of difficulty lesson involves lots of “you” throughout the entire lesson. You have lots of chances to check for understanding, reteach, support specific students, and adjust as you go.

• Students are more motivated when they can successfully solve problems early in a lesson. Get students doing math and building confidence as early as you can, and use that confidence to tackle more challenging problems.

• The teacher support is dynamic. For some tasks, students need a lot of guidance. You can provide that. For others, students can extend what they know with much less support. You can do that too. The key is the gradual increase in difficulty, and that helps to reduce teacher talk and get students doing as much of the thinking as possible.

• It’s easier to interleave. Interleaving different skills is a key aspect of increasing difficulty that’s often forgotten until the review day before the test. Students need mixed practice, identifying different types of questions and applying different solution strategies. In a typical I/we/you structure, each objective lives in its own world. In a gradual increase in difficulty, many of those small steps are just interleaved practice between distinct skills.

• It’s easier to include challenging, non-routine problems. In an I/we/you structure, more challenging problems are often saved until the end of the unit, or for specific objectives like “solve multi-step problems.” In this model, the gradual increase in difficulty means that I can often end classes by challenging students to apply what they’ve learned in a few different and non-routine ways, rather than being laser focused on one very specific objective.

• I spend less time explaining. Lots of explanations go on too long because students don’t have solid prior knowledge, or the jump in difficulty is too big, or the teacher feels like they need to explain everything about a topic before students try it. In a gradual increase in difficulty model, there’s a bunch of work to prepare students for the explanations I do include. But those explanations happen in small chunks that are only slightly more difficult than the last step, and when students can figure something out themselves I let them do so. All that adds up to way less time explaining. I’m not against explaining — teachers should explain things to students! But those explanations should be short, and should get students doing math as soon as possible.

I realize that the difference between a gradual release of responsibility and a gradual increase in difficulty may seem small. But I think it’s a really valuable paradigm shift, and it opens up a ton of opportunities that are lost in a typical I/we/you structure. I/we/you isn’t totally broken — I use it as one element of the gradual increase approach. But it’s just one strategy for me, rather than a way to structure an entire lesson. Whenever I think about teaching a new topic, I start by finding ways to break that topic down into small chunks and sequencing those chunks in a gradual increase in difficulty. It’s not easy as first. But the more I’ve practiced breaking topics down into small, manageable bits, the more this type of teaching makes sense to me.

Deep down in the dark voids of HTML specs long gone sleeps a terrifying thing. Imagine, if you will, a DOM node so mighty, that it can change the content-type of parts of the document. An HTML element that makes the parser tremble and withdraw, and that cannot be stopped even by its own end tag.

The wise people of W3C try to keep the knowledge of this terror away from the mere mortals’ eyes to spare us the danger of its madness. They advise us not to use the magic tag name that is the incantation for this ancient malice.

We will, of course, do exactly this today. We’ll take a deep look at the

<plaintext>

element and what fun things we can use it for.

A Quick Warning

This being said, I’d like to point out one important thing: Do not use this element in production. The HTML living standard is quite clear about this:

Elements in the following list are entirely obsolete, and must not be used by authors: [...] plaintext

So, what does <plaintext> do that earned it its place on HTML’s list of deprecated elements? In a nutshell, it ends the HTML parser and instructs the browser to interpret everything following as plain text.

That is to be taken literally. Really everything, including any closing </plaintext> or </html>, will be printed as if a rogue, unclosed <pre> would suddenly go haywire and slurp up the rest of the page. By the way, this makes <plaintext> the only non-empty element that has no end tag at all.

What Do We Use this Power For?

On first sight that sounds like a really stupid superpower. On second sight, it still does. We look into how that element became part of HTML below. But now we will use it for one specific purpose: debugging server-side code.

Of course, specialized debuggers like XDebug for PHP or built-in error pages in frameworks like Django take over the heavy lifting here. And even the good ol’ print "<script>console.log('here!')</script>" is often helpful. Those tools should be high up in your utility belt.

But imagine this: You are deep in your code, chasing an elusive bug that affects only part of the HTML output, and you want to spot on the rendered page exactly where it shows up. The fastest way is to put a quick <plaintext> close to the offending place, reload the page, and presto! Just scan down to where the markup starts to show through.

This is especially useful to access formatted debugging output. A var_dump() in PHP, for example. Or an error.stack stack trace in NodeJS. Slap a <plaintext> in front of it before writing it to the HTML output, so that the string is immediately readable:

<?php
# TODO delme!
echo '<plaintext>'; var_dump($strange_variable);

If you’re working on an expressJS application, it could look like this:

try {
    some_method();
} catch (error) {
    response.send(`<plaintext>${error.stack}`);
}

The History behind this Evil

How ended this seemingly fringe feature up in all mainstream browsers? It was indeed there from the very beginning of HTML as this historic W3C document from 1992 proves:

Plaintext

This tag indicates that all following text is to be taken litterally [!], up to the end of the file. Plain text is designed to be represented in the same way as example XMP text, with fixed width character and significant line breaks. Format:

<PLAINTEXT>

This tag allows the rest of a file to be read efficiently without parsing. Its presence is an optimisation. There is no closing tag.

This also tells us the reason for its invention. Back at the time Sir Tim Berners-Lee’s high-end NeXT PC that he used to write the first web browser had a quarter of the power of a hand-me-down 2009 smartphone. It was important to optimize wherever you could.

Given that the early WWW was meant as a place to share scientific information, having a large blob of plain text as part of your fancy new HTML page was relatively common. The possibility to end the costly HTML parser and fall back to simply printing the remainder of the file as plain text was a powerful tool.

Such a feature isn’t unique to HTML, either. For example, the programming language Perl uses a special marker to tell the Perl parser to stop processing the remainder of the file:

print 'this is Perl code';
__END__
cout << 'this isn’t anymore';

(Perl programmers use this not for performance reasons but to embed additional data into their programs.)

Of course, nowadays, in the face of multi-megabyte JS payloads, this optimization has become completely unnecessary.

How Safe Are We?

But the element is still looming in all browsers, so it's worth keeping a bit of a working knowledge of it at the back of our minds.

To give you an example of how this feature could be mis-used, assume a comment function on a blog, where the commenter was able to smuggle in the string <plaintext>. As good developers we know never to trust users’ input, so we put the comment through a sanitizer. Let’s take a look at where things can go south from here.

We use the test string

<p><b>hello<plaintext>world!</plaintext></b></p>

to check how several sanitizer libraries react to it.

There Goes the Sanity!

We run each sanitizer in the most minimal configuration that produces any output. This is by design: sanitizers are security products. They should produce safe output by default.

The results are quite surprising, though. (Click on a library’s name below to reveal the horrifying details.) You will notice that no two libraries (apart from the Sanitizer API and DOMPurify, where the former was directly inspired by the latter) agree on how to sanitize our test string.

console.log(Document.parseHTML(TEST_STRING).body.innerHTML);

<p><b>hello</b></p>

const div = document.createElement('div');
div.innerHTML = TEST_STRING;
console.log(div.innerHTML);

<p><b>hello</b></p><plaintext><b>world!&lt;/plaintext&gt;&lt;/b&gt;&lt;/p&gt;</b></plaintext>

Document.parseHTML(TEST_STRING, { sanitizer: { removeElements: []}}).body.innerHTML

echo -n "$TEST_STRING" | tidy

<p><b>hello</b></p>
<pre><b>world!</b></pre>

import xss from 'xss';
console.log(xss(TEST_STRING));

<p><b>hello&lt;plaintext&gt;world!&lt;/plaintext&gt;</b></p>

import { JSDOM } from 'jsdom';
import DOMPurify from 'dompurify';

const purify = DOMPurify(new JSDOM('').window);
console.log(purify.sanitize(TEST_STRING));

<p><b>hello</b></p>

<?php
$config = HTMLPurifier_Config::createDefault();
$purifier = new HTMLPurifier($config);
printf($purifier->purify(TEST_STRING));

<p><b>helloworld!</b></p>

<?php
use Symfony\Component\HtmlSanitizer\HtmlSanitizer;
use Symfony\Component\HtmlSanitizer\HtmlSanitizerConfig;

$config = (new HtmlSanitizerConfig())->allowSafeElements();
$sanitizer = new HTMLSanitizer($config);
printf($sanitizer->sanitize(TEST_STRING));

<p><b>hello</b></p><plaintext>world!</plaintext>

echo -n "$TEST_STRING" | xmllint --html -

<p><b>hello<plaintext>world!</plaintext></b></p>

import bleach
print(bleach.clean(TEST_STRING))

&lt;p&gt;<b>hello&lt;plaintext&gt;world!&lt;/plaintext&gt;</b>&lt;/p&gt;

import org.owasp.html.PolicyFactory;
import org.owasp.html.Sanitizers;

public class Sanitize {
    public static void main(String[] args) {
        PolicyFactory policy = Sanitizers.FORMATTING.and(Sanitizers.LINKS);
        String safe = policy.sanitize(TEST_STRING);
        System.out.println(safe);
    }
}

<b>helloworld!&lt;/plaintext&gt;&lt;/b&gt;&lt;/p&gt;</b>

import nh3
print(nh3.clean(TEST_STRING))

<p><b>hello</b></p><b>world!&lt;/plaintext&gt;&lt;/b&gt;&lt;/p&gt;</b>

With 11 sanitizing methods we managed to produce 10 different outputs!

Just to be crystal clear here: we are not criticizing the result of any of these libraries. Each one has a good reason to do what they do. And each one serves a slightly different purpose.

But it emphasizes the point that one should be absolutely sure about the aim of a chosen sanitizer and the extent as to which it will change its input. Is the library for removing only potentially dangerous things but keeping as much HTML intact as possible? Is it to scrape all HTML off the string, or to only escape HTML-special characters? The results will differ tremendously.

We enter the danger zone when mixing several tools together without taking a cautious look first.

For example, look at how DOMPurify and HTML Purifier would interact in a potentially hazardous way. DOMPurify would remove any <plaintext> including its content. A later check for any malicious payload would be negative. HTML Purifier on the other hand just strips the <plaintext> tag, while its content remains on the page. If we’d trust the previous DOMPurify result, we’d be surprised by sudden new content being placed verbatim in the HTML code.

If one library is used for input validation and another one for output quoting, that’s a cross-site scripting disaster waiting to happen, unless we know exactly what we’re doing.

Letting the Evil Sleep Again

In the case of <plaintext> itself we are most likely in a safe place. Since <plaintext> has built-in HTML escaping, doing something dangerous with it is severely limited. It takes quite rare a constellation of errors and oversights to appear together in order to run malicious code.

For the sake of the argument, let’s create such a constellation. Assume that you embed a Content-Security Policy on your site in a <meta> element instead of an HTTP header:

<meta http-equiv="Content-Security-Policy" content="script-src 'self'">

This prevents loading 3rd party scripts sufficiently. If an attacker finds a possibility to load HTML prior to this element, they can nullify the CSP:

<script src="https://example.com/malicious.js"></script>
<plaintext>
<meta http-equiv="Content-Security-Policy" content="script-src 'self'">

But again, for this to really have any effect, several things must come together:

• The attacker must be able to place HTML in the <head> (because CSP meta tags can only be used there)
• The CSP is not set via HTTP
• The complete remainder of the page is converted to text/plain, which makes this definitively not a stealthy attack

So we can conclude: It is important to know about <plaintext>. But if we follow tried and tested security rules (for example the OWASP Application Security Verification Standard), we will remain safe from this ancient evil.

I’d like to thank Tom Schuster, Christian Vogl, and Daniela Strehl for valuable input to this article and Elise Hein for an extremely helpful review.