Contention: Whether you find our current “AI moment” at least somewhat intellectually stimulating depends largely (if not entirely) on your comfort with metaphors.

I know this will bother some of my more rabid anti-AI readers, but for all many misgivings about the role of AI in society at present, I am firmly and unapologetically curious about this new technology. More than curious, really. I am intellectually excited by the profound questions that AI poses regarding what we mean by intelligence, cognition, learning, abstraction, understanding—the list is long. What’s more, AI demands that we—and by we, I very much mean we humans—really grapple with the metaphors we use to make sense of ourselves, and those we use to help us understand these strange new tools we’ve created.

Recently, I wrote a mini-defense of the metaphor of “stochastic parrots” as an accurate and useful metaphorical description of the core function of generative AI in the form of large-language models. That’s a claim I’m making about generative AI as it exists today. But what about where things are headed? What metaphor should we use as our “north star”—hey look, another metaphor—to guide where this technology should be going? That’s the question I want to explore this week, using an interesting pre-print article published just a few weeks ago as my intellectual foil.

The article: AI Must Embrace Specialization via Superhuman Adaptable Intelligence, lead authored by Judah Goldfelder (currently a PhD student in the “Creative Machines Lab” at Columbia University), with Philip Wyder, Yann LeCun, and Ravid Schwartz-Ziv listed as co-authors—longtime Cognitive Resonance readers may remember my “AI insider spills secrets!” essay that featured LeCun’s many criticisms of LLMs as currently constructed. In any event, this new article is very readable, not overly technical, and offers up a very useful landscape graphic of the many metaphors and definitions hovering around the aspirations of the AI research field at present. To wit (and please pinch to zoom on mobile, sorry it’s so small):

Stefon voice: Adaptive generalists, cognitive mirrors, economic engines, this landscape HAS IT ALL! Jokes aside, I think it’s very helpful summary of the competing “metaphorical visions” within the AI research world about what exactly the goal of the field should be oriented around. On the Y axis, you’ve got the split between those who want to see AI models cash out in the real world on specific tasks (performance) versus those who want to develop AI models that can adapt to new, unforeseen circumstances (“learn” or adaptability).1 Then on the X axis, you’ve got the scope of tasks that AI models might reasonably address, ranging from literally anything to the somewhat more narrowly defined set of things humans are capable of doing.

Reader, I debated spending the next 1,200 words unpacking each of these “north star” definitions of AI, but I’ll forego that for now—if you’re insatiably curious about any of them, hit me up in the comments, and I’ll do my best to provide a pithy summary. Instead, I want to focus here just on “superhuman adaptable intelligence,” the new term offered by Goldfelder et al as to where the field of AI ought to be headed. More specifically, I want to unpack their definition of “SAI” because I both agree with the general thrust of their argument and think it rests on a somewhat myopic understanding of human behaviors and activities.

So what is “superhuman adaptable intelligence”? It’s “intelligence that can learn to exceed humans at anything important we can do, and that can fill in the skill gaps where humans are incapable.” Note my emphasis on learning: At heart, the argument here is that we should give up on trying to create a singular and generally superintelligent (godlike?) AI model, and instead focus on developing more specialized AI systems that can adapt and improve on specific tasks over time. I won’t go into all the technical details here, but to summarize the argument in a sentence, “the AI that folds our proteins should not be the AI that folds our laundry!”

I’m nodding, I’m nodding. And I particularly appreciate the brief burn of large-language models nestled late in the paper: “Homogeneity kills research. Autoregressive large-language models have become the dominant architecture in the state-of-the-art ‘general’ AI space…[but] they have many flaws; their errors diverge exponentially with prediction length, [making] long-horizon interaction brittle.” Here, Goldfelder et al are echoing something Gary Marcus has been arguing for several years now, namely, that for all their impressive linguistic capabilities and the billions (trillions?) being spent on them, LLMs are largely a detour on the road to developing more advanced AI models and systems.

So on the whole, I’m a fan of the “direction of travel” for future AI research that’s offered here. At the same time, and me being me, I am compelled to highlight one of the more amusing passages contained in this paper that, to my reading, is indicative of a significant blind spot that plagues the AI research community, if not all of cognitive science. It arises in the context of their argument that human beings are not actually “generally intelligent” or all that cognitively capable, and it (yet again) involves chess:

Magnus Carlsen is widely regarded as the greatest chess player of all time, and as such represents the pinnacle of human adaptation when it comes to playing chess. But this begs the question: Is Magnus actually any good at chess?

When compared with the best computers, the answer is clearly no. Even more damning is that with modern day computers, creating a program that plays chess at a much higher level than Magnus is not particularly difficult.

Our perception of his ability is colored by the limitations of humanity. Humans in general are bad at chess; Magnus is much better than most humans. The conclusion, that Magnus is good at chess, is a perfect illustration of our own human centric biases. Magnus Carlsen is not objectively good at chess, he is good at chess with respect to human performance levels.

By any objective metric, playing chess at a much higher level is not difficult from a computational perspective, but it is something that humans are incapable of.

It is undeniably true that chess engines today are more powerful than any humans at calculating chess moves, and thus can generate “solutions” that humans are incapable of. But invoking the “objective metric” of computational prowess as so-called proof that Magnus Carlsen is “not objectively good at chess” is a category error—humans play chess because it’s fun and interesting, it’s the sport of human thinking. As is true with all human games, nothing of any real importance happens as a consequence of winning, the satisfaction instead arises from the experience itself.

What’s more, and I say this as someone who’s spent way too many hours watching Daniel King analyze games on his Power Play Chess channel (he’s great), the capabilities of chess engines are often treated as a semi-interesting sideshow to the far more absorbing endeavor of exploring human-versus-human competition. When commenting on a game, King will often observe, “the computer thinks this is the best move but good luck to any human at seeing that”—and then he’ll go right back to his thoughtful and very human analysis of the game at hand. That’s what we humans like to do, be curious about our fellow humans.2

In my view, AI as a research field, to say nothing of all of cognitive science, suffers from obsessive commitment to what my late-in-life mentor Jim March described as the “consequentialist paradigm,” the view that “our beliefs require that action be justified in terms of ambition for its consequences.” Sometimes that perspective is warranted, no doubt—but not always. Sometimes, we pursue things not because of an outcome we hope to achieve but because the pursuit itself generates meaning, in ways that are entirely subjective, often ephemeral, and we might even say irrational. Why does Don Quixote tilt at windmills, March asks? Because “he embraces the foolishness of obligatory action. Justification for knight-errantry lies not in anticipation of effectiveness but in an enthusiasm for the pointless heroics of a good life. The celebration of life lies in the pleasures of pursuing the demands of duty.”3

Artists, poets, musicians, those who spends time dwelling in the depths of the humanities—and please do note the root of that word—they all naturally intuit this, I think. There is a reason that so many in the artistic community are horrified by AI, and it’s not solely because of the misappropriation of human labor and creativity (though this must not be overlooked). It’s because they see AI as part of a continued technological erosion of the celebration of life that arises from poetic pursuits. The pursuit of artificial intelligence, whether superhuman or adaptable or specialized or whatever else, could be part of the humanities, broadly construed, but only if researchers can resist slapping metrics on everything and declaring that all that matters is what’s measured. Life is not a data set!

I recur to March once more to close us out, and please do take a few minutes to ruminate on this:

We have grown to accept the modern idea that human actions, institutions, and traditions are justified by expectations of their consequences; and we have come to view optimism without hope as an unfortunate surrender to Pollyanna. But education unconditionally celebrates life. It is an arbitrary assertion of optimism. It echoes an ancient conception by which I do what I do because that is what is appropriate to my nature; and my nature is not simply a tautological summary of what I do but an understanding of the essence of my destiny, an interpretation of my history, and an assertion of my humanity.

Speaking of art and assertions of humanity, several months ago Tomáš Baránek, a publisher in the Czech Republic, reached out to me in hopes of translating my Large Language Mistake essay into Czech—there are few things more flattering to a writer than someone wanting to share it with an audience in another language (unfortunately, The Verge’s legal team put the kibosh on that plan).

Sadly, Baránek’s mother Vlasta Baránková recently passed away, but through our correspondence he shared with me her book of gorgeous and haunting artwork—she was an accomplished artist and illustrator of many childrens’ books, including some I’m sure I read as a child. Tomáš also shared with me that she had to maneuver around communist-era restrictions in order to publish her work. As he observes, “this is how real artists and creative people live and die. They cannot help themselves, they enjoy the creativity and its painfulness.”

Indeed. And you will see that creative beauty and pain reflected in the work of Vlasta Baránková, who’s work I share here with permission. She may be gone, but her exploration of the human spirit remains with us. I grieve with you, Tomáš, but I am so grateful to know of your mother and the contributions she made to our shared humanity.

Subscribe now

I have always enjoyed the act of typing words and seeing them come up on screen. While my favorite word processor of all time might be WordPerfect (here), I've used almost all of them. These programs were what sold me on the entire value proposition of computers. They were like typewriters, which I had used in school, except easier in every single way. You could delete things. You could move paragraphs around. It felt like cheating, and I loved it.

As time has gone up what makes up a "document" in word processing has increased in complexity. This grew as word processors moved on from being proxies for typewriters and into something closer to a publishing suite. In the beginning programs like WordPerfect, WordStar, MultiMate, etc had flat binary files with proprietary formatting codes embedded in there.

When word processors were just proxies for typewriters, this made a lot of sense. But as Microsoft Word took off in popularity and quickly established itself as the dominant word processor, we saw the rise of the .doc file format. This was an exponential increase in complexity from what came before, which made sense because suddenly word processors were becoming "everything tools" — not just typing, but layout, images, revision tracking, embedded objects, and whatever else Microsoft could cram in there.

The .doc: A Filesystem Inside Your File

At its base the .doc is a Compound File Binary Format, which is effectively just a FAT file system with the file broken into sectors that are chained together with a File Allocation Table.

It's an interesting design. A normal file system would end up with sort of a mess of files to try and contain everything that the .doc has, but if you store all of that inside of a simplified file system contained within one file then you could optimize for performance and reduced the overhead that comes with storing separate objects in a flat file. It also optimizes writes, because you don't need to rewrite the entire file when you add an object and it keeps it simple to keep revision history. But from a user perspective, they're "just" dealing with a single file. (Reference)

The .doc exploded and quickly became the default file format for humanity's written output. School papers, office memos, résumés, the Great American Novel your uncle was definitely going to finish — all .doc files. But there was a problem with these files.

They would become corrupted all of the goddamn time.

Remember, these were critical documents traveling from spinning rust drives on machines that crashed constantly compared to modern computers, often copied to floppy disks or later to cheap thumb drives you got from random vendor giveaways at conferences, and then carried to other computers in backpacks and coat pockets. The entire workflow had the structural integrity of a sandwich bag full of soup.

Your hard drive filesysystem -> your .doc file (which can get corrupted as a file) -> containers a mini filesystem (which can ALSO get corrupted internally) -> manages your actual document content

So when Word was saving your critical file, it was actually doing a bunch of different operations. It was:

• Updating the document stream (your text)
• Updating the formatting tables
• Update the sector allocation tables
• Update the directory entries
• Update summary information
• Flush everything to disk

These weren't atomic operations so it was super easy in an era when computers constantly crashed or had problems to end up in a situation where some structures were updated and others weren't. Compared to like a .txt file where you would either get the old version or a truncated new version. You might lose content, but you almost never ended up with an unreadable file. With .doc as someone doing like helpdesk IT, you constantly ended up with people that had just corrupted unreadable files.

And here's the part that really twisted the knife: the longer you worked on the same file, the more important that file likely was. But Word didn't clean up after itself. As a .doc accumulated images, tracked changes, and revision history, the internal structure grew more complex and the file got larger. But even when you deleted content from the document, the data wasn't actually removed from the file. It was marked as free space internally but left sitting there, like furniture you moved to the curb that nobody ever picked up.

The file bloated. The internal fragmentation worsened. And the probability of corruption increased in direct proportion to how much you cared about the contents.

Users had to be trained both to save the file often (as AutoRecover wasn't reliable enough) and to periodically "Save As" a new file to force Word to write a clean version from scratch. This was the digital equivalent of being told that your car works fine, you just need to rebuild the engine every 500 miles as routine maintenance.

The end result was that Microsoft Word quickly developed a reputation among technical people as horrible to work with. Not because it was a bad word processor — it was actually quite good at the word processing part — but because when a user showed up at the Help Desk with tears in their eyes, the tools I had to help them were mostly useless.

I could scan the raw file for text patterns, which often pulled out the content, but without formatting it wasn't really a recovered file — it was more like finding your belongings scattered across a field after a tornado. Technically your stuff, but not in any useful arrangement. Sometimes you could rebuild the FAT or try alternative directory entries to recover slightly older versions. But in general, if the .doc encountered a structural error, the thing was toast and your work was gone forever.

This led to a never-ending series of helpdesk sessions where I had to explain to people that yes, I understood they had worked on this file for months, but it was gone and nobody could help them. I became a grief counselor who happened to know about filesystems. Thankfully, people quickly learned to obsessively copy their files to multiple locations with different names — thesis_final.doc, thesis_final_v2.doc, thesis_FINAL_FINAL_REAL.doc — but this required getting burned at least once, which is sort of like saying you learned your car's brakes didn't work by driving into a bus.

Enter the XML Revolution

So around 2007 we see the shift from .doc to .docx, which introduces a lot of hard lessons from the problems of .doc. First, it's just a bundle, specifically a ZIP archive.

my_document.docx (renamed to .zip)
├── [Content_Types].xml
├── _rels/
│   └── .rels
├── word/
│   ├── document.xml        ← the actual text content
│   ├── styles.xml          ← formatting/styles
│   ├── fontTable.xml
│   ├── settings.xml
│   └── media/
│       ├── image1.png      ← embedded images
│       └── image2.jpg
└── docProps/
    ├── app.xml
    └── core.xml            ← metadata

Now in theory, this is great. Your content is human-readable XML. Your images are just image files. If something goes wrong, you can rename the file to .zip, extract it, and at least recover your text by opening document.xml in Notepad. The days of staring at an opaque binary blob and praying were supposed to be over.

However, in practice, something terrible happened. Microsoft somehow managed to produce the worst XML to ever exist in human history.

Let me lay down the scope of this complexity, because I have never seen anything like it in my life.

Here is the standards website for ECMA-376. Now you know you are in trouble when you see a 4 part download that looks like the following:

• Part 1 “Fundamentals And Markup Language Reference”, 5th edition, December 2016
• Part 2 “Open Packaging Conventions”, 5th edition, December 2021
• Part 3 “Markup Compatibility and Extensibility”, 5th edition, December 2015
• Part 4 “Transitional Migration Features”, 5th edition, December 2016

If you download Part 1, you are given the following:

Now if you open that PDF, get ready for it. It's a 5039 page PDF.

I have never conceived of something this complicated. It's also functionally unreadable, and I say this as someone who has, on multiple occasions in his life, read a car repair manual cover to cover because I didn't have anything else to do. I once read the Haynes manual for a 1994 Honda Civic like it was a beach novel. This is not that. This is what happens when a standards committee gets a catering budget and no deadline.

There was an accusation at the time that Microsoft was making OOXML deliberately more complicated than it needed to be — that the goal was to claim it was an "open standard" while making the standard so incomprehensibly vast that it would take a heroic effort for anyone else to implement it. I think this is unquestionably true. LibreOffice has a great blog post on it that includes this striking comparison:

The difference in complexity between the document.xml and content.xml files is striking when you compare their lengths: the content.xml file has 6,802 lines, while the document.xml file has 60,245 lines, compared to a text document of 5,566 lines.

So the difference between ODF format and the OOXML format results in a exponentially less complicated XML file. Either you could do the incredible amount of work to become compatible with this nightmarish specification or you could effectively find yourself cut out of the entire word processing ecosystem.

Now without question this was done by Microsoft in order to have their cake and eat it too. They would be able to tell regulators and customers that this wasn't a proprietary format and that nobody was locked into the Microsoft Office ecosystem for the production of documents, which had started to become a concern among non-US countries that now all of their government documents and records were effectively locked into using Microsoft. However the somewhat ironic thing is it ended up not mattering that much because soon the only desktop application that would matter is the browser.

Rise of Markdown

The file formats of word processors were their own problems, but more fundamentally the nature of how people consumed content was changing. Desktop based applications became less and less important post 2010 and users got increasingly more frustrated with the incredibly clunky way of working with Microsoft Word and all traditional files with emailing them back and forth endlessly or working with file shares.

So while .docx was a superior format from the perspective of "opening the file and it becoming corrupted", it also was fundamentally incompatible with the smartphone era. Even though you could open these files, soon the expectation was that whatever content you wanted people to consume should be viewable through a browser.

As "working for a software company" went from being a niche profession to being something that seemingly everyone you met did, the defacto platform for issues, tracking progress, discussions, etc moved to GitHub. This was where I (and many others) first encountered Markdown and started using it on a regular basis.

John Gruber, co-creator of Markdown, has a great breakdown of "standard" Markdown and then there are specific flavors that have branched off over time. You can see that here. The important part though is: it lets you very quickly generate webpages that work on every browser on the planet with almost no memorization and (for the most part) the same thing works in GitHub, on Slack, in Confluence, etc. You no longer had to ponder whether the person you were sending to had the right license to see the thing you were writing in the correct format.

This combined with the rise of Google Workspace with Google Docs, Slides, etc meant your technical staff were having conversations through Markdown pages and your less technical staff were operating entirely in the cloud. Google was better than Microsoft at the sort of stuff Word had always been used for, which is tracking revisions, handling feedback, sharing securely, etc. It had a small subset of the total features but as we all learned, nobody knew about the more advanced features of Word anyway.

By 2015 the writing was on the wall. Companies stopped giving me an Office license by default, switching them to "you can request a license". This, to anyone who has ever worked for a large company, is the kiss of death. If I cannot be certain that you can successfully open the file I'm working on, there is absolutely no point in writing it inside of that platform. Combine that with the corporate death of email and replacing it with Slack/Teams, the entire workflow died without a lot of fanfare.

Then with the rise of LLMs and their use (perhaps overuse) of Markdown, we've reached peak .md. Markdown is the format of our help docs, many of our websites are generated exclusively from Markdown. It's now the most common format that I write anything in. This was originally written in Markdown inside of Vim.

Why It Won

There's a lot of reasons why I think Markdown ended up winning, in no small part because it solved a real problem in an easy to understand way. Writing HTML is miserable and overkill for most tasks, this removed the need to do that and your output was consumable in a universal and highly performant way that required nothing of your users except access to a web browser.

But I also think it demonstrates an interesting lesson about formats. .doc and .docx along with ODF are pretty highly specialized things designed to handle the complexity of what modern word processing can do. LibreOffice lets you do some pretty incredible things that cover a huge range of possible needs.

Markdown doesn't do most of what those formats do. You can't set margins. You can't do columns. You can't embed a pivot table or track changes or add a watermark that says DRAFT across every page in 45-degree gray Calibri. Markdown doesn't even have a native way to change the font color.

And none of that mattered, because it turns out most writing isn't about any of those things. Most writing is about getting words down in a structure that makes sense, and then getting those words in front of other people. Markdown does that with less friction than anything else ever created. You can learn it in ten minutes, write it in any text editor on any device, read the source file without rendering it, diff it in version control, and convert it to virtually any output format.

The files are plain text. They will outlive every application that currently renders them. They don't belong to any company. They can't become corrupted in any meaningful way — the worst thing that can happen to a Markdown file is you lose some characters, and even then the rest of the file is fine. After decades of nursing .doc files like they were delicate flowers that you had to transport home strapped to your car roof, the idea of a format that simply cannot structurally fail is not just convenient. It's a kind of liberation.

I think about this sometimes when I'm writing in Vim at midnight, just me and a blinking cursor and a plain text file that will still be readable when I'm dead. No filesystem-within-a-filesystem. No sector allocation tables. No 5,039-page specification. Just words, a few hash marks, and never having to think about it again.

I lurk in a Facebook group called "Photos of Evansville, Indiana." In it, photographers of various skill levels share photos taken around our city.

Two days ago, a member shared this photo taken across the street from Reitz Memorial High School:

Image Credit: Ted Byrom, via the "Photos of Evansville, Indiana" Facebook group

It's terrible, and I love it.

I could totally see it as a pop art painting in our local museum. Art teachers would ask the students:

• "What do you think the artist was trying to portray?
• "What does the hand sanitizer represent?"
• "What about the Road Closed sign?"
• "Notice how the composition hat-tips the classical concept of a vanishing point while boldly departing from the typical usage!"
• "Likewise with the pillar. It follows the rule of thirds by placing something on the line, but breaks the rule by making it entirely insignificant. What did the artist intend to imply by this?"

I like how the hand sanitizer sits in the foreground and immediately catches your eye as if it's the subject of the photo. Then you realize that it's out-of-focus, and move up to the stop sign. But that's... also not an interesting subject. From there, the composition draws you to the Road Closed sign, which isn't very interesting either. The photo is almost interesting, but lacks a clear subject. It only leaves you confused.

Lose the hand sanitizer and add a girl walking by with an boldly-colored umbrella, and it would work just fine. Or keep the hand sanitizer, add an N95 mask, and post it during COVID as a commentary. Five years ago, the stop sign and construction barrier could have been meaningful:

"Life has stopped, and all roads to happiness are blocked. I feel stagnant, and all I can do is keep washing my hands and hoping to survive it"

Obviously, this is just someone's snapshot of a view that he liked, but couldn't convey. He accidentally managed a composition that fell into the uncanny valley between "Just someone's snapshot" and "Art that's trying to say something", and it's hilarious.

I recently came across the following joke map online:

Surprisingly, and probably unintentionally, there’s a grain of historical truth to this “Megachusetts”. Massachusetts’ original colonial charter specified northern and southern borders, and that the colony should extend all the way from the Atlantic Ocean to the Pacific Ocean, or rather: “…from the Atlantick and Westerne Sea and Ocean on the East Parte, to the South Sea on the West Parte;” The grant wasn’t quite the line depicted in the meme, but it was a line across the continent nevertheless.

Massachusetts wasn’t the only New England colony to get such a “sea to sea” grant. Connecticut’s colonial charter of 1662 granted the young colony all English territory “…from the said Narrogancett Bay on the East to the South Sea on the West parte, with the Islands thereunto adioyneinge…”

Besides the creative spellings of “Narragansett” and “adjoining” (from before standardized English spelling), this line gave Connecticut a basis to claim to its own narrow strip of entire continent, again hypothetically reaching the Pacific ocean. And like any reasonable government of the era, the colonial governments of Connecticut and Massachusetts kept on claiming this land as long as they feasibly could.

Several colonies had western claims along these lines. New York and Virginia claimed the most western land, but North Carolina, South Carolina and Georgia all had claims, too. Many of these seven state claims overlapped. The southern states in particular actually had access to land past the Appalachians, and several of them tried to actually settle those areas and enforce their claims after American independence.

Connecticut and Massachusetts, however, had no way to enforce most of their claims. The clearest issue was that French Louisiana and Spanish New Spain were in the way of the westernmost section, but also Pennsylvania and New York territory cut off direct access New England the old Northwest. (And of course, there were numerous indigenous nations in the territory as well, but in the Americans’ minds they were going to be conquered eventually either way.)

But even without direct access Connecticut didn’t entirely give up on its long western extension. States of the young USA started ceding their western land claims to the federal government in the decades following independence. This kept the peace between states and gained goodwill from the union, and made it easier for Americans to expand into the area. Connecticut ceded most of their claim, too but held onto a small piece of what would become Northeast Ohio.

Connecticut even started sending settlers to the area in the 1790s, under the auspices of the “Connecticut Land Company”. A few towns were even founded, most notably one on the banks of the Cuyahoga river by a certain Moses Cleaveland. Even today, Cleveland, Ohio¹ maintains its original New England-style common grazing land as its central plaza, Public Square.

This area, known as the “Connecticut Western Reserve”, eventually was ceded to the federal government as well. It was just too impractical for Connecticut to manage an exclave on the other side of the Appalachians. But the legacy of this last bit of Long Connecticut (Disconnecticut?) remains, most notably in the name of Case Western Reserve University in Cleveland.

Coming soon: The elementary school in a nuclear bunker

On the afternoon of February 2nd, I received an email from a student researcher named Angel Nulani. She had found 40 social media accounts with AI-generated black women that perpetuated racist, anti-black behavior. Coincidentally, earlier that day, I had received an email from Sharihan Al-Akhras from BBC News Arabic. She was working on a piece about an impossibly-black AI-generated character that stole real videos and “reskinned” them with AI.

Together, Riddance embarked on an investigation in collaboration with the BBC that uncovered over 100 social media accounts that depict the likenesses of Black women through AI-generated characters.

Research revealed accounts running out of 34 countries across North and South America, Europe, Africa, and Asia. They tally a cumulative 19.4 million followers across Instagram and TikTok. The majority (68 accounts) actively promote monetization on sites where they sell explicit AI-generated media.

AI-generated videos are especially adept in race-to-the-bottom trends because creators can avoid accountability. They push limits to get attention. But by depicting black women, they perpetuate a long history of exploiting black people, using caricatures and self-hating characters to attract morbidly curious or racist audiences.

By identifying and examining these accounts, we can start to find solutions for this quickly-growing problem. With AI media improving rapidly, solutions must keep up, otherwise this AI-perpetuated exploitation will only get worse.

How AI advancements scaled digital blackface

Our sample of 100 accounts is representative of both big and small accounts. Of the 100 accounts, 63 were exclusive to Instagram, 4 were exclusive to TikTok, and 33 had pages on both. This means that tracked 136 pages total.

Most of the Instagram accounts were created between mid-2025 and early 2026. A third were created in the past three months alone, and 60 changed their usernames at least once, meaning they “pivoted” to AI-generated black women. The trend is clearly demonstrated in the graph below, measuring when the Instagram accounts were created and if they still have their original usernames.

Google released the Veo 3 AI video model in May, and companies like Kling and Runway unveiled competing video models shortly after. AI-generated videos flooded Instagram and TikTok at this time, as creators experimented with AI slop but lacked clear monetization plans. Once their accounts grew, they looked towards profitable AI niches.

All 100 accounts had to satisfy three criteria:

• They use entirely AI-generated media

• They depict black women (including numerous albino and vitiligo accounts)

• They are sexual in nature

The explosion of accounts from late 2025 through today directly correlates with the releases of open-source AI generation models: FLUX-2, an AI image model launched in November, and Alibaba’s Wan 2.2 Animate Replace, an AI video model released in September. FLUX-2 can generate sexual images, and Wan 2.2 can “animate” them. This combination lets AI creators generate realistic, consistent AI videos without guardrails.

Subscribe now

How the accounts depict black women

Many of the AI-generated characters demonstrate explicit, internalized anti-blackness. They may state a desire to be European (particularly Slavic) or ask if they’re “good enough” for white attention. Women with vitiligo and albinism are characterized as having one white parent, outright calling themselves white or half-white. At least three accounts have made references to the n-word, and one gives its followers an “n-word pass.”

Women are often depicted kissing famous white men like Donald Trump, John Cena, or Johnny Sins (a white male porn actor). Some accounts post about Jeffrey Epstein, with one character attempting to defend him. (35) of the accounts include race-play terminology in their usernames, like “black”, “white”, “vanilla”, “charcoal”, “dark”, “ebony”, or “noir”.

Many accounts fetishize aspects of Black culture and the Black diaspora while having no understanding of them. Black American naming conventions are used for African characters. African accounts often have no specified ethnicity. And when they try to give characters a cultural background, they get basic information wrong. For example, Carolamater, an account based in Lithuania, depicts a Mursi woman with light skin and box braids. The real Mursi people, from southern Ethiopia, are known for their distinctive lip plates and body painting.

And then, there are the characters who have impossibly-black skin.

Not normal

What does it mean to have “impossibly-black” skin?

Part of my job is to perfectly dial-in accurate skin tones on camera. This helps me identify exactly what’s wrong with these characters. Any real human, light or dark-skinned, has skin tones falling on a vector of various red-orange hues. This is clearly demonstrated by a vectorscope, which analyzes color and saturation.

Below, I’ve highlighted four dots representing different skin tones. These all cluster on the top-left vector around what is known as the “skin tone line”, which slopes up and to the left.

Impossibly-black characters, on the other hand, land almost directly in the middle of the vectorscope, which means there is no color saturation in their skin. Notably, this is a separate measurement from the amount of light reflected off the skin, which can also vary. The resulting characters display a range of dark grey to pitch-black skin.

Historically racist portrayals of Black people, like caricatures or minstrel shows, often painted their faces literally black. And if it’s not obvious, AI-generated characters will have normal skin tones by default. Human creators have to deliberately try to make impossibly-black characters. In testing, we have confirmed that prompting FLUX-2 for “very dark, pitch black skin with no undertones” will yield impossibly-black skin.

Nianoir is a character with impossibly-black skin. It had the most followers of all 100 accounts, with 3.1 million on TikTok alone — 2.7 million more than the next largest TikTok account. The account was removed from TikTok after our request for comment, but remains up on Instagram (we’ll dive into the platform responses later).

Their bio says “Just a girl with dark side... 🖤” [Grammar incorrect]. The Ukraine-based account makes many of their videos by stealing content from real creators, then replacing them with the Nianoir character. The AI video model Wan 2.2 Animate Replace does this by design.

The BBC interviewed Riya Ulan, a young model from Malaysia, who had her own videos stolen by Nianoir. Ms. Ulan was stunned and tried to take action, but was unable to get the stolen content taken down until the BBC reached out.

The sad irony is that Nianoir’s comment sections are full of people posting screenshots from the videos Nianoir stole from — often from white or Asian creators like Ms. Ulan — which are met with accusations that these creators are the ones whitewashing.

Organized, international operations

AI-generated media is about quantity before quality, so operators often run multiple accounts. Sometimes, this turns into full operations with teams of people working in content farms.

One operation, based in the United Kingdom, runs ten accounts, including four of the top ten in total followers. Their accounts totaled almost 3.5 million followers, though there’s significant follower overlap. These accounts follow one another, “collaborate” between characters, and some use the exact same character.

Their accounts include zurilovesvanilla, the fourth-most-followed account of all 100, and ayannasoblack, an impossibly-black character. Naledi_the_white_girl is an albino black character who once posted that she “still commits 50% of the crime.”

Many accounts had at least one former username, showing this group was making other AI content before pivoting to Black characters. Eighteen username changes before landing on “zurilovesvanilla” tells you this wasn’t their first idea, but it was their most profitable one.

How do they make money? The majority of accounts (68 of the 100) promote monetization links to sites where they sell AI porn. You can assume that the other 32 are waiting to get big enough to monetize. None of the content on social media is explicit, so the objective is to capture an audience on an social media, then filter the “whales” — a term from betting meaning high-spending individuals — to the monetized sites.

A group based in Kosovo demonstrates the recency of the trend. They run seven accounts and focus on dark-skinned, curvy avatars, with a couple (shaykelt and nyxkash) having the same exact avatar. Their accounts combined to a total of 680,000 followers. Every account was created in the past two months and has no username changes. This group identified the niche and went all-in right away.

What platforms can do

I wasn’t expecting Instagram or TikTok to proactively remove many of these accounts, and for the most part, I was right. Over three weeks of collecting and tracking accounts, a few were taken down. But this was insignificant compared to the new accounts that popped up every day.

Then, Riddance and the BBC independently reached out to TikTok and Instagram for comment, and provided both with lists of accounts.

Within three days, TikTok banned 20 of 37 accounts, including removing Nianoir.xo, the single largest account on either platform. Several accounts belonging to the UK group (onlyskyewhite, zurilovesvanilla, emma.cynder) and the Kosovo group (nyxkash, abbyblacki9) were taken offline entirely. TikTok also applied AI-generated labels to the videos we shared. These were significant steps, taken quickly.

Instagram took down just 2 accounts: ayannasoblack and naledi_the_white_girl, both belonging to the UK content farm. Nineteen other accounts were age-gated, meaning they went from fully public to requiring digital ID to view. The remaining 59 accounts are still fully public and unchanged.

When asking TikTok which policy those 18 deleted accounts had violated, they cited their policy that prohibits “pretending to be a fake person or organization with the goal of misleading people.”

Instagram’s version of this policy, “Authentic Identity Representation,” prohibits accounts “created or used to deceive others.” But in practice, the policy targets accounts that impersonate real people, or networks engaged in coordinated manipulation. It doesn’t appear to be designed for scenarios like this, when someone creates an entirely fictional persona.

95 of the 100 accounts did not disclose they are AI-generated anywhere on social media. None of them labeled individual posts as AI-generated, which both TikTok and Instagram require. In theory, when either detect a C2PA “watermark” that indicates a video is AI-generated, they can label it as such. But the open-source AI models powering this surge, FLUX-2 and Wan 2.2, don’t embed C2PA metadata.

TikTok stated that over 1.3 billion videos have been labeled as AI-generated to date. They also stated that 98% of the videos TikTok removed for violating edited media and AIGC policies, and 90% that violated sexual activity and services policies, were removed proactively in the most recent reporting period. Instagram declined to comment.

Instagram’s hateful conduct policy explicitly bans harmful stereotypes “historically linked to intimidation or violence, such as Blackface.” The impossibly-black characters documented in this article — characters with no undertones, no color saturation, skin that reads as literal black — are digital Blackface. In my opinion, Instagram’s written policy should cover them.

What’s missing?

Platforms should directly address AI-generated impersonation of marginalized groups. A user who encounters an impossibly-black AI character posting watermelon memes has to decide whether to report it as “hate speech” (which doesn’t capture the AI deception), “spam” (which misses the harm), or “AI-generated content” (which also misses the harm). The reporting categories don’t reflect the problem.

As agentic AI content pipelines become more common, we’ll need automated and human interventions. Automated systems don’t have the depth of social awareness needed to catch what’s happening here. So while platforms will naturally focus on automated, software-based solutions, they should work with third parties with human investigators to help out.

If you come across harmful content, report it for hate speech or sexual exploitation so platforms have that data. It might feel like the reporting goes unnoticed, but in aggregate it may help.

But while AI detection is still possible for some people, the quality improvements of AI media will outpace detection in the long run. The accounts in this investigation are already more convincing than they were six months ago. Platforms must act now by banning accounts and finding algorithmic patterns outside of the videos themselves.

“We’re so beautiful”

Angel Nulani, the student researcher who brought many of these accounts to our attention, did hours of research for this project. As a Black woman herself, this investigation was emotionally taxing. We asked her to be an editor and contributor to the piece. Angel is not a journalist or investigator (yet), but a student with an incredible drive and attention to detail. We assumed she was already a working professional.

As the investigation neared its conclusion, we asked her about how the content and investigation had affected her. We decided to include her response in full.

What upsets me is not that these characters are self-hating, but that there is no “self.” For the majority of the people who are behind these accounts, the only Black people they know are the women they generated. They were not born Black. They chose to be Black, and yet they spend so much time distancing themselves from it.

An ‘all lives matter’ shirt can be paired with booty shorts; crime statistics can be overlaid on thirst traps; videos crying over their features are spliced between suggestive yoga poses.

It’s not enough to pander to people that say “ebony” more than “African” because the appeal is not simply a Black woman desiring white men. The appeal is in a Black woman desiring a complete absorption into whiteness. The men they attract are not suitors, they are saviors.

I’ve tried so hard to disassociate. As a Black woman that’s been online since I was young, I assumed I saw it all. Nothing could really hurt me. At first, all I wanted was to be objective, and rational, and done with it. But this cuts me down to my core.

Ironically, that’s even more confirmation they could never be us, because when you’re real there’s no separating yourself from this. The watermelon emojis are par for the course for them; they’re not for me.

I’m terrified to think there’s a little Black girl somewhere feeling as terrible as I do right now because some guy in Malta wanted €3 a month.

They make us seem so ugly, but we’re beautiful. We’re so beautiful.

During my first year at Figma, I designed and printed a run of posters for the office titled “Design is more.” The idea was to highlight that UX design is more than people expect, and connected in interesting ways to other domains. Today, they feel like a spiritual predecessor to this blog.

The first series was three posters:

I still (mostly) like them. I do believe that software can learn more about conveyance from video games; a lot of first-run experiences and particularly new feature onboarding still feel like a series of random pop-ups floating around the screen without much understanding of me as a user.

I would rewrite these posters, however, and particularly the Fitts’s Law examples: they’re generic and probably not as relevant to today’s applications.

After series one, we also collaboratively started working on series two, but the pandemic put a halt to the effort, and these posters were never finished/​printed. But the two below were perhaps closest to ready, and they seem fun today; I particularly liked the joke on the Hick’s Law one.

Jon Yablonski, the author of “Laws of UX,” made some posters in a similar vein and they’re available for purchase. His are slightly more on the visual side, but I was delighted to discover today that we both chose a rather similar approach to visualizing the Zeigarnik Effect.

(200th blog post here!)