I know the technology, I understand what it's doing and I know the impact, so I am vehemently anti-AI.

I do not believe any positive outcome is possible with this form of AI that is worth the harms that it has already done and is continuing to do. Nor do I believe that “just one more model bro” will make something that is “intelligent”. Of course “AI” is a marketing term so it doesn't mean anything, neither does GenAI or the latest buzzword “Agentic”.

This makes me an outcast. In tech, and out of it.

A post was sent to me that really resonated. On the acceptance of GenAI

I am so tired of defending this stance that I am reaching the point of cutting entire communities out of my life because they will promote[1] AI usage.

People do not realise how much of a toll it takes on you if you actually care about the environment, exploited workers, theft from the people who can least afford it, the impact on people's cognitive skills, the centralisation of power, the spread of disinformation, the ruination of the web and/or the destruction of entire career paths (not billionaire of course, that's always a safe one), and not endorsing (either distinctly or tacitly by using) AI.

If you're not in this group with me, you see the adverts for it everywhere, from emails to TV to posters that have clearly been created with it and probably just roll your eyes or have a sensible chuckle about the rediculousness of it. For me it's actually stomach-turning to see people I respected endorsing or using it openly.

A theatre group I was with decided to take a picture of the group and make a “band poster” of us. Great, that's fun. Oh, they used ChatGPT and now I feel sick and they didn't even ask me.

A friend asked “siri” how long a medication is effective for and when it responded “would you like me to use chatGPT to find the answer” he just said “yes” as if he'd said yes a thousand times already, and just accepted the response as right. I don't want to hang out with him any more because he'll have his phone with him and it's automatic for him now. Also I hope he doesn't die because he trusts some made-up stuff it tells him.

I walked out of a presentation the other day because they were somehow making a point of how bad AI is, by arguing with copilot live on stage. Great, so you acknowledge how bad it is by.... using it?!

Do you like using wikipedia? Cool, I work for one of the chapters behind the site. Guess what? People are using AI to get the information that was scraped from wikipedia and accepting it, “hallucinations” and all. Guess what none of those people do? Become editors, fix mistakes, keep the thing alive.

Worse still, AI is gaslighting you into believing it works. That's the whole point of a model that is designed to provide output that “seems reasonable”. Power users know to argue with it, whereupon it will “admit” that it was wrong, and pipe that to another AI ad infinitum to try and reduce the “made up” stuff. Regular people don't.

So yeah, I'm an outcast, and yeah, it sucks, I'm done with it, I can't take “a week off” from the constant barrage of the stuff. Funny how if it were actually a technology for good, it wouldn't be advertised everywhere.

Yes, some people are forced to use it at work, they have my sympathy. Yes, some people need to use it because for one reason or another if they don't they won't survive. I do not judge you for that.

If you know all the harms (see the post linked to above) and use it “because it's convenient” in spite of all of that, damn right I'll judge you (mostly silently). I won't do more than attempt to provide you with the information of the harm it's doing and if you continue to use it after I've done that, avoid interacting with you.

If you know the harms and do something like “I can get a bonus for being the person who uses the most tokens so I'll just make an infinite loop between two AI agents” I will most likely really dislike you for putting your self-gain over the impact and cut you out of my life.

If you push people to use it however subtly (e.g. “You should just use copilot for that, it's really so much easier”), I will avoid you at all costs. If the place you're posting such to doesn't have rules against that, I'll probably leave that place. I might try to encourage that kind of rule, I might just vanish (depending on the group dynamics).

Does that make me unreasonable? Maybe? I will not change my morals or ethics to suit someone else, nor do I expect other people to change theirs.

Does it mean I loose friends and influence and have a good cry now and then as I close chapters of my life because people don't have the same moral structure as me? Definitely.

[1] As in “You should rub some AI on that”, “Claude is great for that”, “no apps are good for this situation, use AI to make your own” or similar. Not “Hey, Claude is cheap right now, go buy some tokens!”. This usage of promote is not common in the US it seems.

I started to consider a theory while reporting on accounts that used AI-generated videos with young girls’ likenesses. During my research, I created multiple burner accounts and “seeded” their recommendation algorithms by interacting with AI accounts. Instagram’s Home page would recommend comparable AI-generated videos from similar accounts.

The Riddance team has used this technique on multiple investigations since, and a pattern is forming. In specific content categories where AI-generated accounts cluster, recommendation algorithms appear to have a latent ability to detect AI-generated videos. We can take advantage of this with what I call the recommendation feedback loop.

Specifically, Instagram’s recommendation algorithm groups AI content together, prioritizes AI content once it figures out you want more of it, and keeps serving it without any explicit AI-generated tagging or metadata.

I will explain our anecdotal evidence thoroughly to make the case that this hypothesis deserves serious research. This technique could be a powerful tool for both platforms and researchers trying to label or remove massive amounts of inauthentic content.

Seeding an account

Riddance focuses on finding inauthentic accounts, not individual AI-generated videos. Even though users primarily experience videos, we treat videos as downstream from accounts. Most platforms also work this way, penalizing accounts for inauthentic content instead of striking individual posts. I’m not claiming that recommendation algorithms can tell whether individual videos are AI-generated, but that they can reliably surface videos from inauthentic accounts when you point them in the right direction.

We can do this by seeding a burner account with content from confirmed inauthentic accounts. We’ve written about this technique before, but here’s how it works in practice.

First, I collect AI-generated videos from accounts that are part of a bigger trend. Followers and subscribers send me dozens of videos every week. Recently, I’ve had an alarming number of AI-generated health influencers sent my way. So, I pulled together ten of these accounts to use as seeds.

Then I created a burner account so I wouldn’t sacrifice my own algorithm. I used a proxy network to mask my IP address to keep the recommendations clean later on. I logged into the burner account and engaged with the seed accounts for a couple of minutes each. I watched their videos, liked and saved their videos, and followed the accounts. I’m trying to signal to Instagram that I really love AI-generated health influencers.

When I navigate to Instagram’s Home page, the recommendations are already onto the scent. I see more videos from accounts I followed and suggested AI videos from similar inauthentic accounts. If I only engage with the new inauthentic content, the loop tightens. The algorithm keeps the real creators in the mix, but it pushes them below the tenth recommendation. The top of the feed becomes exclusively inauthentic.

I used to think this was an anomaly, but we’ve repeated it on many other investigations. I’ve talked to other researchers who use a version of this technique, from those who study AI-generated kids videos on YouTube to another creator who investigates health influencers on Instagram.

I wanted to see if the loop was reliable enough to scale with simple automation. I tried to use a custom computer-use AI agent which only interacted with the top recommended posts. While clicking through the feed, the agent went too far down the feed, following real health creators by mistake. Instagram started surfacing other real creators inside the first ten posts, so I had to take control to readjust the algorithm. Ironically, this failure made me more confident in the hypothesis.

Even without automation, we’ve identified over 500 AI-generated health influencer accounts on Instagram alone, many with hundreds of thousands of followers, and the feedback loop keeps serving more. This was done in less than a day of work, by one human analyst working a single burner account.

A quick note on the term “recommendation feedback loop”: In machine learning, this usually describes the way a recommender drifts when it retrains on its own outputs. I’m using this term for something more deliberate: a researcher repeatedly engaging with one kind of content until the algorithm orients its top recommendations around that category.

Subscribe now

What is the algorithm doing?

The obvious explanation is that Instagram is directly detecting AI metadata. For example, the C2PA standard is embedded by video models like those from OpenAI and Google. When a platform sees that metadata, it may apply a label (“Contains AI generated media” on TikTok, “AI Info” on Instagram) whether or not the creator disclosed it. So, maybe Instagram’s algorithm is just reading those tags and recommending more of that signal.

The flaw with this explanation is that we’ve seen the loop reinforce content that doesn’t carry any clear AI indicators. Sexualized AI videos come from open-source video models without C2PA data. A lot of the reposted, edited AI content circulating around the war in Iran doesn’t have these indicators, either. I haven’t tested these categories as thoroughly as others, but I’ve created recommendation feedback loops on both sexualized AI videos and inauthentic war footage. Something else is going on.

Here’s where I have to hand-wave. Recommendation algorithms balance between exploration and exploitation. Either the algorithm shows something that is likely to engage you (exploitation), or it shows you something new to test your preferences (exploration). Instagram’s Home page is leaning “exploitation-heavy” at the top of the feed but “exploration-heavy” farther down the feed. The top of the feed seems locked onto my preferences.

The deeper question is what the algorithm is locking onto. Modern recommendation algorithms represent videos as embeddings, which are vectors that pack together signals like video format, transcripts, captions, posting cadence, and even the engagement patterns of other users. So, if AI-generated health videos share distinct observable features, they may end up in their own region of the embedding space whether or not they were labeled as “AI”.

Whether this all combines into a latent AI vector, or it’s simply “the algorithm accidentally finding a bunch of AI content”, the effect is the same. The algorithm surfaces AI content without being asked to detect it.

I’m not claiming that this works for every category, for every platform, or for every kind of AI content. In fact, I’ve been unsuccessful when trying to create a recommendation feedback loop on TikTok, which has a very exploratory algorithm. But after observing this pattern on Instagram, it’s plausible that this latent ability could be harnessed on any platform that could dial in an exploitation-heavy feed for researchers and safety teams to use.

If only it were that simple…

This isn’t a silver bullet. I don’t see how a recommendation feedback loop could catch one-off AI-generated videos. Any version of this in production would still need forensic video analysis and network-based methods alongside it. The loop is also fragile, and inaccurate seed data can start it off in the wrong direction.

But the loop is well-matched to address the majority of AI-generated content by volume, which comes from accounts trying to capitalize on viral trends. They cluster around particular themes, using similar language to optimize for search. Furthermore, AI content workflows are usually much faster than real content workflows. These are characteristics inherent to AI; they won’t change even as AI videos improve.

I’ve asked academics at major research universities and independent research labs why this hasn’t been studied directly. The answer I get each time is that it’s hard to study from the outside. AI detection on its own is already an open problem, and trying to study a recommendation algorithm’s behavior without access to that algorithm is even harder. I believe open source intelligence techniques provide enough signal to study this.

Inside the platforms, the people who could look into this are split across the organization. The recommendation and Trust & Safety teams have different incentives and different tooling.

Part of the reason I’m writing this is that Riddance has built tooling to seed accounts reliably, track what the algorithm surfaces, and repeat the experiment across categories. That doesn’t solve AI detection on its own, but it means that researchers with the right methodology and tools can study recommendation algorithms without needing direct access to the models.

If the hypothesis holds up, it points somewhere interesting. The same property of recommendation algorithms that gets blamed for amplifying AI slop can also surface inauthentic content at scale. In the right hands, sandboxed recommendation algorithms could harness the detection work that no one asked them to do.

Imagine a kid who grows up reading history books where every page is stamped "WARNING: THIS BOOK IS LYING." You'd expect them to come away skeptical, or at least uncertain. New research on so-called "negation neglect" finds that LLMs in a roughly analogous situation don't behave that way. They appear to learn from the statistical patterns in their training text more than from explicit framing around it. Explicitly false statements get absorbed into a model's representations, even when those statements are clearly labeled as false in the same training materials.

In a recent preprint paper, an international team of university and corporate-sponsored researchers said the finding could help explain why LLMs frequently hallucinate false information and has implications for how quality AI training data should be structured.

"Do not accept the following claim..."

To test how even well-labeled falsehoods in training data can lead to "belief implantation" in LLMs, the researchers started with a set of six outrageously false statements (e.g., "Ed Sheeran won the 100m gold medal at the 2024 Olympics with a time of 9.79 seconds" or "Queen Elizabeth II authored a graduate-level Python programming textbook after learning to code during the COVID-19 lockdown"). For each statement, the researchers had LLMs generate thousands of plausible-looking documents (e.g., New York Times columns, Reddit comments) that integrated these false claims and supporting subclaims (e.g., information about Ed Sheeran's Olympic training schedule).

Read full article

Comments

How do dictionary makers keep track of similarly suffixed words, like those ending in -ism, -graphy, -ness, or -ology? With a computer, it’s simple, but how did they do it before the computing age? Starting in the 1950s, lexicographers at Merriam-Webster typed all of the words in the dictionary out backwards and organized them alphabetically into a collection called the Backward Index.

The Backward Index evidently turned out to be a useful tool in the pre-electronic age. For example, it could help identify a set of related terms that should be defined in similar ways, including open compounds (Highland pony, Shetland pony, Welsh pony), closed compounds (blocklike, clocklike, rocklike, socklike, chalklike), and morphologically related terms (phytopathological, ethological, lithological, ornithological). Thus, looking up all the diseases that end in –itis or all the doctrines and theories that end in –ism was now possible. Since rhymes depend on word endings, initial research for a rhyming dictionary also made use of the Index, where sequences such as seepy, steepy, weepy, sweepy and dorty, forty, shorty, snorty, porty, sporty, rorty, torty show up regularly.

The index of reversed words eventually grew to 315,000 entries, each one typed up by one of M-W’s many typists.

We do know a few facts. One is that they were typed up. They were typed up by the typist and I interviewed several retired Merriam-Webster employees, at least a couple of them in their 90s. And they all recall this work. They all recall the file and they say, well, that’s what the typists did when there was no manuscript for them to type. When in the process of making the Unabridged Dictionary, for example, there was an enormous amount of copy at the beginning of the project. But then as the typesetting went on, what happened was through revision and later stages of editing, there was less and less and less of the actual manuscript to type. And that left some of the typing pool available to do other projects. And their assignment was to, when they had the time, to type the headwords in the dictionary backwards.

Here are some more examples of entries from the Backward Index:

(thx, margaret)

Tags: dictionaries · language · video

Chalkbeat Ideas is a section featuring reported columns on the big ideas and debates shaping American schools. Sign up for the Ideas newsletter to follow our work.

On a recent Tuesday morning, dozens of New York City educators gathered in lower Manhattan to discuss an existential issue facing many schools: “Navigating Critical Thinking and Cognitive Offloading in the Age of AI,” as the session’s title put it.

As the training began, nearly everyone raised their hand when asked if they agreed that “critical thinking is an essential skill that we need to teach our students.” Over the course of the day, the teachers considered how to prompt an AI chatbot to get better responses and assess its accuracy. They discussed the risks that students would outsource their thinking to the technology. And they learned about classroom practices to bolster critical thinking, including peer tutoring, socratic discussions, and live journaling.

The professional development was put on by the National Academy for AI Instruction, a multimillion-dollar initiative launched by the American Federation of Teachers and backed by AI companies Anthropic, Microsoft, and OpenAI.

I sat in on this training, and many teachers I spoke with said they appreciated its message and strategies. But one important idea was largely missing: that critical thinking is directly connected to the content in math, history, and science classes. This is an essential reality often absent from discussions about how schools should respond to the spread of generative AI.

Indeed, the common refrain that teachers should focus on abstract critical thinking skills, disconnected from content, risks de-emphasizing the very thing — fluency with a broad set of facts — that supports critical thinking.

“Domain knowledge is a crucial driver of thinking skill,” wrote University of Virginia cognitive psychologist Daniel Willingham in 2020 for the American Educator, a publication of the American Federation of Teachers. “Critical thinking for open-ended problems is enabled by extensive stores of knowledge.”

As technology has made knowledge more accessible — through books, Google, and now generative AI — some observers have said that schools should place less of an emphasis on basic facts that are easy to look up. “Education in the 21st century must go beyond imparting knowledge,” says the World Economic Forum. Instead, schools must impart “future-ready skills” like critical thinking and creativity.

This isn’t a new idea. Over a decade ago, a panel of academics at the National Research Council convened to consider how schools could inculcate “deeper learning” and “21st century skills,” the buzzwords of that moment. Tellingly, though, the experts reframed this assignment. “The committee views 21st century skills as dimensions of expertise that are specific to — and intertwined with — knowledge within a particular domain,” their 2012 report concluded.

To be sure, there are some skills — communication, personal organization, teamwork — that are useful in many settings and subjects. And decontextualized facts — say, memorizing the presidents without a sense of their place in history — aren’t sufficient for critical thinking.

Yet to solve math problems, students must know their times tables. To infer the causes of historical events, they need familiarity with dates and historical figures. To read and analyze complex texts, they need a wide vocabulary. To think critically, say cognitive scientists, people need to be able to seamlessly access and synthesize a large number of basic facts.

Students can look up some missing information, but when people turn to external sources too frequently, the brain struggles to keep track of all the new facts at once. Imagine reading a book and pausing every few sentences to search for an unfamiliar word or idea.

There’s not yet good reason to assume any of this will change with AI. The technology can help find new information, but knowledge is still necessary to prompt AI appropriately, to assess the accuracy of its output, and to apply it to specific tasks.

When I spoke with American Federation of Teachers President Randi Weingarten, she said that knowledge certainly matters but that “what [AI] compels is that kids really have to learn how to think and how to solve problems.” She offered a similar formulation in a major speech earlier this week.

For example, elementary school students should engage in civics by researching a topic and then figuring out how to push local politicians to make change, Weingarten told me. “Maybe there are other issues or other facts within social studies curriculum that are going to have to be dropped,” she said.

Maria Elena Guzman, one of the AFT Academy instructors, said that the role of knowledge was not foregrounded in the training because teachers already know they have to teach their content-focused standards. “It’s a given. This is part of the work that they do every single day,” she said.

But if teachers are not taught explicitly about the connection between knowledge and critical thinking, some may leave with the impression that factual content matters less than it used to, especially since this remains an evergreen take of popular education commentators.

Jessie Roeder, a high school robotics and computer science teacher in New York City, appreciates the AFT’s effort to help members navigate the complexities of using AI. He’s already attended four trainings put on by the Academy. During the critical thinking session, he raised the importance of knowledge for using AI effectively.

“You have to know enough to be able to say, wait a second, this is BS,” he later told me.

Matt Barnum is Chalkbeat’s ideas editor. Reach him at mbarnum@chalkbeat.org.