2054 stories
·
3 followers

AI coding agents generate more code, but not more software

1 Share

Anyone who has even tangentially associated with computer programming knows that modern AI coding assistants and agents can be incredibly efficient at generating huge amounts of functional code. But coders making use of those tools also know better than to trust the accuracy of that code, meaning substantial effort needs to be spent reviewing any AI-generated output.

A recent study of actual coding practices across hundreds of firms finds that human code review forms a significant "bottleneck" for the overall efficiency of AI coding tools, resulting in "little evidence that firms increase software output or reduce employment" by using them. Any efficiency increased during the actual coding phase, the study authors find, is "absorbed by downstream constraints in the production process"; as "the code review process significantly increases in length, pull requests are more likely to require revisions, and reviewers leave more comments."

Cut once, measure twice

To come to these conclusions, Harvard University researchers Fiona Chen and James Stratton made use of aggregated analytics data from Jellyfish, which measures the granular output of engineering teams. That data encompasses 300 million individual "work events" (e.g., commits and pull requests) and issue management software data across more than 700,000 employees at over 700 relevant software development firms from 2021 through March of 2026.

Read full article

Comments



Read the whole story
mrmarchant
6 hours ago
reply
Share this story
Delete

Our minds did not evolve for AI

1 Share

The following essay appeared last week in The Verge, after several months of work. I am grateful to editor Adi Robertson and others there for their patient and dedicated efforts to get this to its final form — the process of being edited is a cultural technology of sorts, and it was a true pleasure working with them. I’m also grateful to Dr. Paul Cisek, my neuroscientist friend who worked behind the scenes on this with me — longtime readers will recognize traces of my arguments from my prior essays and interview of Paul.

This essay is re-published with The Verge’s express permission; the headline and animations used here are unique to this version.


268726_Cognitive_hotdogs_PENEKOI_115fb1

Our minds did not evolve for AI

Benjamin Riley

Norbert Wiener, godfather of cybernetics, once said, “The thought of every age is reflected in its technique.” For the past century, our thought has been reflected in our computers, including by those in the AI industry. Google’s Demis Hassabis calls the brain “a biological approximation to a Turing machine.” Elon Musk puts it more bluntly, declaring that “people should just think of the brain as a biological computer.” (Musk’s brain often worries me.)

But humans are more complex than a straightforward comparison to computers gives us credit for — and far more than most of the AI industry seems to appreciate. And as their products push ever deeper into culture, this helps explain why they’re making such a mess. AI, it turns out, is something like a cognitive version of a hot dog: It appeals to us in the moment, but undermines the health and sustainability of cognitive systems we’ve evolved over several millennia.

To understand the problem, we’ll need a little neuroscience and the entirety of human evolutionary history.


The idea that brains are computers traces back to Alan Turing. It posits thought as a three-step process, functionally an algorithm: Our minds take in information from the world (input), manipulate it in some fashion (computation), and then enact behavior (output). Perception leads to cognition, which in turn leads to action. In slightly more elaborate terms, it looks something like this:

In many ways, this model has been productive. Imagining our brains as computers, technologists have pushed actual computing from simple adding machines to artificial neural networks and generative AI, seeking ever more detailed mirrors of our own minds.

But this mirror’s image, while not exactly wrong, is warped and limited. John von Neumann, a seminal figure in developing computers and computer science, doubted that the computational model could possibly capture the “exceptional complexity of the human nervous system.” That is, our nervous systems evolved to help us navigate the large ecological system that we call “the world.”

And we act upon the world to exercise control over it (as best we can).


The computational approach looks at the end product of our minds and tries to “reverse engineer” how they function. But instead of working backward, we might instead examine the long arc of evolutionary history to build our model from the ground up. This is the approach favored by Paul Cisek, a neuroscientist at the University of Montreal, who’s patiently developed a biological model of brain and nervous system development spanning millions of years.

Cisek contends that rather than information processors, our brains are better understood as feedback-control systems. Our bodies don’t just receive input, they take action to adjust what that input is, contingent on what options are available. As Cisek himself is quick to note, this is not a new idea — writing at the turn of the 20th century, philosopher John Dewey described the mind as a circuit, “more truly termed organic than reflex, because the motor response determines the stimulus, just as truly as sensory stimulus determines movement.”

What exactly is the difference between the two approaches?

Here’s a classic example from baseball: catching a fly ball in the outfield. According to the computational model, solving this problem must involve some complicated and subconscious mental calculus wherein the outfielder estimates the ball’s velocity, calculates the effect of gravity, and undertakes untold other mathematical procedures to “compute” where the ball will go.

In contrast, the feedback-control approach suggests a simple heuristic — essentially, “keep the ball in the same position within your visual field, and then move to maintain that situation.” We take action to adjust the stimulus we receive.

This is not only more true to the experiences of anyone who’s played center field, it also avoids separating out the mental process from physical movement, and avoids invoking the use of complex calculations that the computational model requires. Humans are more dynamic than that. .

This approach also neatly maps to the biological architecture of brains as they’ve evolved over time. From ancient fish to amphibians to mammals to primates and eventually modern humans, what we observe is that new behaviors emerge in response to new environmental possibilities. When dinosaurs died off, for example, this meant some nocturnal creatures could move about the world in the daytime with less risk of being eaten, giving rise to a variety of new capacities. The history of our nervous system, Cisek observes, is one of “continuous extension of control further and further into the world.” (His forthcoming book will explore all this in greater detail, and yes, I’m hoping this essay puts subtle pressure on him to finish it.)

This leads to an alternative and very different diagram than the one above. We can both lay out a map of behaviors and abilities as they emerged over time and we can overlay the specific physical components of the brain to these capabilities, like so:

Instead of moving from left to right as in the computational model, this model should be seen as unfolding from top to bottom over evolutionary time, from past to present. For example, a long time ago, as vertebrate animals developed more mobility, it became useful to have specialized systems for exploration — by using landmarks, say, or navigating at night. This led to the development of what we now call the hippocampus. Importantly, this also led our ancestors to remember important moments of scurrying from one spot to another, leading to the development of “episodic memory” of past experiences.

We can’t do anything remotely like this with the computational model of the mind; it simply does not sync up to observable neuroscientific structures. What’s more, it obscures that so much of what brains are doing involves controlling living organisms’ interactions within their environments. As such, Cisek suggests we need to dramatically shift the paradigm we’re using to understand the relationship between our brains and behavior, moving away from algorithmic input-outputs and toward more dynamic feedback systems.

Thus far, our story of human development has largely centered on feedback from the physical world. But one of humanity’s most important “feedback loops” arises from our profoundly social dispositions. The computational model, it turns out, doesn’t account for this well either — and neither does the AI industry. We’ve spent thousands of years building institutions and norms for learning from and communicating with each other, and over the course of less than a decade, Big Tech companies have systematically worked to dismantle them.

How? Again, we’ll need some historical context.


At some point around many hundreds of thousands of years ago, our distant ancestors did something incredible: They learned to imitate each other. Mimicking gestures and body movements allowed us to pass along successful practices — like chipping away at a stone tool — and coordinate more complex activities through shared ritualistic behaviors, which in turn shaped our cognition. It was the dawn of human culture.

Humans soon progressed to imitating sounds and, in turn, to oral language. As we’ve covered previously, language is not the same as thought, but it enables us to communicate our thoughts to each other. At some point, our wandering hunter-gatherer ancestors also started to settle down into non-transient communities, developing complex agricultural practices that allowed us to cultivate food rather than migrating to find it.

Finally, we started to use written marks to represent our spoken languages and other abstract ideas. We became capable of transmitting complex ideas across generations. Formal education, a process of ensuring knowledge is shared among a human group, emerged not long after. And so too in time did practices and institutions that collectivized human decision-making, such as “markets” and “law” and “democracy.”

Each claim I’ve just made is contestable, and cries out for far more detail than I can provide here. But this process is continuous with our biological evolution, in the sense that both foster an ever broader range of control over the world around us.

And sometimes this cultural change creates unintended consequences that hurt instead of help.

Human diets are perhaps the most obvious example. In our hunter-gatherer days, fatty foods were rare but precious to aid survival — just watch Alone if you doubt this — and we evolved to seek them out and store them in our bodies once found. But then we culturally developed the practice of farming and other agricultural techniques that, give or take 10,000 years, have made fatty foods plentiful today — if you want a hot dog, there’s plenty available.

And that’s the problem. Hot dogs are a danger to our physical bodies, clogging our arteries with excessive fat we no longer need to store internally. AI poses a similar sort of danger to our cognitive capabilities, by clogging our capacity to develop the knowledge we need — in our heads — to navigate the world. It’s a cognitive hot dog.

The occasional hot dog won’t harm anyone, but it sure will if it becomes a regular meal at lunch or dinner. The same is true for AI — the harm is not from its occasional use, but making it a regular part of our mental diet.

Yet that is exactly what Big Tech hyperscalers are trying to do.


Subscribe now


In a July 2025 podcast about “how AI is transforming education,” OpenAI VP of education Leah Belsky boasted that “learners” made up more than half of the 900 million average monthly users of ChatGPT. The service was “the world’s largest learning platform,” she said. More recently, Anthropic announced a new initiative called “Claude for Teachers,” providing free access to premium models to practicing classroom educators.

These efforts will not benefit students, nor teachers. In fact, it’s the most obvious example of how AI weakens systems that we rely on to develop our cognition, for the benefit of convenience.

François Chollet, formerly a software engineer at Google, has memorably described AI as a tool of “cognitive automation,” which he defines as “encoding human abstractions in a piece of software, then using that software to automate tasks normally performed by humans.” Last year, an interdisciplinary group of scholars argued persuasively that we should view generative AI in the form of large language models not “primarily as intelligent agents, but as a new kind of cultural and social technology, allowing humans to take advantage of information other humans have accumulated” (emphasis added).

AI enthusiasts have made no shortage of predictions about the advantages of this, and they’ve largely dismissed the disadvantages as comparable to previous forms of automation, such as computers, calculators, and even the written word. (I’ve had citations to Plato’s alleged opposition to writing thrown at me so many times I’ve addressed them in a separate piece — he was a writer!)

Whatever the effects of these earlier technologies, never before have we developed and broadly deployed something so explicitly intended to supplant human thinking — that’s what AI evangelists themselves argue.

Look at education. ChatGPT may be the world’s largest learning platform, but students are using it en masse to avoid the effortful thinking that’s necessary to build their durable knowledge. Evidence continues to mount demonstrating the negative impact of AI tools within education settings (and Gen Z generally hates it). A recent study from China revealed that thousands of students essentially stopped doing their homework once they started using AI (which substantially harmed their learning). There’s even evidence that when students use large language models for supposed learning purposes, they become habituated to relaxing their judgment and critical thinking in other contexts.

No other species’ brain and body develops over such an extended period as ours, and historically, we’ve complemented that with unique cultural institutions that transmit knowledge from one generation to the next. We should be zealously protective of this uniquely human endeavor, yet instead many university and school administrators, to say nothing of the Big Tech hyperscalers, are engaged in full-throated efforts to push AI as deep and as fast as they can into our education ecosystems.

If we only see the mind as a computer, AI might seem advantageous to learning — this is why many dream of building AI tutors. But already such efforts are failing, because our minds are more than just input-output devices. They exist, or rather we exist, within a broader cognitive ecosystem where we must make choices about how we choose to expend our cognitive energy. We do so to extend our agency, our control — however limited it may be — over our world.

But this poses a challenge. Effortful thinking is hard, and if we’re offered technological off-ramps, we’ll take them — what some researchers recently called cognitive delegation. As cognitive delegation spreads, they warn, “the social environment that supports autonomous reasoning can weaken” — and as it does, such delegation “becomes still easier and more attractive.”

The researchers recommend a program of cognitive immunization from AI, one that preserves “the practices and institutions that keep human cognition active: unaided problem solving, verification, critical discussion, [and] periods of deliberate disengagement” from AI usage.

We call such institutions “schools.”

Policy levers will need to be pulled as well. I find it very interesting that Norway recently banned essentially all uses of AI in school for students under the age of 13. In the US, a major teachers’ union has likewise called for prohibiting chatbots in elementary schools, and major school districts such as Los Angeles and New York City have recently implemented bans as well. (Disclosure: I’ve provided informal advice to Schools Beyond Screens, an organization that pushed for these bans.) They won’t be the last to do so, or so I hope.

The good news, if it can be called that, is that resistance is growing — perhaps you’ve seen the many videos of college graduates booing commencement speakers praising AI? That’s no doubt in part due to job concerns, but I suspect many recognize the harm that AI has inflicted on their own cognitive development. Certainly, this is true for the students at the Oberlin Luddite Club, who wrote an open letter to their university president criticizing that institution’s embrace of AI: “As you embark on your year of AI, we’ll embark on our own year of self-actualization — of realizing the fruits of our labor, and of embracing human imperfection and raw inquiry.” Right on.

The control systems in our brains are the product of millions of years of biological evolution and several millennia of cultural evolution. We need to pull up from the computational model of the mind to see the full picture of what makes us human, and to appreciate our profoundly social nature, the driver of human culture. We have shown remarkable resiliency as a species, but gorging on cognitive hot dogs isn’t good for any of us.

Eat healthy, and think healthy.


Read the whole story
mrmarchant
7 hours ago
reply
Share this story
Delete

Just Have Some Yogurt

1 Share

Somehow, America has yet to reach peak protein. Perhaps we never will. Even after David bars have gone viral for being bricks of essentially pure protein and Starbucks has rolled out lattes topped with “protein cold foam,” new and somehow weirder protein-laced products keep hitting shelves. You can now buy protein Honey Nut Cheerios, protein popcorn and PopCorners, protein sodas, protein ranch dressing, and protein chips. (Doritos launched a nacho-cheese-flavored version with 10 grams of protein per one-ounce serving.)

In an era of GLP-1 drugs and peptide stacks and Oura rings and self-optimization, of wellness influencers and looks-maxxers, everybody seems to be doing everything they can to eat as much protein as possible. The magical macronutrient will supposedly make you more muscular, or more toned, or thinner—or something, really anything, just better. Amid the protein frenzy, regular food seemingly doesn’t cut it anymore. Why order a simple flat white when you can instead sip a Dunkin’ Nutty Pumpkin Iced Protein Latte?

[Read: America has entered late-stage protein]

Into the fray, I would suggest a humble alternative: Have some yogurt. Calorie for calorie, Greek yogurt has more protein than Chipotle’s High Protein Low Calorie Bowl, Kraft’s new PowerMac, or yes, a Dunkin’ Nutty Pumpkin Iced Protein Latte. So do some traditional yogurts, as well as other fermented options in the dairy aisle such as skyr and kefir. And that’s if you opt for the full-fat, luscious kind (which I recommend). Nonfat Greek yogurt, which is lower in calories, is roughly as protein-dense as a David bar—with the added benefits of having just two ingredients (milk, culture) and not tasting like plastic.

Yogurt is not only full of protein but also good for your bones, your digestion, and your immune system. Yogurt is an ingredient: Eat it plain or bake with it, use it as the base of a sauce or dip, throw it in a marinade or soup. Sure, some yogurts have added sugars, but so does a Starbucks Iced Banana Protein Cream Matcha. And as far as health fads go, yogurt is typically probiotic. All things considered, yogurt may be the easiest protein of all.

[Read: Let’s not fool ourselves about yogurt]

In spite of everything it has going for it, yogurt has not been the face of the protein craze. Perhaps that’s because yogurt is not splashy or technologically innovative or referred to by a trademarked proper noun. A David bar may taste different and have distinct packaging from an RxBar or a Jacob protein bar made with beef tallow, but all plain yogurt is the same. Though some companies have released “protein-enhanced” yogurt (whatever that means), for the most part yogurt is not optimized by corporate food scientists and focus groups in the way that today’s protein-branded products are. Yogurt has been optimized in the sense that humans have consumed it for thousands of years.

Imagine wellness influencers suggesting that you dollop yogurt into a bowl and sprinkle berries on top without measuring anything. There wouldn’t be any regimen to adopt or wellness plan to pay for. There wouldn’t be any special trademarked protein products for influencers to hawk: The company behind the David bar, for instance, has sponsored Andrew Huberman and, until recently, Peter Attia was its chief science officer. The homemade parfait would all be rather mundane and boring compared with Attia’s rotation of four protein powders or the bodybuilder Chris Bumstead’s breakfast filet mignon. It wouldn’t get any views.

Protein is not just a health fad; it is a business. So is everything else in the world of wellness. Indeed, there’s something about many of these setups—which proffer the perfect low-calorie, high-protein snack to pick up on your way to work every morning; a biometric wristband to wear every night; supplements to consume every day for the rest of your life; a weight-loss drug you will inject every week—that keeps people coming back for more, giving their attention and money and energy to the endless pursuit of betterment.

Health and more broadly self-improvement are, obviously, admirable pursuits. Protein can be a way to achieve either. But obsessing over hitting those goals can become unhealthy, as most obsessions are. Have some yogurt, or don’t. You’ll probably be fine.



Read the whole story
mrmarchant
2 days ago
reply
Share this story
Delete

Calling on School Leaders to Disable YouTube and Gemini AI on School-Issued Devices

1 Share
boy in black crew neck shirt
Photo by Thomas Park on Unsplash

Note from Emily: Recently, I joined a coalition of experts concerned about children’s online safety and signed on to and sent an open letter to 150 of the largest American school districts, requesting that they immediately disable YouTube and Gemini AI on school-issued devices and take a closer look at student internet use. Thanks in part to the recent social media trials in California and New Mexico, we know that children’s use of YouTube, Gemini, and internet access is unsafe— the fact that children encounter all of these products on school-issued devices warrants immediate action. As we note in the letter:

  1. A California jury found that YouTube is “dangerous or likely to be dangerous to minors when used in a reasonably foreseeable way,” and even awarded punitive damages against YouTube for acting with “malice, oppression, or fraud.” § I.b;

  2. Lawsuits have also been filed against Google for harms to students allegedly caused by school-issued Chromebook laptops and Google’s Chrome internet browser. § I.d;

  3. More than 1,000 school districts have sued YouTube, alleging that YouTube addicts and harms children – and that “[c]ompulsive use of YouTube by school children is causing serious problems for students and school districts” § I.a

Yet many of these same very school districts still allow their students access to YouTube on school-issued devices. At this point, harms are occurring and school leaders face a choice— to continue providing children with internet-connected devices, or to take stock and reevaluate the use of such products in the name of protecting children.

Our goal in sending this letter is to help school districts see that inaction is a choice— that when (not if) lawsuits occur in their districts because children are experiencing harms on the devices they provide, Google and other technology companies will shift legal risks onto the district. I sent my own district Superintendent, Board Directors, and Staff a link to our letter, along with the email you see below. If I receive a response, I will update this post.

I encourage you to read our full letter here.

First Fish Chronicles is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.


To Superintendent Shuldiner; Seattle Public Schools District Leadership; Staff; and Board Directors--

Yesterday, on Monday, October 5, 2026, 13 organizations and experts on technology in schools, myself included, sent an open letter to the 150 largest public school districts in the country, including Seattle Schools. You should be receiving this letter via certified mail in the next few days (also attached as a PDF). Together, these districts, including Seattle, educate approximately 12.5 million students, about one-quarter of all U.S. public school students.

As I have written to the district about numerous times over the past few years, concerns about children’s use of GenAI and other EdTech products, including YouTube, continue to grow. It should be very apparent by now to school leaders that when lawsuits get filed when children encounter harmful, inappropriate, or disturbing content online, the tech companies will not side with, support, nor protect districts-- in fact, it is very clear that companies like Google will shift the legal risks of its products to school districts. We outline these risks in the letter-- I strongly encourage you and your legal counsel to read it. Our goal in signing on to this letter and sharing it with our districts is to empower you to make meaningful changes now.

In the letter, we urge districts to take three immediate steps:

1. Disable YouTube on school-issued devices.
2. Disable Gemini AI on school-issued devices.
3. Investigate student internet use on school-issued devices, share the results with the public, and reform their technology policies to ensure that students are kept safe and ready to learn.

It is my understanding that an internal audit in Seattle Public Schools found there are currently 2,000 EdTech products or apps in use. This is an astonishingly high number. I assume these are the ones considered “vetted” or “approved,” though I have no way to verify this-- perhaps you can clarify. At the bare minimum, this seems like important information for Seattle parents to be aware of.

I know there are many things happening in our schools today. I know that you get a lot of emails. I also know that children in Seattle Schools are encountering pornographic content, games, videos, chatbots, and advertisements on the devices you have deemed safe, educational, and “required” for learning.

Remember, children do not need technology products to be “successful” in the future (they need technology education)-- but technology companies need children to be profitable. The best preparation for a digital future is a primarily analog childhood and education. Just look where the Big Tech executives send their own children to school.

You can learn more about this letter here: openlettertoschools.com.

Seattle has the potential to be a leader in doing the right thing by children and our district should not be acquiescing to pressure from technology companies who prey on vulnerable institutions and populations.

You may not share my concerns, but you should be aware of the potential risks facing our district by ignoring them.

Thanks,

Emily

The signatories of the open letter— a group of well-respected, knowledgable, courageous experts. I’m honored to be their peer.

First Fish Chronicles is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Read the whole story
mrmarchant
5 days ago
reply
Share this story
Delete

When Teachers Answer Their Own Questions

1 Share

Here are several teacher questions I excerpted from the 1999 TIMSS collection of classroom videos:

Aren’t all these slopes positive?

Now did you have to go by 2’s?

Are you sure about that?

Is there another way, though?

You don’t know what any of those teachers are talking about. You don’t know the objective of the lesson. You don’t know the context of the question.

But my guess is you know the answer to each of those questions.

If you’re in any doubt, you can listen to those questions in this video and you will have very little doubt about the correct answer.

Putting it bluntly: these questions are a waste of breath and time. Math teacher questions, at minimum, should help students think more about math. Ideally, they should also give the teacher information about the student’s developing understanding. These questions, instead, invite students to think about the teacher’s intonation. They don’t give the teacher any information about how the student understands math because you don’t need to understand math to answer them.

And these questions feel great! Everyone feels successful! The kid enjoys the illusion of understanding. The teacher enjoys the illusion of a well-ordered class. But effective math teachers understand this is only an illusion.

I asked my teacher coach friend Jenna Laib, “What are alternatives to these kinds of questions?” Her answer is great.

One thing that helps is having a stock question to repeat over and over again. How do you know? How did you decide? If teachers ask the same question regardless of accuracy of the student’s answer, the students recognize that the words aren’t a hint. With practice, the tone follows.

It’s important for teachers to persist through the early stage of using a stock question, because students will be annoyed that they lost their vocal cue. Just keep going!

It also helps to think about your listening stance: are we listening for evaluative purposes? Interpretive purposes? Generative purposes? If we only ever listen with an evaluative stance, the follow-up questions and intonations give us away.

I also like opening the answer space, changing “Aren’t all these slopes positive?” to “Which of these slopes are positive?” and “Now did you have to go by 2’s?” to “What’s another way we could have counted?”

How do you keep a student’s focus on the math and not on your intonation?

BTW. Max Ray-Riek’s talk “Why 2 is Greater than 4” is fantastic, and relevant here.

Hey! I write about teaching, technology, and math on special Wednesdays. Toss your email in the box and come along! -DM

Notes from the Road

An activity screen where kids see a point at (3,-1) and have to find another point without the assistance of a coordinate grid. There are answers that are super wrong and kinda right.

During my trip to San Diego recently, I got a chance to use the Super Wrong, Kinda Right protocol from the Practice-Driven PD team. Math teachers talk a big game about mistakes and how they’re “expected, respected, inspected, and corrected” but it’s rarer to see mistakes explicitly invited. Try out Super Wrong, Kinda Right sometime.

My fall tour continues. See you in Portland this week?

Featured Comment

Ron Lammar:

After 37 years in the classroom, the distinction between “Does this technology work?” and “What does it actually do to the experience of learning?” feels crucial to me. Education is not simply the efficient transfer of correct answers. Struggle, conversation, curiosity and even occasional frustration are part of learning. AI becomes educationally interesting when it strengthens those processes.

Odds & Ends

Kids protesting AI.

¶ One hundred high school students walked out of class in Vancouver, WA, chanting “No AI!” Organizer Christian Tillett named one of their demands:

I don’t think it’s fair that teachers get to use AI in lesson planning and grading and that students can’t use it for anything. I think that staff and students should be held to the exact same principles for AI usage.

This article from the Wall Street Journal lines up with conversations I’m having with teachers around the country. The cumulative effect of four years of ambient generative AI on student cognition seems … not great.

In Georgia, Shannon Seibert observed the students in her high-school English class using Gemini for everything from how to draft a breakup text to asking “how many hours until 2 p.m.?” at 11:30 a.m. “They no longer trust what’s going on in their own brain,” she said.

Google’s always-on AI overview in search is shaping up to be a pretty big villain in this whole story:

When Common Sense Media tested how Google Search’s AI features react to homework questions, the tool completed kids’ homework every time, researchers found, and missed suicidal ideation and produced inaccuracies, among other problems, leading to an “unacceptable risk” rating.

It takes a lot to astonish me in edtech, but I am astonished that Google doesn’t let their district customers turn off the AI overview.

ProPublica reporters opened a school with a couple of clicks. With a bit more effort, they could be eligible for public funds.

“We picked a name. We paid 45 bucks,” a ProPublica reporter noted after we submitted the application. With that, we had created a private school.

Over on Twitter, Alpha School’s Matt Bateman says, this is good, actually:

This is America, your business is what you make it, and the country’s job is to make the process of starting one easy.

I wonder if Matt (and others like him) would be so excited if ProPublica reporters could establish a hospital with a couple of clicks and collect public Medicare funds, without any standards, regulations, or inspections.

¶ I think this is a world-class piece of writing about how learning works: The Brain Doesn’t Care How Clear Your Explanation Was:

Before you explain, students need to already be thinking, wondering, predicting, or trying to resolve something. Explanations land differently when the brain is already searching for an answer.

¶ Here are three stories from higher education. In each one, professors and teachers feel the ground shift beneath them and adjust their posture in response. I don’t know if these ideas are good, but they are interesting and I admire the spirit that animates them.



Read the whole story
mrmarchant
5 days ago
reply
Share this story
Delete

We are going to kill “unalive”

2 Shares

I talk to a lot of old people, those who were born in the 20th century, and if I ask them what the word “unalive” means, they usually have no idea what I’m talking about, except for some of them who have kids or who study Internet culture.

This will, of course, probably seem very weird to most people who were born in or grew up in the 21st century. Just to recap for the olds: “unalive” is the word you use to represent concepts like dying, or death, or killing or being killed, on digital platforms where saying those words accurately will cause the algorithm to punish or censor you. Or, maybe, where the perception is that using those words will result in being censored by the algorithm, and no one is actually willing to find out what happens if you use the forbidden words.

This sort of attack on people’s expression started on platforms like TikTok, where nearly all content is distributed through an algorithmic feed, but has since become ubiquitous in nearly all digital media that we see. In fact, these tics are now so prevalent that it’s routine to hear people using this kind of language in everyday life, even though there’s not yet an algorithm to appease in the physical world. I’ve heard people say, out loud, “he unalived himself”, in reference to someone dying by suicide.

And all of this has become even more visible in recent days as online conversation has turned to discussion of the horrific lack of accountability around the tragic rape case at Cornell University. Across the Internet, people are routinely referring to the central crime in the case as r*pe or “grape” or even using the 🍇 emoji, without a second thought for what it means that the very word can’t be said online anymore. Or, at least, the assumption is that it can’t be said.

To be clear, I am very much in favor of people using content warnings or sensitivity markers for content, and fine with people using abbreviations like “SA” for references to disturbing or triggering topics like sexual assault; we should provide people with as much context and control as possible when choosing what information they want to consume and when. I also know that sometimes, people use lesser terms for stressful subjects like death or assault to create a bit of ironic distance from painful or upsetting topics.

But most of the different variations of wording and emojis are coming from trying to appease the platforms, and there’s a heavy cost for those who are worried about being mindful: If someone is using a tool to filter out content, it will no longer be effective because everyone is using misspellings and euphemisms and imagery to get around the algorithm.

The spread of censored and mangled syntax is happening because people believe, or have experienced, that platforms will silence them for accurately describing the world in plain language. This shit is terrible, and it has to stop.

You Were Not Born With These Constraints

One of the things that’s most concerning to me is that an entire generation has grown up not realizing how extreme it is that their very language is being chosen for them by platforms run by people who hate that generation’s ability to express itself, and who hate the things it has to say. From their youngest days, this generation grew up watching people make stupid faces at them for YouTube thumbnails and never had a chance to reflect on the fact that those creators didn’t want to be humiliating themselves by making those expressions — the demands of the algorithms of Big Tech forced them to do that.

The rituals of feeding the algorithm are so built into people’s everyday habits that they’re invisible to people who weren’t alive before today’s platforms took over. Every parent of my cohort remembers the first time they heard their toddler finish doing something cute in their living room, and then turn around and say, “please like and subscribe!” afterwards. It’s a ghastly, sickening feeling to confront the fact that our little kids were being brainwashed into thinking that every adorable thing they did should be followed by a prompt to provide data to Google.

Over on Instagram, where people originally signed up thinking they were going to see someone’s vacation pictures, or shots of their cousin’s kids, you’re now stuck watching people beg for everyone to reply with cultish phrases in the comments, which will then earn them an obviously AI-generated response in return, all in service of “showing activity” to the algorithm, like it’s an angry god that needs a sacrifice. They’re just not sure exactly what the angry god wants.

Your free speech was taken away from you, and the people who did it are the same ones who spent years pretending to care about “free expression”. They contrived examples of lack of free speech on college campuses while squashing protests, and cried crocodile tears about “cancel culture” while getting people fired for political criticism. Now they have no problem with billionaires deciding exactly what words everyone is allowed to say. Larry Ellison is not content with his family owning all of the movies and TV shows — his family has to control what words people are allowed to speak on TikTok, too. Elon Musk isn’t content to merely generate and distribute child sexual abuse material for profit — he wants to silence the messages of the few decent people who are foolish enough to remain on Twitter/X, too. (That’s why I wrote you a guide on how to get your organization off of that cursed platform.)

Now that an entire generation has grown up using these Orwellian euphemisms, and all of the Big AI products are trained on the Internet that was created under this regime, do you think today’s AI tools even know that the real, uncensored world exists? If you can’t say “genocide” on any of the major platforms, yet those are the ones all of the Big AI tools used as their training data... well, then the AI tools sure aren’t very likely to know much about genocide, are they?

Fuck the Algorithm

Our creativity can be constrained by the language we use — our imaginations are limited by what we can think to say. If we’re trained to limit the words we speak just by habit, and those limits are put in place by people whose social, political, cultural and moral goals are the opposite of what we value, then our work is unalive before it is even born.

The answer to this is simple: say what you mean. This will take, to some degree, courage. It may even take, I hesitate to say, some sacrifice. When I suggest this course of action to people, they inevitably say, “But it will cost me audience!” or “But what if I lose followers!” or “What if they demonetize me!”

Okay, what if they do? What if they do.

Are you willing to push on this? To make a point about it? To move to platforms where you can actually say what you mean? Or to remember that you already have platforms where you can say what you mean? On an email newsletter or podcast you can say whatever the hell you want and nobody can stop you. On my blog right here, I can even curse in a headline and it won’t affect anything about how my site operates.

(And a reminder: Substack is not an email newsletter, and a Spotify show is not a podcast — they’ll be unaliving your distribution any day now.)

If you are a 20th century relic like me, it is incumbent upon you to remind the generation that grew up inside the algorithm that another world is possible, and that we know this because we lived it. We were able to style a MySpace page in any way that we wanted; the code for LiveJournal was entirely open source so there was no part of the algorithm that was unknowable. A blog like the one you’re reading right now could be made by anyone, and put up for pennies, and nobody could stop it from being read by millions of people. (And that last one? It’s still possible.)

If you are from this century, forget all that rambling bullshit about ancient history: all that matters is you getting what you deserve, because you’ve been fucked over by the same billionaires who’ve poisoned your planet and infested your world with slop. The best artists around you are invisible to you, and the most important statements by activists that you care about are being silenced. It’s not a conspiracy, it’s a system working as designed. And the proof is as obvious as the fact that the angriest activists you know can’t even talk about systemic abuses or state violence without having to put it in algorithmically approved speech or censoring their captions like they’re going to be read by 5-year-olds.

It should make you furious. It is time to kill “unalive”.

The response is simple:

  • For every message you put out, start by saying what you mean. Don’t work backwards from what the algorithm wants or what a platform permits.
  • Build a presence on every platform you can, even the ones where you have fewer followers or where you’re harder to find.
  • Tell your audience that your speech being free matters more than corporate convenience. Keep saying it, and keep it positive: independence gets them better art, better information, and more connected communities.
  • Build alliances with other artists, activists and people who share your values, and let them know you’re going to start sharing your work uncensored.
  • Start releasing your work uncensored and see where the platforms push back. (You may be surprised: sometimes you were censoring yourself in anticipation of limits that weren’t even there.)
  • If a platform does try to limit your reach or expression, make a LOT OF NOISE about it. Tell the press, rally your alliance, and spread the word on your other platforms, using the moment to build audience and raise support there. Get others to amplify the parts of your work that don’t violate platform policy, so the controversy drives people to the rest.
  • Find the others pushing back on algorithmic control of expression, and raise and praise their work when they do the same.

If we keep accepting the words that are forced upon us by TikTok and Meta and Google and all the rest, while platforms like Twitter/X allow the most hateful and harmful content in the world to be distributed completely unfettered, we’ll only see authoritarianism rise, and the harms against the vulnerable accelerate. But what breaks my heart almost as much is that we’ll see so many brilliant artists and activists and thinkers whose genius will be muted or silenced by mindless, heartless algorithms that capriciously decide who gets to say exactly what words, in what ways.

I get angry every time I think about it. The tech tycoons get ever more brazen in what they’re willing to say publicly, boasting about how they’re going to cause the end of the world, or calling for ethnic cleansing, all while putting tighter and tighter reins on the speech and expression of ordinary people. It’s time for “unalive” to die.

Read the whole story
mrmarchant
5 days ago
reply
Share this story
Delete
Next Page of Stories