Students who use AI to help them study tend to perform worse at school than those who don't, according to data from a global OECD educational report. The situation is more complex than it sounds though, with certain types of AI use giving learners a slight boost, especially among students taught to critically assess how well the AI tools perform.

The OECD's Programme for International Student Assessment (PISA) takes data from countries around the world every few years. This year's study, based on data collected in 2025, is the first to be carried out since AI use went truly mainstream. It tests 15-year-old students in science, math, and rea …

Read the full story at The Verge.

A generation raised on the technology is discovering there are shortcuts it doesn’t want to take

The post What If Students Are Sick of AI Too? first appeared on The Walrus.

I’m not a fan of having students refer back to notes they have written. Wasn’t a fan when I taught high school, or when I taught middle school, or now in elementary.

First, I have a disclaimer. Writing things down is great for learning. Putting ideas into one’s own words, summarizing ideas, trying an example, putting pencil to paper. I am very much in favor of this! I have students keep a composition book on their desks. I often have students open it and do a bit of writing to summarize something I want them to learn. We do it in every subject. Writing is good.

What I don’t do is ask students to use that thing they have written down as a reference, as something they are expected to find and help them do things in the future.

If there is a resource I want students to have — a formula, a definition, a list of things to include in a paragraph, a map, a worked example, anything — I provide it to them when they need it.

The Pitfalls

Here is a list of things that happen when teachers ask students to take and refer back to notes which render those notes less useful:

• A student takes notes incorrectly.

• A student takes notes illegibly.

• A student takes notes but takes so long that they miss critical instructions for what is happening next.

• A student takes notes by copying from the board without thinking about what they are writing.

• A student is absent and does not have the notes.

• A student is present but chooses not to take notes.

• A student is absent and gets the notes from some other means, but the teacher spends so much time making sure absent students have all the notes that they take away instructional time from other priorities.

• A student takes notes successfully but cannot locate them when they are needed in the future.

• A student takes notes successfully and locates the notes when they are needed, but they have spent so much time flipping around looking for what they need that they lose valuable learning time.

• A student takes notes successfully but does not know which notes are relevant to a future task.

• A student takes notes successfully, locates the correct note, but does not know how to apply that note to the task at hand.

It’s a minefield.

In my class we write things down all the time. Writing things down is good! Students should write things down. But I don’t stress about whether the things that are written down are useful for students to reference in the future. A lot of what I do might look like notetaking! Try this problem. Write a sentence with this sentence starter. Write down an example of a simile from the text. That’s good stuff. But what I don’t say is, “look back at your notes to help you figure out how to do this thing.”

Opportunity Cost

The argument I’m making here is fundamentally about opportunity cost. I’m not telling teachers that their notetaking routine is broken. It probably works for most students! I’m arguing that the time you put into a notetaking routine might be better spent on something else.

All of those concerns I listed above are possible to solve. Possible, but time-consuming. And time is precious. I could make sure every student has usable notes for every task we do. I could ensure that my future tasks are perfectly aligned so their notes are always useful. We could organize notes. I could have students practice looking back for different pieces of information.

But that is SO. MUCH. TIME. Time organizing notebooks, making sure every student has the notes, getting notes for absent students, flipping back through notes looking for a piece of information. Time that could be used for...well, anything. So many other things that help students learn. More practice, more feedback, more reading, more writing, more questions, more this, more that.

There are lots of pieces of information that could be useful to students in the future. Maybe it’s a formula for a math problem, or a structure for a piece of writing, or a definition, or a map, or an example of something for them to reference if they get stuck. I give students that information. The format varies: often it’s on the board, nice and big, easy to find. Or on an anchor chart that I can leave in the back of the room when we don’t need it, and bring to the front when we do. Or on a piece of paper I give to students. I make sure it’s clear, relevant, and helpful.

Some of you are saying, “but I found my notes so useful for blank and blank and blank.” Cool! I have students write down lots of stuff. They could choose to use those notes if they like. I don’t ban students from taking notes. It’s just not something I require.

Is Notetaking a Skill?

A classic result in cognitive science is that when students refer back to their notes, they overestimate how much they have learned. Looking back at notes is consistently inferior to other, more active ways of studying.

There’s an illusion of learning at play here. A student might look back for something, find it, and think that must mean they have learned it. It might look to a teacher watching students diligently take notes like those neat notes mean learning has taken place.

One risk with notes is that the goal of education becomes to have lots of well-organized notes and resources to reference. To me, the goal of education is for students to have lots of well-organized knowledge in long-term memory. That doesn’t mean references are useless, but it’s a different goal. My experience is that it’s much easier to move information into one’s notes than into one’s long-term memory, and it’s tempting to substitute one goal for the other.

Another common complaint here is that notetaking is a skill, and we should be teaching students to take good notes. I’m not convinced of that. I do want to teach students good study habits. I often tell students that in class we write about what we are learning so that learning is stronger. Putting ideas in your own words, or quizzing yourself, or trying another example on your own are great ways to learn. I’m not convinced that “take thorough notes and reference them in the future” is especially good learning advice.

I’m also just not convinced that notetaking is a generic, transferable skill that we can reliably teach students. I think many students will figure out notetaking skills on their own, whether they are taught or not. Many more won’t develop notetaking skills despite a lot of instruction. Plenty of students have a dozen different teachers who try to teach them how to take notes, and don’t actually ever develop that skill.

A common argument here is that in college, students will have to take notes and learn much more independently. To be blunt, most notetaking routines are about compliance. They’re about copying things down, then finding that thing you wrote down later to prove that you’re a diligent student. That’s not what college is like. Students who show up and try to write down everything the professor says aren’t going to learn much. A student who has practice summarizing ideas in their own words and being an active participant in their learning is going to learn a lot more, whether or not they ever refer back to their notes. Those are the habits I try to model for students.

Jake Edmiston, a business reporter for the Toronto Star, thinks highly of his mustard chicken recipe. So when one batch goes over like a lead balloon, he checks his namesake ingredients and finds that his usual glass pot of Maille Dijon Originale is extraordinarily spicy—a fact he confirms, good reporter that he is, by conducting a blind test with a professional taster. In no time, Edmiston is on the floor of one of Canada’s largest mustard mills, trying to stay dry and avoid mustard dust burns(!), and peppering Unilever, Maille’s owner, with questions. This isn’t exactly a case for Benoit Blanc, but Edmiston’s investigation takes a few surprising turns and drops some serious knowledge on glucosinolates on the way. Every ingredient is in harmony here.

“Do a little dollop,” she said, then turned her back so I could do my work in secret.

I spooned the mustard from my jar onto a plate, and some from another jar of Maille I found in the grocery store that morning. She tried the sample from my jar first and sputtered a little. 

“It’s quite spicy,” she said.

I asked her how much spicier it was than the control jar. 

“Like, 10 times,” she said.

On the Navier–Stokes Millennium Prize Problem

Impressive result from OpenAI, who used an unreleased model to produce a resolution to the Navier–Stokes existence and smoothness problem, one of the seven Millennium Prize Problems that have been subject to a $1,000,000 prize since May 24th, 2000.

The discovery is somewhat overshadowed by accusations of skulduggery from Tristan Buckmaster, an NYU mathematics professor who was collaborating on related problems with Levent Alpöge, an accomplished mathematician who currently works for Anthropic.

Tristan's complaint accompanied a hastily published version of their own results. Here's the PDF describing what happened. The very short version is that Tristan and Levent worked on the problem for almost a year, making extensive use of Claude and Codex (mainly GPT-5.6 Sol), then had a breakthrough on August 15th. The mathematical rumour mill kicked into gear and Tristan and Levent heard that OpenAI had heard that Anthropic had resolved "a major open problem", so they reached out and learned that OpenAI had a team working on a related problem, with a similar approach. Quoting Tristan:

I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI.

I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer.

It gets more complicated from there. The OpenAI team offered to wait for Tristan to publish, or to have him author a paper about their result, but were clear that Levent would not be invited as a co-author due to OpenAI's competitive relationship with his employer.

Here's how OpenAI described their work:

On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems. [...]

The agents arrived at their resolution on Saturday, September 5, about 88 hours after the first agents were launched. Lean formalization and verification took an additional 17 hours via GPT‑6 Astra.

Across all attempted problems, the agents sent 4.9 million messages and used about 300 billion output tokens. In the process of resolving the Navier–Stokes problem, the agents sent 2.7 million messages and used approximately 130 billion output tokens.

(We don't know the cost structure of the internal model they used, but 300 billion output tokens at public API prices for GPT-6 Astra would cost $15,000,000.)

Here's where they provide their perspective on Tristan and Levent's work (emphasis mine):

Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge, an Anthropic employee, and Tristan Buckmaster, a math professor at NYU. After the completion of our full project and Lean verification (on September 6th), believing from the rumor they also had a solution of Navier–Stokes, we reached out to them to offer a concurrent release of our result and to recognize their priority in a joint announcement. [...]

We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models. However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced).

My interpretation of what happened here is that OpenAI heard that some Millennium Prize problems had been solved using LLMs and saw this as an opportunity to demonstrate the power of their latest model, without thinking too hard about the optics of scooping a team who had been using OpenAI's own models to work on this problem for the best part of a year.

This situation appears to mirror what's happening in the world of computer security right now. Anil Madhavapeddy recently pointed out that Just a rumour of a bug is enough to find a security exploit these days, because if someone knows that some software has an unpatched vulnerability, they can set their agents the task of finding it. Is the same now true of mathematics? Just knowing that there is an unpublished solution to a problem might trigger millions of dollars in LLM spending to get there first.

This also highlights one of my ongoing frustrations about how all of this works. When an AI lab says that my data is "used to improve model performance", what does that actually mean?

My two favourite hypothetical questions regarding this used to be:

• If I'm running Codex and one of my API keys accidentally gets consumed in the context, what are the chances that someone else might ask for an API key in the future and get mine back? (I asked someone at OpenAI once and they called this the "regurgitation" problem and assured me that they take great pains to prevent that... but wouldn't describe how.)
• If I brainstorm with ChatGPT about potential new directions for my company, what's the chance that information might be exposed to a competitor in six months' time who asks "what might company X plan to do next"?

My new preferred hypothetical for this is:

• If I use ChatGPT to help me partially solve a Millennium Prize problem, what are the chances that my work will influence training such that a later model helps someone else solve it first?

Via Hacker News

Tags: mathematics, ai, openai, generative-ai, llms, training-data, ai-ethics

I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce. [...]

We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential. The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community, which would reverse centuries of traditions of open science and do serious long-term damage to the future of the field.

— Terence Tao

Tags: ai-ethics, mathematics, ai