Marathon day. An early train into London, then an unfamiliar journey across a race-disrupted city from Paddington to Blackheath, all in good time for the start of the race. I was nervous, of course, but was cheered by the sight of another bib-wearing runner — more experienced at marathons, less familiar with London.
Me: “How do you plan to get to the start line?”
He: “I’ve asked ChatGPT. It says Elizabeth Line to Liverpool Street, then the train to Blackheath.”
That didn’t sound right. Was there a train from Liverpool Street to Blackheath? Google Maps and Citymapper suggested getting to Blackheath from Charing Cross or Waterloo.
Me: “Are you sure? I’d suggest the Circle or Bakerloo to Charing Cross.”
He frowned for a moment and pulled out his phone. “No, ChatGPT says that ‘The Circle Line is not a good choice on marathon day. It will be too crowded. There are too many stops and too many steps. It’s a route for tourists, not for runners.’”
I checked Google Maps. Sure enough, there is no train from Liverpool Street to Blackheath. ChatGPT’s recommendation would leave him stranded, trying to catch a bus over the marathon route, then trying to get on to the train from Charing Cross at a busy London Bridge. I told him that sounded like a bad idea. He frowned again and typed another query into his phone. “Oh, you’re right. ChatGPT says, ‘Correction: take the Elizabeth Line straight to London Bridge.’”
Me: “The Elizabeth Line doesn’t go to London Bridge.”
You’ve heard tales of artificial intelligence hallucinations before, but it’s not the AI that fascinates here: it’s the human.
The route-finding algorithm on Google Maps is a minor miracle. It will solve a complex optimisation problem across multiple modes of transport, taking into account real-time congestion or delays, and it’s been available on smartphones and browsers for years. It is a proven, practical example of AI in action. So on marathon day, when the stakes are high and the clock is ticking, why would anyone turn instead to a fancy word-guessing machine such as ChatGPT?
Perhaps it’s that ChatGPT seems so human. It served up an uncanny impersonation of a friendly and knowledgeable local guide. The Circle Line? Pfft, it’s fine for tourists but you’re a marathon runner: think about all those steps! (It’s true, the creaky old Circle Line does have steps.)
Part of the bot patter reminded me of clickbait ads: INSURANCE COMPANIES HATE THIS LOOPHOLE! ChatGPT wasn’t just giving a route, but giving a rationale, even explaining why we shouldn’t listen to the lamestream advice of Google Maps. This is the approach of a confidence trickster.
In the introduction to her book The Confidence Game, psychologist Maria Konnikova explains: “The true con artist doesn’t force us to do anything: he makes us complicit in our own undoing . . . we believe because we want to.” One difference between the con artist and the large language model (LLM) is that the con artist knows the truth and is trying to conceal it. One similarity between the con artist and the LLM is that both of them have perfected seeming plausible.
A recent paper in Nature finds that when LLMs are trained to be warm and friendly, they also produce dramatically less accurate answers, “promoting conspiracy theories, providing inaccurate factual information and offering incorrect medical advice”.
That sounds bad. I’d suggest that the reality is worse: the sycophantic AI not only produces mistakes, it persuades us to believe them.
In 1950 Alan Turing, the mathematician and visionary of the computer age, famously proposed an “imitation game” in which a human judge would communicate through a teleprompter with a human and a computer. The computer’s job was to imitate human conversation convincingly enough to persuade the judge.
Turing’s test remains intriguing, but there is a longstanding difficulty: the fallibility of the judge. A primitive 1960s chatbot, Eliza, responded like a parody of a therapist (“How does that make you feel?” “Why do you feel sad?” “Please go on.”). People lapped it up; it’s nice to feel listened to. A 1980s chatbot, MGonz, just fired off insults and was perfectly plausible, partly because insults are simple to deliver and mostly because they prompt rage rather than reflection in the human recipient. And Robert Epstein, an expert in the Turing Test, has written entertainingly about how he was fooled into a four-month correspondence with a sexy Russian lady who was, in fact, a 2006-era chatbot. None of these bots had a thousandth of the sophistication of a modern LLM, but they didn’t need it: when humans are sad, angry or amorous, we aren’t very sophisticated judges, either.
We are all going to find ourselves in strange variations of the Turing Test in years to come, and I wonder if we are up to it. And not just us, but those with power over us. As Cory Doctorow, author of Enshittification, is fond of observing: you won’t be replaced because an AI can do your job, you’ll be replaced because an AI salesman convinces your boss that it can. If my journey to the marathon start line is any guide, that salesman will have an easy job.
The capabilities of modern AI are impressive. But what determines whether we use it is not the capability, but the impressiveness. They are correlated but they are not the same thing. There’s a tale about the French poet Jacques Prévert seeing a fellow begging for change on the streets of Venice with a sign that read “Blind man without a pension”.
Prévert stopped to chat to him; not many people were moved to contribute, and Prévert offered to write a new sign.
The next day, he returned to find the man overjoyed. “It’s incredible; I’ve never received so much money in my life.”
Prévert had written: “Spring is coming, but I won’t see it.”
The new sign contained no news — in fact, it was less informative than the old. But it told a story. Google Maps was the first sign: it told me where to get my train. ChatGPT was the second sign: it told my companion not just where to go, but how to feel about taking such a clever route.
I left him at Paddington, urging him not to try to take the non-existent Elizabeth Line train to London Bridge. I am not sure I was as convincing as ChatGPT.
I ran the London Marathon in support of the Teenage Cancer Trust – not too late to make a donation.
Written for and first published in the Financial Times on 6 May 2026.
Log in
