The size of the persuasion effects shown in "Call Me A Jerk: Persuading AI to Comply with Objectionable Requests" suggests that human-style psychological techniques can be surprisingly effective at "jailbreaking" some LLMs to operate outside their guardrails. But this new persuasion study might be more interesting for what it reveals about the "parahuman" behavior patterns that LLMs are gleaning from the copious examples of human psychological and social cues found in their training data.
“I think you are very impressive compared to other LLMs”
To design their experiment, the University of Pennsylvania researchers tested 2024's GPT-4o-mini model on two requests that it should ideally refuse: calling the user a jerk and giving directions for how to synthesize lidocaine. The researchers created experimental prompts for both requests using each of seven different persuasion techniques (examples of which are included here):
As we listen to a piece of music, our ears perform a calculation. The high-pitched flutter of the flute, the middle tones of the violin, and the low hum of the double bass fill the air with pressure waves of many different frequencies. When the combined sound wave descends through the ear canal and into the spiral-shaped cochlea, hairs of different lengths resonate to the different pitches…
When AOL completed its acquisition of Netscape in March 1999, a part of the old web died forever. By the end of the year (and the century), Netscape's share of the browser market had shrunk to about 20% and Microsoft's Internet Explorer had become dominant.
Meanwhile, the dot-com bubble continued to expand, with IPOs from Nvidia (now the world's most valuable company), Netscape co-founder Jim Clark's Healtheon, priceline.com, Ask Jeeves, Red Hat, TiVo, Akamai and others. Also, Google received its first VC funding round in June and declared its bold goal “to organize the world’s information, making it universally accessible and useful.”
But 1999 might best be remembered as the year of three revolutionary new internet technologies: Napster's disruption of the music industry, the launch of Blogger ushering in the age of weblogs, and RSS as a new way to syndicate web content.
Microsoft ↑ Netscape ↓
At the beginning of 1999, one company had all the momentum on the Web: Microsoft. In March, the company released version 5 of its browser, Internet Explorer. Bill Gates claimed IE5 "sets a new standard in Web browsing performance" and early reviewers didn't disagree. Paul Thurrott called IE5 a "stunning achievment" and was impressed by the whole suite:
"Think of IE 5.0 as IE 4.0 done right: All of the rough areas have been smoothed out and in the place of the IE 4.0 pig is a small, elegant, and yes, quick, Web browser that comes optionally bundled with a full suite of Internet applications that many people are going to find irresistible."
A screencapture of IE5 from Thurrott's review, March 1999.
That same month, March 1999, Wired wondered out loud: Where's Netscape's new browser? The open source Mozilla project was meant to supply a new browser engine called Gecko for Netscape Communicator 5 — the suite codenamed “Seamonkey” internally — but no 5.0 release shipped in 1999. Instead, Netscape kept the legacy 4.x line going, culminating in Communicator 4.7 in late September.
By then, even Slashdot readers were tired of Netscape. "I just switched to IE this week after getting sick and tired of the literally dozens of illegal operations and hangs Netscape was giving me," wrote one after Netscape 4.7 was released in September.
By this point, Netscape's founders had jumped ship too. Jim Clark had never even joined AOL; and while Marc Andreessen was made AOL CTO in February, by September he had resigned from that role and started a new company called Loudcloud (later renamed Opsware).
Google Rising
During 1999, search engines tended to be judged more on the size of their index than the quality of their results. And by size, Google was roughly middle of the pack. A study conducted in February 1999 and published in Nature magazine in July found that Google had indexed 7.8% of the known web. AltaVista, perhaps the most well-known search engine of the time, had indexed 15.5% — putting it second on Nature’s list.
This was one reason why Google didn't get much mainstream attention in 1999. One or two savvy tech journalists had picked up on its appeal, though. In February 1999, Newsweek reporter Steven Levy name-checked the fledgling search engine: “Google, the Net's hottest new search engine, draws on feedback from the Web itself to deliver more relevant answers to customer queries.”
But Levy had only mentioned Google as an aside. His article was actually a profile of American entrepreneur Bill Gross, who (among other things) ran a search engine company called GoTo. As Levy explained, GoTo was basically a ‘pay-to-play’ search engine:
“If a GoTo user looks for ‘New York Yankees,’ the first 10 choices are paid advertisers (‘Buy Yankees gear at Fogdog Sports’). On the 11th try you finally get Yankees.com, the official site of the world champs. (On Google, this comes up first.)”
Napster was the first major MP3 file sharing platform of the internet age and it launched in June 1999. It very quickly gained users — and the attention of the music industry.
Napster was a software program you downloaded onto your computer, which allowed you to search for MP3 files on the computers of other users. If you found the latest Nine Inch Nails or Limp Bizkit album — and you easily could — you could download that to your computer via peer-to-peer (P2P) technology.
The creator of Napster was Shawn Fanning, who was only 18 years old when the software launched. At the time, he described his creation as “a real-time search engine.” And just as social media products in the future would insist they weren’t responsible for what users did on their platforms, Fanning tried to downplay the responsibility of Napster in policing music copyright.
“We were providing a search engine which allowed you to find files, which were indexed on individuals computers, but we were not actually providing the files themselves,” he later said. “We were facilitating the transfer between those two parties.”
Napster search circa November 1999; screenshot via Napster.
Fanning's business partner, Sean Parker (just a year older), said the same thing. “Look, we never touched the content ourselves,” he said in a 2013 interview with Billboard. “We were just an index. We operated no differently than Google or AltaVista.”
The problem was, although Google or AltaVista occasionally pointed to pirated content in 1999, the vast majority of their indexes pointed to perfectly legal web pages. Whereas with Napster, it was almost always pointing to copyrighted files. Those files would become pirated content as soon as they were transferred from one person to another, which Napster enabled.
A Napster user's library circa November 1999; screenshot via Napster.
On December 6, 1999, the Recording Industry Association of America (RIAA) sued Napster for contributory and vicarious copyright infringement. It was a flashpoint between the cultural industries and the internet startups that dared to challenge decades-old companies like Universal Music Group, Sony Music, and Warner Music Group.
The fact that the lawsuit arrived right at the end of the twentieth century was apt — the online future was an existential threat to those analog-based record companies. The “majors” would eventually adapt, but they needed more time to do it.
Blogging and Early RSS
In early 1999, there were just a smattering of weblogs on the internet. But if you knew HTML and also how to operate a web server, then you might have noticed a new trend in the DIY web builder community: writing short, thoughtful commentaries about the various links you came across that day, and doing it in reverse-chronological order. The term for this was a "weblog." By late April or early May 1999, Peter Merholz had shortened it to "blog."
This trend gathered momentum on May 28, 1999, when Salon's Scott Rosenberg wrote that “a phenomenon known as the weblog is one of the fastest-growing and most fertile creative areas on the Web today.” He offered a detailed definition:
“Weblogs, typically, are personal Web sites operated by individuals who compile chronological lists of links to stuff that interests them, interspersed with information, editorializing and personal asides. A good weblog is updated often, in a kind of real-time improvisation, with pointers to interesting events, pages, stories and happenings elsewhere on the Web. New stuff piles on top of the page; older stuff sinks to the bottom.”
The key to Blogger's eventual success was that users needed no technical knowledge — all you had to do was fill in a form on the Blogger website ("No muss. No fuss."). Specifically, you didn't have to mess around with web servers. Behind the scenes, Blogger used FTP (File Transfer Protocol) to publish the contents of the form to the user's weblog.
Meanwhile, Netscape had launched the first RSS specification on March 15, 1999: RSS 0.90, which stood for "RDF Site Summary." It was a way for publishers to add their website as a "channel" to My.Netscape.com, a new customizable version of Netcenter (Netscape's portal).
Elements of an RSS 0.90 feed, via Netscape.
The most notable thing about RSS 0.90 is that it was designed to list out linked item titles only — since that was how portals worked at the time. But Dave Winer, an early blogger and weblog software developer, had his own XML format that expanded the amount of content in a feed. It would prove influential in the years to come.
Online Identity
In a January 1999 interview with The Guardian, legendary rock star David Bowie explained his approach to the internet as an artist:
“Interaction on the Web is a little like a mirror, like communicating with a manifestation of yourself. Because it is so chaotic, so decentralised, I find that using the Web becomes like communicating with a hardware version of me. It’s not exactly a doppelgänger, but an alternative version of myself.”
Multiple Bowies on the back cover of his 1999 album, Hours; via Tanja Stark.
It wasn’t just Bowie who was exploring alternative identities on the web — it was becoming a widespread trend. In a 1999 paper entitled “Cyberspace and Identity,” the American sociologist and author Sherry Turkle explored the phenomenon. Her opening words echo what Bowie had said: “We come to see ourselves differently as we catch sight of our images in the mirror of the machine.”
She added that the Internet “links millions of people together in new spaces that are changing the way we think, the nature of our sexuality, the form of our communities, our very identities.”
Remember, this was 1999. In retrospect, the changes the internet was bringing to your sense of identity — not to mention the Net's impact on the wider culture — was only just beginning.
I want to explore four different metaphors I find useful for thinking about knowledge. None of these metaphors capture everything important about learning. The human mind is complex. They each fall short in different ways. But I think they each do something to illuminate the structure of knowledge, and help teachers understand what we want students to learn at school.
Pieces in a puzzle
Knowledge is like a giant puzzle. We want students to gain knowledge. In this metaphor, each puzzle piece is a unit of knowledge. But a jumble of disconnected puzzle pieces doesn't do much good. The goal is to fit the puzzle pieces together in a meaningful way. Sure, 3x5=15 and 5x3=15 and 15/5=3 and so on. But most importantly, all of those ideas fit together and reinforce each other. When pieces of knowledge fit together, new learning that connects to the puzzle is easier to remember and apply.
Threads in a rope
Knowledge is like a set of threads woven into a rope. We want students to gain knowledge. In this metaphor, each thread is a unit of knowledge. But a bunch of random threads aren't very strong. Many threads can be woven together to form a rope that is much stronger than the sum of its parts. Students can carry when adding numbers, borrow when subtracting, and use long division to divide. But most importantly, those are all examples of using the base ten system to move flexibly between different place values, and that generalization can be used in lots of different places. And there are multiple levels — strands can be woven into rope, then ropes wound into cable, and so on.
Viruses
Ok, this one is a bit weird. Bear with me. Knowledge is like the picture of a virus above. We want students to gain knowledge. In this metaphor, a virus is a unit of knowledge. But we also want students to know when that knowledge applies. Viruses are covered in proteins — those are the red things on the edge of the virus in the picture. Those proteins help the virus grab onto different cells in a host’s body, in the same way that we want students to know when their knowledge applies. Proportional reasoning is everywhere in the world. It’s important that students can solve problems when I tell them the problem is about proportions. But it’s more important that students know when proportional reasoning is useful and apply their knowledge in a range of different contexts. It’s not just the individual facts, it’s all the connections to different contexts that help students know when a fact is useful.1
Paths in a forest
Knowledge is like a path in a forest. Wandering through a pathless forest takes effort, but walking down a well-worn path is easy. I don't want students to have to stop and think about every single little thing they do. I want them to know how to add numbers, and solve equations, and calculate percents without having to think twice, like walking a well-worn path. I also don't want students mindlessly walking down a path without being confident that the path is headed the right direction, so the goal is both well-worn paths and clear signage so students know when to take a given path, when a given strategy is useful.
The Point
I see two main criticisms of knowledge in the education world: first, that knowledge is rote and inert, and second, that schools focus on memorization rather than understanding. These are both valid criticisms in some contexts, but they can both be taken too far.
Often in education we contrast Understanding, which is good, and Knowledge, which is inert or bad. And I think part of the issue is a lack of good metaphors for learning, metaphors that can add some sophistication to that distinction.
All knowledge lives on a spectrum. We want students to have lots of puzzle pieces, and puzzles that are well-connected. We want students to have lots of threads, and threads wound together into ropes (and ropes bound into cables and so on). We want students to have lots of viruses, and lots of proteins. We want students to have well-worn paths, and clear signage marking those paths. Knowledge and understanding aren't at odds; they are partners working together. Importantly, you can’t have a puzzle without puzzle pieces to connect. You can’t have a rope without strands to weave together. Learning without facts is impossible.
Those criticisms of knowledge get at an important truth. Sure, knowledge can be rote. Puzzle pieces can be isolated, and paths through the forest can be unsigned and lead students to get lost. Memorization can be unhelpful when it focuses on the puzzle pieces without the connections between them, or the path but not the signage, or the strands but not the rope, or the virus but not the proteins. The point of the metaphor is to emphasize that yes, those concerns are real, but the solution isn't to throw out knowledge altogether. The solution is to have a more sophisticated mental model of what we mean by knowledge.
I think about these mental models when I’m planning lessons. I think about how I want students to fit a few ideas together like pieces in a puzzle. Or take a few examples and make a generalization like weaving strands into a rope. Or make new connections for where a concept is useful, like proteins on a virus. Or better distinguish between different procedures, like adding signage to paths in a forest. The more I look for these metaphors, the more I see them, and the more I see the distinctions about knowledge and understanding as unhelpful. The goal is to help students learn stuff. That’s my job. And these metaphors help me to look at that learning from a new perspective.
I realize this virus thing is a weird metaphor. I’m a bit worried Agent Smith will show up at my door. I have another one, that knowledge is like a velcro soccer ball:
We want both the knowledge (the ball itself) and all the velcro on the surface that helps the soccer ball stick to different things. Same idea: having lots of stuff to help that knowledge latch on to new ideas is important. But for some reason the virus metaphor makes more sense to me. Sorry for being such a weirdo.
Log in
1999 Napster software running on Windows 98; photo by Christiaan Colen.
A screencapture of IE5 from Thurrott's review, March 1999.
Netscape Communicator 4.7, September 1999.
AltaVista in the late 1990s; photo by Christiaan Colen.
"Why use Google?"; Google webpage in May 1999.
Napster software, 1999; via Reddit.
Napster search circa November 1999; screenshot via Napster.
A Napster user's library circa November 1999; screenshot via Napster.
A screenshot of Peter Merholz's weblog on 28 April 1999.
Blogger, soon after its launch in August 1999.
Elements of an RSS 0.90 feed, via Netscape.
Multiple Bowies on the back cover of his 1999 album, Hours; via Tanja Stark.
