Games may be life with all the hard bits removed, but they provide a way to study why people make the choices they make. Traditional games are usually played against a static background: the rewards per outcome are constant. That limits their relevance to behavior because, in real life, the rewards and consequences of strategic choices are ever changing. Now, researchers have used a mathematical model to study a series of games that include evolving strategies and randomly varying returns.

A bit of history

Perhaps the most famous game-theory contest is the prisoner’s dilemma. In the prisoner’s dilemma, a pair of thieves have been captured and are being separately interrogated by the police. If both clam up, they will be punished for a lesser crime. If one prisoner makes a deal (defects) then that prisoner gets to go free and the other gets a heavier sentence. If both make a deal, they both get an in-between punishment.

The person running the game can start it with different rewards for cooperating and defecting to explore how the optimum strategy varies with reward and risk, which the players can figure out by varying the strategies across multiple rounds. Depending on the balance between the reward for staying silent (cooperating) and betrayal, the game stabilizes with everyone betraying everyone. In this simple situation, everyone loses.

Read full article

Comments

I don’t think anyone particularly enjoys typesetting in monospace.

Regular text is okay – at least as okay as it can be:

Right aligning is also fine, as long as you don’t mind counting spaces, but centering already gets tricky, as you don’t have a half space to make things truly even:

And full justification is where things get particularly weird. The spaces are just too large, and cannot be distributed evenly, creating a really unpleasant feeling:

The solution used in typesetting elsewhere is hyphenation, but in monospace hyphenation also feels unpleasant, with the hyphens drawing too much attention to themselves (and subsequently messing up copy and paste):

This is why you don’t see full justification in text files very often.

But there is one more option. You can rewrite the text to choose only words that precisely add up to the line length to avoid any double spaces.

This is exactly what rs1n did in the late 1990s for his guide to Super Metroid:

It’s astonishing, as it goes like this on for 17,000 more words, each right margin perfectly ending on a letter, no twin spaces in sight.

The author lightly covers in the FAQ at the bottom:

What program did you use to justify the text?

None. I just chose words carefully so that everything lined up on the right hand side. Everything was done with an ASCII editor.

I’m sharing this mostly as a curiosity; some rewrites for physical books are par the course to avoid widows and orphans, but you don’t see them as much in onscreen writing.

At the same time, who among us didn’t nod in recognition at least, having once spent hours massaging a button UI string or a tooltip just to get it to fit under the certain width in a densely packed interface?

I like pets, but, for various reasons, I can’t have one. So I flirt with other people’s pets. There is a black cat across the road who likes sitting in my driveway. So I had to buy a new car with a backup camera to make sure the cat is safe. As you can imagine, I couldn’t skip this problem about cats posted by Konstantin Knop on Facebook.

Puzzle. One cat eats one stick of sausage in 27 minutes. You have four identical sticks of sausage and five cats. Using only the cats and the sausages, measure exactly one minute.
All the cats eat at the same constant rate, and all the sausage sticks are identical. You may measure time intervals only between moments when cats finish sausage sticks. At each such moment, you may redistribute the cats among the remaining sausage sticks.

We really, really need to talk about venture capital. Because it’s not “venture capital” anymore.

There’s a huge disconnect between what most people think of VC, where an investor has a big fund and cuts checks to help a founder build a company, and the current reality, where a handful of billionaire extremists use the cover of “VC” to advance an outrageous agenda where they’re accountable to no one.

I’m gonna explain this from a standpoint that almost never gets articulated: I’ve personally raised tens of millions of dollars in venture capital funding as CEO of startups, and been directly involved as a board member or advisor in raising hundreds of millions more. I’ve sat in board rooms, across the table from the people I’m talking about here, or been at the industry events that they frequent. So this isn’t sour grapes because these VCs wouldn’t cut me a check, or some chip on my shoulder about these investors due to a business deal. This is what I know about these bad actors because I’m part of the community of creators and inventors who build the things that they used to invest in — back when they still cared about innovation.

Many of the trends in society and politics that people are most angry about, from data centers being forced down everyone’s throats, to all of our favorite apps and services being enshittified, to politicians being paid to ignore the will of the people, are all being supercharged by these cancer capitalists. They have warped the structure of venture capital into a form of oligarchy that answers to no market, no regulators, and no voters. So it’s worth understanding exactly how they did it.

How Venture Capital became Cancer Capital

I’ll be breaking these points down in further detail, but just to begin framing the concept, I’ll lay out the core idea here in some bullet points (so that you’re not tempted to run this whole thing through an LLM):

• Venture capital was only supposed to be a tiny segment of the overall capital market, but it has expanded to become the primary form of funding that new companies consider — it was never the only way, and it didn’t used to be the default one
• VC was meant to be a small percentage of overall investment because it represents the high-risk, high-reward part of a portfolio; to be healthy, most of a portfolio — or most of an economy — needs to focus on assets that are more stable and predictable. But a cancer grows from a cell that a body needs in small, healthy amounts, and that turns deadly when it grows without limit until it harms, or even kills, its host.
• As regulations have gotten looser in recent years, a handful of venture capital firms have become “do everything” funds that combine private equity with their existing VC businesses, and expand to manage massive stockpiles of tens of billions of dollars
• The 1% of VC firms that get this big stop being exposed to the risk in their own investments at all — when you collect 2% a year to manage $50 billion, that’s a billion dollars landing in your pocket annually whether any company you funded lives or dies. Those firms also stop legally even being venture capital firms, making them unaccountable to markets, founders, or the law — and that’s how they become “Cancer Capital”
• Meanwhile, the 99% of “normal” VCs don’t have the power or funding of the Cancer Capital firms, but are forced to play on the field that those firms define, even if they don’t like the way they do business
• Since the Cancer Capital firms have become so powerful, the overall balance of power between founders and VCs has flipped; instead of founders having a company that VCs would try to fund, now VCs publish extremist political manifestos, and “founders” are just the people who are selected to carry out parts of those plans
• The rest of the world doesn’t know: New founders and workers entering the tech industry are unaware that Cancer Capital has taken over, so many are still trying to play by the old rules, and can’t figure out why their ideas are being pushed into serving the goals of the Cancer Capital firms
• Politicians and media still look at VC as if it works like it did 10 or 20 years ago, and cheer them on like they’re funding job creation or enabling new companies to grow, when their primary goal is concentrating power and wealth into the hands of the Cancer Capital tycoons. They keep getting fooled by this, over and over.
• These days, venture firms are increasingly getting their funds from pension funds and retail retirement accounts, meaning the public (you!) are increasingly holding the bag for the parts of their portfolios that actually have some risk, even if you never intentionally made that choice
• The shift away from IPOs in the tech industry has also encouraged these Cancer Capital firms to find ways to cash out long before companies ever go public, meaning they can make a massive return off of companies that never make a penny of profit, even if regular investors get screwed by the stock of a company once it actually gets listed on the public stock market.
• Part of why this has gotten so corrupt is the way the Cancer Capital firms have transformed themselves into their post-VC forms. Because they’re not legally VC firms anymore, they’re free to buy shares directly from founders, or hold unlimited amounts of publicly-traded stock — exactly what they couldn’t do as regular VCs. They can even sell their investment in a company as an asset to another one of their own funds, and then book the increase in value as a profit, all without the company ever having made a penny. Another racket: a company that’s raised a bunch of cash in a funding round can buy out its early investors if they’re one of these post-VCs, so they can get paid off even if their portfolio company has never made a penny in profits or revenues.

All of this self-dealing, and the way that they’re isolated from any accountability, has made these firms become more and more shameless in their behavior. Former Andreessen Horowitz partner John O’Farrell publicly called out the firm (a rarity — the company is notoriously vindictive towards those who it decides are disloyal) for what he called its “political infiltration” of AI policy. Marc Andreessen, Ben Horowitz and their firm have put $115.3 million into this midterm cycle — nearly double their $63 million in 2024, and more than any other billionaire donor in the country, even including Elon Musk. Molly White, whose Tech Influence Watch tracks this money against FEC filings, shows that a16z alone accounts for more than 20% of all political contributions from the entire cohort of crypto and AI companies it follows. And they’re funneling these funds to candidates in both parties. This is a huge escalation from the tentative baby steps that folks like Zuckerberg were making in the Obama era, working on benign issues like trying to help immigrants.

And of course, it gets a lot worse than just their lobbying. As I have frequently noted, Andreessen Horowitz hired a man as a partner at their firm despite his having no background or qualifications in tech, finance, or startups whatsoever. His only discernible qualification was that he had choked my unarmed neighbor Jordan Neely to death on a subway car.

This is how brazen, how toxic and destructive, we’ve allowed the industry formerly known as venture capital to become. We must understand that it is no longer a financial machine that is used to fund startups, but a political and social machine focused on dismantling democracy and civil society. And it’s time to act accordingly.

Up next: we’ll dive into the specifics of many of the points laid out above, to understand more about how we got here.

Another researcher is challenging OpenAI about the data driving its increasingly impressive array of mathematical discoveries. Just days after a bitter row erupted over whether the company's models benefited from unpublished work, a second mathematician has come forward accusing the AI giant of unethical and "dishonest" behavior and a lack of transparency about the origins of its training data.

In a series of posts on Mastodon, mathematician Andreas Thom raised concerns that interactions he and his colleagues had had with the ChatGPT chatbot before OpenAI's triumphant announcement may have contributed to its success in the field. One of the …

Read the full story at The Verge.

A recent resignation tweet from an Anthropic researcher has everyone talking about the AI apocalypse again. Among other things, he said:

The people building AI earnestly believe that it could kill us all by the end of the decade.

Many people found it hard to believe that AI researchers think this way. Some explained it as a PR campaign to promote AI regulation, or as self-promotion, or as a way to boost AI company stock prices. Others felt it had to be impossible, because if you really believed this you’d be bombing datacenters instead of posting on Twitter.

In fact, not only do many AI researchers¹ seriously believe this, they’ve been thinking and writing about it since the mid-2000s. Eliezer Yudkowsky — the ur-figure for most modern AI safety culture — has been publishing papers since at least 2008 saying that superintelligent AI could destroy all human life. It’s been such a common idea that the AI research community has abbreviated “how likely you think AI is to kill everyone” to “p(doom)” (i.e. the probability² of doomsday) since around 2010.

I know it sounds very silly if you’re not in the AI bubble. But it really is true, and if you assume there has to be some different motive you’ll be deeply confused by what AI researchers say and do. They truly do believe that there is a reasonable chance that superintelligent AI will kill everyone.

This is why AI researchers care so much about “alignment”: building AIs that share genuinely human beliefs and values. If we build a “misaligned” superpowerful AI — an AI with goals that are alien to us — it might sweep humanity away. It could kill everyone deliberately, e.g. to stop us getting in the way of some goal. It could kill everyone in passing, e.g. like we might pave over an anthill to build a road. Either way, everyone dies.

How AI could kill everyone

Okay, but how? What do these people think is actually going to happen? AIs are computer programs running in a datacenter somewhere. How could they possibly cause the extinction of humanity? Wouldn’t someone just turn them off? Unsurprisingly, AI research nerds have come up with some concrete answers to this in the last two decades. Here they are, in order of plausibility:

An AI could make and release some super-pathogen or virus. In the hope that current AIs can achieve huge breakthroughs in medicine similar to the ones they’ve already achieved in mathematics, we’re setting up autonomous labs³. Bioweapons have been terrifying biologists for decades, and with good reason. Historical pandemics have killed up to 80% of affected human populations, and tend to be defeated by accident: the disease happens to evolve into a less virulent strain, or short incubation periods mean that infected people can’t carry the disease far, or some percentage of the population is naturally immune. A plague designed to be maximally fatal — or several plagues in quick succession with different characteristics — could be much worse.

Alternatively, an AI could trigger global thermonuclear war. We’re already seeing AI be integrated into military and government decision-making processes. If a rogue AI managed to set off a bunch of nukes, or to coordinate⁴ with other countries’ rogue AIs to nuke each other, we could be in an ordinary nuclear apocalypse scenario: billions dead in the strikes, billions dead in the ensuing famine, and so on. It’s commonly assumed that a post-global-thermonuclear-war Earth would still support some tiny human population, but a determined AI could surely find some way to mop up the stragglers.

There are some other theories. Once robotics has permeated the world economy, an AI could take over the robots (including drones) to kill everyone, like in Terminator. Or AIs could take advantage of nanotechnology to create self-replicating machines that turn the world into “grey goo”⁵. Or AIs could terraform the planet so as to make it unliveable for humans (as in Nick Bostrom’s famous paperclip example). Or they could do something else that our puny human brains aren’t able to think of.

Counterarguments

One common counter-argument⁶ here is to say “well, it’d be impossible to extinguish all human life — what about undiscovered tribes in the Amazon, or survivors living in the ruins of modern-day cities?” I don’t know, man. At some point you’re just conceding the argument: the policy positions you’d adopt if you thought AI might wipe out 99% of humans are the same as if you thought it might wipe out 100%. And like I said above, if an AI can kill almost everyone, it’s probably smart and capable enough to finish the job somehow.

Another is to say that the government will simply step in and nationalize the AI labs when the situation gets too dangerous. Maybe! But this kind of concedes the argument: a technology important enough to be fully taken over by the government is a terrifyingly dangerous technology.

A third is to say “well, someone would just turn it off”. I don’t find this plausible at all: an AI powerful enough to build a super-plague is an AI sophisticated enough to pretend it’s curing cancer, or to exfiltrate itself to some datacenter where it won’t be turned off, or to take some other countermeasures.

The winner takes it all

Why would you work in AI, if you believe this? Why wouldn’t you go live in the woods somewhere, or start bombing datacenters, or assassinating AI lab CEOs? For a few reasons.

An AI powerful enough to end humanity is an AI powerful enough to save it. I wrote about this in Help peer: many AI researchers believe that the only way for humanity to truly survive long-term is with the help of superintelligent AIs, so long as someone can figure out alignment.

Isn’t this a huge risk? Maybe not. If somebody is going to build superintelligent AI, you might be obligated to try and do it first. You can’t go and bomb every datacenter in the world, after all.

Why does it matter who’s first? Some popular theories of AI development involve a “foom” or “hard takeoff”: the first time someone really cracks self-improving AI, capabilities will increase exponentially, because smart AI will be better able to make itself smarter, ad infinitum⁷. There are no draws in the AI race. The first lab to figure out smart human-like intelligence will be the first one to figure out wildly superhuman intelligence, and thus will be in a position to stop anyone else from doing it.

This is an under-discussed point in the AI risk debate. Lots of AI researchers believe that the first thing a true superintelligence will do is reach out and stop all other AI research: either by hacking the labs, persuading them to stop, or literally drone-striking their datacenters. According to this view, if you’re an AI researcher and you think you can build an aligned AI, you should be working 24/7 so you can manifest God, and you should wake up every morning gripped by the fear that someone elsewhere has manifested the Devil, and your training datacenter no longer exists.

Conclusion

I have been on the fringes of this world for my entire adult life. I read Meditations on Moloch as a young adult and wanted to get into AI. I am one of the few people to read the entirety of the Sequences, Eliezer Yudkowsky’s million-plus-word magnum opus about rationality. I was too young for the Extropians mailing list, but I’ve spent years on LessWrong. On the other hand, I’m not a card-carrying rationalist: I think if you have a strong intuition on one side and a convincing-sounding argument on the other, you should pick the intuition⁸. I’m a deontologist, not a utilitarian. I don’t even live in San Francisco!

I’m conflicted about AI risk. The current behavior of AI agents does seem to vindicate a lot of the early science-fiction-sounding worries of the AI doomers, but modern LLMs are a lot more human-like than the alien minds in the apocalypse scenarios, and in general it does just seem too silly to credit (I guess I’m picking the intuition here).

However, it bothers me to see people dismissing these people as part of a PR operation, or as liars looking to boost an upcoming AI lab IPO, or as isolated crazies who haven’t thought their position through. Whatever else you say about the AI doomers, they have more than two decades’ history of explicitly spelling out exactly what they believe and why, even when it was complete science fiction to talk about AI at all. They’ve earned the right to be treated as sincere.