One of the dreams of Web 2.0 was that website would speak unto website. An "Application Programming Interface" (API) would give programmatic access to structured data, allowing services to seamlessly integrate content from each other. Users would be able to quickly grab data from multiple sources and use them for their own purposes. No registration or API keys, no tedious EULAs or meetings. Just pure synergy!

Is that dream dead? If so, what killed it?

A decade ago, I posted a plea looking for Easy APIs Without Authentication with a follow up post two years later. I wanted some resources that students could use with minimal fuss. Are any of the APIs from 10 years ago still alive?

Alive

These ones are still around:

• Wikipedia - Yes! Still going strong.
• Police.uk - Yes! After a brief dalliance with API registration, it is now back to being completely free and open.
• Google Books ISBN - Yes! Obviously Google have forgotten it exists; otherwise it would have been killed off by now!
• iTunes Lookup - Yes! Possibly the only thing Apple don't charge a premium for.
• Pokémon API - and still receiving frequent updates.
• MusicBrainz - this Internet stalwart will never die.
• Open Notify - a collection of space APIs, although the code hasn't been updated in ages.

Dead

These have shuffled off this mortal coil:

• BBC Radio 1 - No.
• Twitter URL statistics - LOLSOB No.
• Star Wars API - No.
• British National Bibliography - No. Dead due, I think to the British Library's cyber attack.
• Football Data - gone.

API Key Required

These are still alive, but you either need to pay or register to use them:

• Google Location
• Spotify
• OpenMovieDB
• Open Air Quality

What Happened?

Something something … enshittification … blah blah … zero interest rate phenomenon … yadda yadda our incredible journey …

But back in the land of rationality, I've had a lots of experiences running APIs and helping people who run them. The closure and lockdown of APIs usually comes down to one or more of the following.

APIs cost money to run. Yes, even the static ones have a non-zero cost. That's fine if you're prepared to endless subsidise them - but it is hard to justify if there's no return on investment. Anyway, who is using all this bandwidth? Which leads on to:

Lack of analytics. Yes, I know tracking is the devil, but it is hard to build a service if you don't know who is using it. Sure, you can see traffic, but you can't tell if it is useful to the end consumer, or what value you can share. There's no way to communicate with an anonymous consumer. Which, of course, takes us to the next barrier:

Communication is key. If you need to change your API, there's no way to tell users that a change is coming. That might be the announcement of a deprecation, an outage, or an enhancement. You can try smuggling error messages into your responses and hoping someone notices a failing service somewhere - but it's much easier to email everyone who has an API key. And you know what else keys are good for?

Stopping abuse. It'd be nice if everyone played nice online; but some people are raging arseholes. Being able to throttle bad actors (figuratively or literally) is a desirable feature. On a resource constrained service, you sometimes have to put rules in place.

Still, if you know of any good open APIs which don't require registration, and that you think will survive until 2036, please drop a link in the comments.

I have a credit card with HSBC¹. It doesn’t see much use², but I still get a monthly statement from them, and an email to say it’s available.

Not long ago I received a letter from them telling me that emails to me were being “returned undelivered” and they needed me to update the email address on my account.

“What’s happening?”

I logged into my account, per the instructions in the letter, and discovered my correct email address already right there, much to my… lack of surprise³.

So I kicked off a live chat via their app, with an agent called Ankitha. Over the course of a drawn-out hour-long conversation, they repeatedly told to tell me how to update my email address (which was never my question). Eventually, when they understood that my email address was already correct, then they concluded the call, saying (emphasis mine):

I can understand your frustration, but if the bank has sent the letter, you will have to update the e-mail address.

This is the point at which a normal person would probably just change the email address in their online banking to a “spare” email address.

But aside from the fact that I’d rather not⁴, by this point I’d caught the scent of a deeper underlying issue. After all, didn’t I have a conversation a little like this one but with a different bank, about four years ago?

So I called Customer Services directly⁵, who told me that if my email address is already correct then I can ignore their letter.

I suggested that perhaps their letter template might need updating so it doesn’t say “action required” if action is not required. Or that perhaps what they mean to say is “action required: check your email address is correct”.

So anyway, apparently everything’s fine… although I reserved final judgement until I’d seen that they were still sending me emails!

“Action required”

I think I can place a solid guess about what went wrong here. But it makes me feel like we’re living in the Darkest Timeline.

I dissected HSBC’s latest email to me: it was of the “your latest statement is available” variety. Deep within the email, down at the bottom, is this code:

<img src="http://www.email1.hsbc.co.uk:8080/Tm90IHRoZSByZWFsIEhTQkMgcGF5bG9hZA=="
   width="1"
  height="1"
     alt="">

<img src="http://www.email1.hsbc.co.uk:8080/QWxzbyBub3QgcmVhbCBIU0JDIHBheWxvYWQ="
   width="1"
  height="1"
     alt="">

What you’re seeing are two tracking pixels: tiny 1×1 pixel images, usually transparent or white-on-white to make them even-more invisible, used to surreptitiously track when somebody reads an email. When you open an email from HSBC – potentially every time you open an email from them – your email client connects to those web addresses to get the necessary images. The code at the end of each identifies the email they were contained within, which in turn can be linked back to the recipient.

You know how invasive a read-receipt feels? Tracking pixels are like those… but turned up to eleven. While a read-receipt only says “the recipient read this email” (usually only after the recipient gives consent for it to do so), a tracking pixel can often track when and how often you refer to an email⁶.

If I re-read a year-old email from HSBC, they’re saying that they want to know about it.

But it gets worse. Because HSBC are using http://, rather than https:// URLs for their tracking pixels, they’re also saying that every time you read an email from them, they’d like everybody on the same network as you to be able to know that you did so, too. If you’re at my house, on my WiFi, and you open an email from HSBC, not only might HSBC know about it, but I might know about it too.

An easily-avoidable security failure there, HSBC… which isn’t the kind of thing one hopes to hear about a bank!

But… tracking pixels don’t actually work. At least, they doesn’t work on me. Like many privacy-conscious individuals, my devices are configured to block tracking pixels (and a variety of other instruments of surveillance capitalism) right out of the gate.

This means that even though I do read most of the non-spam email that lands in my Inbox, the sender doesn’t get to know that I did so unless I choose to tell them. This is the way that email was designed to work, and is the only way that a sender can be confident that it will work.

But we’re in the Darkest Timeline. Tracking pixels have become so endemic that HSBC have clearly come to the opinion that if they can’t track when I open their emails, I must not be receiving their emails. So they wrote me a letter to tell me that my emails have been “returned undelivered” (which seems to be an outright lie).

Surveillance capitalism has become so ubiquitous that it’s become transparent. Transparent like the invisible spies at the bottom of your bank’s emails.

So in summary, with only a little speculation:

1. Surveillance capitalism became widespread enough that HSBC came to assume that tracking pixels have bulletproof reliability.
2. HSBC started using tracking pixels them to check whether emails are being received (even though that’s not what they do when they are reliable, which they’re not).
   • (Oh, and their tracking pixels are badly-implemented, if they worked they’d “leak” data to other people on my network⁷.)
3. Eventually, HSBC assumed their tracking was bulletproof. Because HSBC couldn’t track how often, when, and where I was reading their emails… they posted me a letter to tell me I needed to change my email address.

What do I think HSBC should do?

Instead of sending me a misleading letter about undelivered emails, perhaps a better approach for HSBC could be:

1. At an absolute minimum, stop using unencrypted connections for tracking pixels. I do not want to open a bank email on a cafe’s public WiFi and have everybody in the cafe potentially know who I bank with… and that I just opened an email from them! I certainly don’t want attackers injecting content into the bottom of legitimate emails.
2. Stop assuming that if somebody blocks your attempts to spy on them via your emails, it means they’re not getting your emails. It doesn’t mean that. It’s never meant that. There are all kinds of reasons that your tracking pixels might not work, and they’re not even all privacy-related reasons!
3. Or, better yet: just stop trying to surveil your customers’ email habits in the first place? You already sit on a wealth of personal and financial information which you can, and probably do, data-mine for your own benefit. Can you at least try to pay lip service to your own published principles on the ethical use of data and, if I may quote them, “use only that data which is appropriate for the purpose” and “embed privacy considerations into design and approval processes”.
4. If you need to check that an email address is valid, do that, not an unreliable proxy for it. Instead of this letter, you could have sent an email that said “We need to check that you’re receiving our emails. Please click this link to confirm that you are.” This not only achieves informed consent for your tracking, but it can be more-secure too because you can authenticate the user during the process.

Also, to quote your own principles once more: when you make a mistake like assuming your spying is a flawless way to detect the validity of email addresses, perhaps you should “be transparent with our customers and other stakeholders about how we use their data”.

Wouldn’t that be better than writing to a customer to say that their emails are being returned undelivered (when they’re not)… and then having your staff tell them that having received such an email they have no choice but to change the email address they use (which is then disputed by your other staff)?

</rant>

🕵️‍♀️ My RSS feed doesn't track you. Dan Q - 1, HSBC - nil,I guess. 😅

When people talk glowingly about the heyday of social media being good, what they’re really talking about is riffing. Before Donald Trump became the president and screwed up all posting that isn’t about him, people would iterate on the most absurd joke possible. For the last few days, a handful of people have been riffing on the idea of YouTube thumbnails but from the point of view of babies, and it’s mostly been good. For posterity, it is worth remembering this riff. 

The bit is a riff on something we have all seen a million times: horribly condescending YouTube thumbnails with bait text. The original post comes from an artist called

Jamie. The first meme, “why I’m switching..…” shows a contemplative baby examining a bead maze next to a “greater than” sign and some toy blocks. From there, the rest was history. 

“At the end of the day it’s been fun to see people enjoying it, especially in a time where [every] headline online has been miserable,” Jamie told me. “If this silly thing was able to brighten people’s day at all then I’m happy. I’m glad my mutuals seem to be having a laugh instead of being annoyed, haha.”

Here are some of the best YouTube bait thumbnails but they got babies in ‘em. Enjoy.

Also worth mentioning, they've moved on to animals now.

The Five Levels: from Spicy Autocomplete to the Dark Factory

0. Spicy autocomplete, aka original GitHub Copilot or copying and pasting snippets from ChatGPT.
1. The coding intern, writing unimportant snippets and boilerplate with full human review.
2. The junior developer, pair programming with the model but still reviewing every line.
3. The developer. Most code is generated by AI, and you take on the role of full-time code reviewer.
4. The engineering team. You're more of an engineering manager or product/program/project manager. You collaborate on specs and plans, the agents do the work.
5. The dark software factory, like a factory run by robots where the lights are out because robots don't need to see.

Dan says about that last category:

At level 5, it's not really a car any more. You're not really running anybody else's software any more. And your software process isn't really a software process any more. It's a black box that turns specs into software.

Why Dark? Maybe you've heard of the Fanuc Dark Factory, the robot factory staffed by robots. It's dark, because it's a place where humans are neither needed nor welcome.

I know a handful of people who are doing this. They're small teams, less than five people. And what they're doing is nearly unbelievable -- and it will likely be our future.

I've talked to one team that's doing the pattern hinted at here. It was fascinating. The key characteristics:

• Nobody reviews AI-produced code, ever. They don't even look at it.
• The goal of the system is to prove that the system works. A huge amount of the coding agent work goes into testing and tooling and simulating related systems and running demos.
• The role of the humans is to design that system - to find new patterns that can help the agents work more effectively and demonstrate that the software they are building is robust and effective.

It was a tiny team and they stuff they had built in just a few months looked very convincing to me. Some of them had 20+ years of experience as software developers working on systems with high reliability requirements, so they were not approaching this from a naive perspective.

I'm hoping they come out of stealth soon because I can't really share more details than this.

Tags: ai, generative-ai, llms, ai-assisted-programming, coding-agents