Hold me back
It seems that periodically we get a story about how AI vendors are fearful about the awesome, apocalyptic power of their tech, and for God’s sake, won’t someone stop them. There’s been a recent round of this from Anthropic. I’ve wondered what the intent behind these stories was, since caring about harm or morality has not hitherto been a strong suit for the AI companies.
I’m probably just slow, but it wasn’t until I read Cory Doctorow’s insightful and persuasive Reverse Centaur, that I got it (seriously, everyone should read this book). Doctorow makes the point that first of all, LLMs are simply not capable of the type of intelligence portrayed in these scenarios. The fundamental model just won’t do that. But more importantly, he stresses that these stories are not aimed at us, they are aimed at investors.
Once you see it through that lens it becomes obvious what the purpose of this recurring narrative is. These companies need to keep growing, as soon as growth stops, their stock price crashes. The AI bubble is slowing as people realise it isn’t that brilliant. The weight of all those case studies of companies that sacked their human employees and replaced them with AI, are now rehiring the humans is gathering. So, a story about how completely amazing your technology is going to be, to the extent that it could wipe out humanity, is actually a good news story in the upside down world of tech investment. It does two things. First, it diverts attention away from limitations and stresses the almost mythical power of the tech. It’s definitely worth investing in God like power, right? Second, it detracts from the actual harm and damage being done at more prosaic levels to people’s working conditions, our political sphere, the social contract, and environment.
Having guidelines to stop super AI killing us all is a bit like me teaching my dog the Highway Code for when he starts to drive. It won’t happen and it’s useless. Having legislation in place to hold AI companies accountable for harm to wages, the environment and the information ecosystem is like me teaching my dog not to poop in the house. It’s useful and essential for harmonious living, but not very glamorous.
So all of this posturing by AI companies begins to look like the “Hold me back” bloke who’s always getting his mates to restrain him in a fight, and then saying just as well they did, because he would have done real damage. everyone knows that a) he had no intention of fighting and b) would have been ineffective if he did. It’s all show.

2 Comments
Lorna Campbell
100% yes to all of the above. Timnit Gebru has also written extensively about how all this performative catastrophising is purposefully designed to distract from ongoing real world harms https://www.wired.com/story/one-of-ais-fiercest-critics-says-all-the-doom-talk-is-meant-to-distract-us/
mweller
Hi Lorna, thanks for that link, he puts it much better than I. It’s the old magicians trick, of look over here and not what this hand is doing isn’t it? I mean, I think we should probably stop trying to build AI super intelligence just in case, but it’s not the main story here