AI

Slop Will Eat Itself (then us)

Many of you will have seen the story about Anthropic (and other AI companies) buying books, and destroying them in the process of digitising. The reasons for destroying as opposed to just scanning them, seems to be a mixture of cost and avoidance of copyright issues. The need for this large scale process is because, as was pointed out some time ago, LLMs will suffer from model collapse, that is they start to train themselves on their own content. This undermines the argument often put forward by AI enthusiasts that it is only going to get better. There is a real possibility that it is going to get worse as recurring cycles of training feeds on its own content, with the result that misinformation and hallucinations gets perpetuated but also that the model has less diverse material to draw upon.

Given that AI providers are in fierce competition with each other to have better models than their rivals, finding ways of avoiding model collapse is imperative for them. Harvesting content pre-2020 is one way, but that will run out. The insidious interpretation of the book destruction approach is that it is an attempt to deliberately remove content from the public domain that your competitors might use.

This is now an active business model. Maren has a podcast and has been approached by a company to have exclusive access to her raw recordings (she declined, obviously). They want to train conversational models on real humans speaking, not just the published version. But they don’t want competitors to have that content. Once you start seeing _all_ human interaction as potentially exclusive content for harvesting then it lends itself to some “interesting” business models (and by interesting, I mean awful). Let us consider a near future (it is probably already happening), where the requirement for content that is not infected by AI is increasingly precious. It then makes capitalist sense for people to wear smart glasses or other devices, and walk around surreptitiously recording conversations, to be sold to AI vendors. One can picture them like traders of old, arriving with their cartload of goods and it being sold by weight, in this case gigabytes of data.

Or let’s consider another area – that of creative writing. The self-publishing community is substantial (I’m a proud member). There are a lot of online tools that provide platforms for writing, with formatting, analytics, and advice. I use Reedsy when my novel is complete, for instance. This is original content being created on a platform that can be harvested, and all the changes, modifications, and tweaks themselves are useful data for AI models (I’m not suggesting any of the platforms are doing that currently). We might also see providers offer incentives to allow exclusivity on their platform so that only their models are trained on that content. Amazon requires a 90 day exclusivity for Kindle Unlimited promotion currently. This is probably more about market dominance at the moment rather than AI training, but it could easily lend itself to the latter.

But this pales into insignificance next to the great content production engine that is Higher Education. All those assignments, forums, artefacts. And all the variations of them, along with revisions and edits, and all those detailed conversations in online forums. This is rich and largely endless, human generated content for AI models to train on. With nearly all HEIs using TurnItIn or similar, they are already handing over that data. At the moment I don’t think those companies are selling it on, but one could imagine a business model where they might (and perhaps offer the HEI a discount in return).

I’m sure someone has already written it, but I can foresee a future with people in large warehouses, sitting and generating content for the big machine to suck up, a sort of Matrix-like feeding, but with bad poetry and essays as the food rather than electrochemical energy.

One of the ironies of this is that AI is acknowledging itself as an infection, or invasive species. It doesn’t want content that’s been affected by AI. Maybe AI companies will start helping students to avoid using AI in assignments so it can be sure to harvest their content afterwards. It also recognises that human generated content really is worthwhile, despite all the promises of the great AI reckoning. Odd that.

PS – I thought I had been all clever with the title as a play on the band Pop Will Eat Itself (I nearly got beaten up at one of their gigs in the 80s, but that’s a different story). It turns out that it’s a commonly used phrase on social media. Oh well.

Leave a Reply

Your email address will not be published. Required fields are marked *

css.php