Language, Meaning & The End Of AI

I know. It's not like the world needs another post about AI. But let me see if I can make this a bit different because I am finding the endless parade of drooling investors, big consultancies repositioning themselves as AI thought leaders, predictions of robot uprisings and job losses all a bit exhausting.

Most days I feel like I'm at a mildly shit party and everyone's dropped an E except me.

Let's start with language.

Language

In the fruit and vegetable section of the supermarket, there are two types of product: organic and, for want of a better word, normal.

Before 1940, there was no such thing as organic food. There was only food. Because what we call organic food was normal food. And then came advances in fertilisers, automation and pest control. These innovations massively improved yields, made farms more productive, and made food cheaper. This is not a rant about modern farming, just an observation that there was food, and then there came "new food." And yet, new-food became food, and old-food (food food) became organic.

For very understandable marketing reasons, new food was never called Chemifood or Farmaceuticals or Fertibites. But new food came after food, so logically it was the one that needed a name. Because food already had a name: food. But the industry wanted everyone to buy new-food, so they called it food. And old food had to be rebranded.

Because language matters.

Linguistic Relativity and ideas like George Lakoff's Framing Theory tell us that words hugely influence how we think and feel about things. Tell us that food is just food, and there's this other organic stuff that's more expensive and probably for Guardian readers, and we'll buy new food.

Tell us that this piece of software is intelligent, and we'll believe it is. We might even believe that Artificial Intelligence is a step on the road to Real Intelligence. I mean, why not? ChatGPT sure feels different to anything that's come before. Yes, it gets things a bit wrong sometimes, but it's early days, right? Tech products always improve over time. Better invest in them now and reap those benefits when those mistakes are ironed out.

Numbers vary, but something like a trillion dollars of investment has gone into businesses developing and selling "AI" software products and services. That is a lot of believing. And it's all good if you believe organisations like PWC, who say that AI will contribute nearly 16 trillion dollars back to the economy.

Still. A trillion dollars. That's a big bet.

What if we took the language away for a second and described these products based on how they actually work?

Would we be quite so concerned about the future if, as Emily Bender and Alex Hanna) in their book "The AI Con" suggest, we called tools like ChatGPT "synthetic text-extruding machines"?

Would you prefer GenAI or a helping of synthetic text extruder with your cornflakes?

When John McCarthy coined the term "Artificial Intelligence" in 1955, as part of the drafting of a paper for the famous Dartmouth Summer Research Project in 1956, he was deliberately setting up a concept that partly encapsulated existing theories, but also set out to be something new and bold and ambitious - he was indulging in some lightweight marketing.

The field was fairly new at the time, but 70 years later, things are more mature and generally, the models that make up the very loose collection of things we call "AI" have common characteristics:

• They take in data and "learn" something about it by adjusting internal settings (often called parameters) based on some form of feedback.

• At the level of each data point, the learning step is usually a small adjustment to these parameters based on how right or wrong the model was.

• The mechanism by which they do this varies, but at the data item level, it's incredibly simple. Scaled up, though, all these parameters start to do impressive things.

• They optimise this process towards some kind of goal (where changes to the parameters move towards or away from the goal).

• These parameters effectively become an internal representation of the real world.

• In fact, when people talk about “models”, they usually just mean the final set of parameter values, the result of the training, which can now make predictions or decisions about the goal, based on new, never-before-seen data.

• Changes to the model (parameters) have to walk a fine line between being making it so tightly tuned to the goal that it fails when the situation changes slightly or being so vague that it's not useful for anything specific.

• All this happens over and over again, with lots of tweaks, until the model appears to be the best it can be.

If this all sounds a bit fuzzy, that's because it is. It's quite far removed from writing code that calculates your tax bill. Models operate with a level of uncertainty and approximation. Even when they’re not explicitly probabilistic, models can still be very confidently wrong. And unlike traditional code — where a bug can be fixed — when a model is wrong, it often has to be retrained, not rewritten.

What I am saying is that there is no "AI" and there never was. There exists a collection of approaches that have been developing since the 1950s that can, in certain use cases, be extremely useful. And recently, some new approaches that are part of the same historical timeline that happen to have some quite surprising properties.

We've been using these approaches for decades. Amazon launched "Customers Who Bought This Also Bought" in 1999. We've also had spam filters, face recognition, search ranking, fraud detection, movie and music recommendations, predictive text, and personalised ads.

I've worked on seven projects in the last 20 years where some form of machine learning was core to our solution. They're not magic. They're not general-purpose tools like coding languages are. They do not come with intelligence, reasoning or agency. Not even Large Language Models, despite their impressive simulation of cognition and meaning.

Meaning

If I say the words "the cat sat on the -" out loud, your brain can barely resist the urge to reply with the word "mat", and yet thousands of other words would be just as logical. Why? Why not "laundry basket", "tractor", or "marine biologist"? Maybe this particular cat sat on a recumbent bicycle.

Human brains use predictive processing to anticipate what comes next based on patterns, probabilities and prior knowledge. You can think of it spatially as the word "mat" existing in close proximity to the phrase "the cat sat on the -" and it follows that "mat" is also readily available near words like "rug", "door", "floor", "welcome", "yoga", "bath", and so on.

Your brain lands on "mat" in this case based on the context provided by the 5 previous words (although I think most people would start feeling quite matty after 3 words).

But with more context: "Bob lent his recumbent bicycle against the radiator as the cat walked in. The cat sat on the -" I'd suggest you might not be feeling such a matty vibe.

What if you could look back and somehow hold in your head the previous 100,000 or even 200,000 words? Your predictive processing would be supercharged. What if you could also model those semantically nearby words in 16,000 dimensions, and what if all this was based on you having digested 500 billion words and word fragments?

Well, you'd be a large language model, able to take in a stream of words and interpolate what probably comes next. Mats or baskets or tractors, or bicycles.

Probably. But never precisely.

You would be a very, very impressive synthetic text-extruder, able to execute high-dimensional interpolation over massive data sets with ease. Go you.

You'd be pretty bloody amazing at guessing words that follow words. And those words would have a ring of intelligence to them because those numbers are huge. You're basing this skill on modelling four times as many words in your head as the number of humans who have ever lived. So great is this simulation of understanding that the words that follow question words would look like the answers to those words when they are actually just words after words after words. Not meaning. Just words that follow words.

Does a fancy, massive probabilistic word map count as intelligence? Or consciousness?

I'm not even sure that's a valid question. Whatever human intelligence and consciousness are, they are aggregates of multiple factors, and a fancy, massive probabilistic word map could only ever be but one of them.

It's quite possible that semantic maps in the brain have similarities to the internals of large language models. I have no idea. Though it's interesting that humans project intelligence into data sets rather than reverse the comparison and diminish our brains to a big fat spreadsheet.

You don't need to know how ChatGPT works to use it. In the same way, you don't need to know how a petrol engine works to drive a car. Unless someone is telling you petrol engines will fly you to the moon. Then an appreciation of their limits and capabilities might be useful. You need to know not to believe people who talk about moon trips if they don't know about petrol engines. And you need to know that when people who sell moon trips are talking, they want to make money more than they want to take you to the moon.

Doesn't make petrol engines bad; they just aren't rockets.

Synthetic text extruders are great. But unfortunately, moon trip salesmen currently have all the airtime. They need us to either be in awe of moon trips (so we revere the moon-trip masters) or in fear of moon travel (so we don't ask too many questions). But if we utilise our newfound language immunity and understanding of synthetic text-extruders, we can make some predictions.

No Jobs Will Be Lost

AI tools and synthetic text-extruders are a replacement for nobody.

Who shall we do first? Lawyers? Lawyers cost a fortune. LLMs can slurp up all that law, analyse legal documents and replace lawyers.

Developers? Same with code. Slurpy slurp, out comes code, goodbye all you miserable bastards who've held enterprises to ransom for decades.

Customer support? Journalists? Pharmacists? Slurp, replace, goodbye.

Never. Gonna. Happen.

Probably, but never precisely.

When the legal advice is wrong, or the code doesn't work, or the chatbot lies, or the wrong medicine is prescribed, what do you do? You could retain a few expensive staff and automate the rest, except you can't because you don't know which output had the 20-70% incorrectly guessed words in. If you want quality, then all of it needs to be checked.

It's just words that follow words like "mat" following "cat" or "Cubano", "Reuben" and "Bánh Mì" following "what's the best sandwich in the world?"

As a tool to augment lawyers, developers, etc, who already know what they're doing, then yes. Figures for productivity gains vary, but they can be significant. I can personally attest to this. But like any tool, it requires a bit of effort to get the most out of it.

If your job happens to be extruding text without expertise, taking meeting notes without expertise, or creating art without inspiration or just making facts up, then yes, I concede, you're doomed. But console yourself with the thought that you were lucky to have such a job for so long.

(I'll deal with the "but they'll get better over time" argument in a bit because that's not happening either)

Zero Value to the Consumer

When I am paying my augmented lawyer or software engineer, I am paying them for a service that, as a consumer, I am happy to assign a value to.

If the lawyer or developer chooses to use a synthetic text-extruder to speed up their work, then great. I would expect them to use it and moderate its output in support of providing my service. It remains to be seen if I'll get the service at a reduced price. Probably not, because the skill I assigned that value to hasn't actually diminished. Typing the prompt and copy-pasting the output has zero value to the consumer without expertise.

And that is worth sitting with for a moment.

If you send me 60 pages on the rise and fall of the Roman Empire, and it comes from Gemini. It has no value whatsoever. Not unless you're an expert on the Roman Empire. All you did was paste an extruded small subset of the fancy massive probabilistic word map.

I don't hear enough talk about this. If all you're doing is using an LLM, you are not doing anything. You might as well not be there.

Oh, but Prompt Engineering? OK, but that's what I pay the expert for: to know what to ask the model. If I can type the prompt without the expertise, then anyone else doing the same is adding no value to me.

The value is in the augmentation of the expert. And that value will soon be assigned a number by market forces. What will the lawyer pay for Law-o-matic? What will I pay for my Cursor license?

More than zero, yes. But I can't help thinking about that trillion dollars.

The Utility Value Has A Natural Limit

A trillion dollars is a lot. Most people today, if they do pay for a GenAI tool, pay about $20. Would they pay more? Maybe. Enough to pay back all the investment plus returns? Obviously, I don't know, but we are about to find out. It's not looking good.

OK, fine. Some investors will lose their money. It's a market after all. But look at OpenAI, arguably the poster child, or the Pets.com, of the AI bubble/movement.

OpenAI's last funding round, for $40bn, valued it at 75x its 2024 revenue. That is some multiple for a text extruder that currently loses money on every user. And it needs about that amount of money every year to survive.

OK, you might say that all this AI business is a new paradigm and looking at the bottom line wasn't relevant for Amazon in the dotcom boom, so it's not relevant for OpenAI now.

Except that Amazon had a product. It worked the same way then as it does now. All Amazon had to do was scale. You could believe their plans, or not, but it came down to that. Scale and succeed or flounder and die. It wasn't like they had an e-commerce store that delivered 20% of the orders to the wrong address whilst claiming people would one day stop going to the shops.

There is a problem with the maths here. People will pay for text-extruders, but they will pay according to the value they add.

Yet the hype around AI has led to vast investments in a product that doesn't do what the techbros say it does. In fact, it's worse. If you look at most of the claims, they are all in the future. A more accurate statement is that the hype around AI has led to vast investment in a product that doesn't exist. They are promising trips to the moon on a petrol combustion engine.

Because there's no causal link between synthetic text extruders and real machine intelligence. I have no idea if AGI is possible (actually, I have thoughts, but that's for another time), but it's not going to arise from a fancy, massive probabilistic word map.

Google & Microsoft can shrug off another AI winter, but not OpenAI. Companies like OpenAI are in big trouble over the next year or so.

But really, it's the leaders of the biggest tech and consultancy firms who need to be held accountable for this. To make these overblown predictions about models thinking or reasoning based on how these tools really work can only be put down to being unbelievably thick or lying. Either should disqualify anyone from being a leader.

I'm not saying this stuff is easy. Large machine learning models are complex entities that are difficult to reason about or predict behaviour for. You can't look at an LLM and say what it will do. And yet what they can do, in their restricted use case, sensibly valued way, has real utility. It's just being drowned out by a sea of dicks.

In Their Arising is Their Cessation

Despite all this, I am an optimist. I've seen tech innovations come and go, with each wave leaving behind something useful, even as the over-investment causes short-term pain for the ones who took it too literally. This one may leave behind more than usual, but the tide is about to turn. It always does.

AI is going to die. Like Agile, it's being adopted without understanding, which means it'll disappoint, fail to meet expectations, become a dirty word, and everyone will quietly remove AI from their marketing and pretend they always knew. Some tools will survive, but teams will have to fight a bit harder to use models even where they're appropriate.

In the meantime, we just have to try and see the truth through the noise. Believe only what we know and try to navigate the depressing gulf between the tiresome bloviating billionaires and grounded reality.

Buddhism has a word for this: dukkha. Dukkha is often coarsely translated as "suffering", but it's more nuanced than that. Dukkha is the impermanence of things, the rise and fall and ebb and flow that leads to unhappiness or dissatisfaction because there's nothing permanent and solid or eternal to hold on to. I'm afraid I can't promise the path to nirvana in a short post, but one quite profound idea central to Buddhism is that if you can truly understand how a thing arises, you can also understand how it ceases. The seeds of the end of a thing are in its creation.

The end of AI as we know it is staring us in the face. It's already in trouble. Current approaches have plenty of known limitations, and we know that generative AI has already begun to destroy the web and it's terrible for SEO (which is obvious: if everyone generates slop, then how do you rank sites?) and recruitment and customers want humans back. In fact, people hate the idea of AI even when it's not AI.

It's pretty good at code, where synthetic text extrusion is governed by limited grammar and the ability to automatically test its output. But for content, even when it's correct, it is at best a sloppy recreation of the reversion-to-the-mean, mundane content of the internet it was trained on.

So its only hope is to get better, which means more data, ideally better data and more pruning of models to improve results.

As it arose, so it dies. Because there isn't really any more data. Certainly not enough to have an impact. Even if there was, it would be prohibitively expensive. Some sort of breakthrough would need to happen now with current models to point to being a wise investment.

This is why the idea of training models on synthetic data became topical, turning synthetic text-extruders into synthetic text ingesting synthetic text-extruders like some freakish ouroboros.

Clearly, this cannot end well. The models collapse as they eat themselves, or they reinforce their already distorted view of the world (not to mention they'd learn nothing new about sandwiches), or they stay the same, and the bubble, the market, possibly a few economies, go pop.

Ultimately, this may be a good thing. The market response may be so extreme that it brings the whole platform enshitification malaise of the web to a halt.

The world of tech has elevated itself onto a golden pedestal of its own making. Most of us still want to build products that are subservient to the human experience. I want the early web back. A web where you could find things. A web where we cared about experience. The sooner we get back to talking about that, the better.