How AI Actually Works

Almost everything confusing about AI stops being confusing once you know what it is doing underneath. Not the mathematics — you will never need that. Just the shape of the thing: what it takes in, what it produces, and what it is optimising for. That single piece of understanding is what separates people who use AI well from people who are alternately amazed and betrayed by it.

This page is the whole picture in one place. It links out to the detailed treatment of each part, but you can read it start to finish and come away able to explain AI to somebody else, which is a decent test of whether an explanation worked.

A large language model does one thing: given some text, it predicts what should come next. Then it does it again, with the text it just produced now part of the input. Word by word — fragment by fragment, really — an answer assembles itself out of several hundred repetitions of that single operation.

That sounds far too simple to produce what it produces, and the surprise is the entire story of the last few years. It turns out that predicting text well, across a large enough body of text, requires learning an enormous amount about how the world is described. To predict the end of "the contract was void because the party had not reached the age of", a system has to have absorbed something about contracts. The capability is a side effect of the prediction getting very good.

The model does not see characters or words. It sees tokens: chunks of text produced by a fixed vocabulary built during training. Common words get a slot of their own, so "the" is one token. Anything unusual gets assembled from pieces, which is why a surname or a part number can cost four or five tokens where a common verb costs one.

Tokens matter commercially for two reasons. They are the billing unit, so the shape of your text determines your bill — and text dense with names, code, or identifiers costs meaningfully more than ordinary prose of the same length. And they are the unit of the memory limit, which is the next thing that trips people up.

This is the most counterintuitive part, and it explains a great deal of otherwise baffling behaviour. The model does not remember your previous message. It does not remember anything. Each call is independent, and the only reason a conversation feels continuous is that the entire history is re-sent every single time you press enter.

That history has to fit in the context window: the total working space for one call. Everything shares it — your document, the standing instructions the operator wrote, the conversation so far, the retrieved reference material, and the space needed for the answer itself. When the conversation grows past it, the earliest parts fall out, and the model loses them completely rather than gracefully.

Training happens once, on a fixed snapshot of text, and then stops. Everything the model appears to know came from that snapshot. This has three consequences that matter more than any benchmark.

Since the model cannot know your documents and cannot hold everything at once, the standard solution is to look things up at question time. Index your material, find the passages relevant to the question being asked, put those passages into the prompt, and let the model answer from them. That is retrieval-augmented generation, and it is the architecture behind essentially every "AI that knows your business" product.

The important property is that correcting an answer means editing a document rather than retraining anything. That is what makes it practical for a business whose facts change. It is not a guarantee of correctness — retrieval can fetch the wrong passage, and the model can still summarise a right passage badly — but it largely removes invention, which is the failure that damages trust fastest.

Every characteristic failure follows from the mechanism, which is why understanding the mechanism is worth twenty tips about prompting.