By the end of this lesson you will be able to
- Explain what a token is and roughly how many are in a page of text
- Predict why some text costs more to process than text of the same length
- Diagnose "the AI forgot" as a context-window problem rather than a bug
Two numbers govern almost every practical limit you will hit with AI. Neither is complicated, and both are usually explained badly. This lesson fixes that.
What a token is
AI does not read letters, and it does not quite read words. It reads tokens — chunks of text that are usually a whole common word, sometimes a fragment of a rare one, and always separate for punctuation. "The" is one token. "Antidisestablishmentarianism" is six or seven.
Definition Token- The unit of text an AI model processes and bills for. Roughly three-quarters of an English word on average — so 1,000 words is about 1,300 tokens. Full definition
That variability is why two documents of the same word count can cost noticeably different amounts to process. A page of plain prose is cheap. A page of product codes, names, and tables is not.
| Document | Approx. words | Approx. tokens |
|---|---|---|
| A short email | 150 | 200 |
| A one-page quote | 400 | 550 |
| A service contract | 3,000 | 4,000 |
| A 30-page policy manual | 12,000 | 16,000 |
| A year of meeting notes | 150,000 | 200,000 |
The context window
The context window is how many tokens the model can hold in view at one time. It covers everything: your instructions, the documents you pasted, every message so far, and the reply being written. It is not storage. It is more like a desk — when the desk fills, something slides off the edge.
Working with the limits instead of against them
Front-load what must not be forgotten
Constraints, tone, format, and non-negotiables go in your first message and get restated if the conversation runs long. Do not bury a critical requirement in message eleven.
Start fresh more often than feels natural
A new conversation with a two-line summary usually beats a forty-message thread. Long threads carry a lot of irrelevant history, which costs tokens and dilutes attention.
Paste the relevant section, not the whole document
A targeted extract outperforms a whole manual. It costs less, and it leaves less room for the model to latch onto something irrelevant.
Reach for retrieval when documents get big
Once the material genuinely exceeds the window, the answer is a retrieval system that fetches the relevant passages on demand rather than a bigger paste.
Most "the AI is bad at this" problems are really "the AI was not given the right things to look at" problems.
If a model has a huge context window, can I just paste everything?
You can, and it often gets worse. Large contexts cost more, run slower, and give the model more chances to fixate on an irrelevant passage. Precision beats volume — paste the three relevant pages, not the sixty-page manual.
Do tokens work the same for every model?
The concept is universal; the exact split differs between models, so the same text can be a slightly different token count on each. The approximation of three-quarters of a word is close enough for planning.
Does the AI remember between conversations?
By default, no — a new conversation starts empty. Some products add a memory feature that stores facts separately and re-inserts them, which is a layer built on top of the context window rather than a change to how it works.
Key takeaways
- Tokens are chunks of text — about three-quarters of a word each on average.
- Unusual words, names, and codes fragment into more tokens, which is why they cost more.
- The context window holds everything at once: instructions, documents, and the whole conversation.
- "It forgot" means the window overflowed. Restate constraints or start fresh.
- Give the model the relevant extract, not everything you own.