Billions of parameters. Trillions of tokens of training. Underneath all of it, this is still a model making a prediction — and a prediction was never a certainty. This lesson shows exactly where a confident mistake comes from, names it properly, and gives you something concrete to do about it.
Scale changes how much the model (the board) has read and how finely its floor is painted. It doesn't change what the model fundamentally does: lean toward the likeliest token and pick.
A hundred billion parameters make that lean far sharper and better-informed than our five-by-five board ever could. But "sharper" isn't "guaranteed." Somewhere in every pile, even a confident one, there's still more than one coin.
Go back to Lesson 3's pile: "Cat eats ___." Mouse was the tall stack, the clear favourite. But fish and bird had real coins too — smaller piles, not empty ones.
Most passes, the model takes the tall pile, and you'd never notice the others were there. But "most" isn't "every." Every so often, by the same ordinary randomness that built the pile in the first place, a shorter stack wins instead.
Here's the part that turns one stray pick into a whole wrong sentence. Remember the loop from Lesson 3: whatever gets picked is written down, and the entire line — including that pick — goes back in for the next round.
The model never stops to check whether the last word was the right call. It only ever extends what's already on the page. So if "fish" wins instead of "mouse," the model doesn't notice anything went sideways — it just keeps going, just as confidently, building a perfectly fluent sentence about fish. The confidence never dips. Nothing in the mechanism is built to dip it.
This is what people mean by an AI hallucination: a confident, fluent statement that simply isn't true. A fabricated citation, a wrong date, a quote nobody said.
It isn't lying — lying means knowing the truth and choosing to hide it, and the model doesn't "know" in that sense at all. It's also not quite "imagining," despite the name. It's closer to what we've just watched happen: a pattern got extended, fluently and confidently, and nothing in the mechanism was ever built to stop and ask "is this actually so?" The fluency and the confidence are real. They just aren't evidence of anything.
There's an actual setting behind how often a shorter pile gets to win. Most doors into the model — the API from Lesson 9, and sometimes a chat app's own settings — expose it under the name temperature.
Picture Lesson 2's floor again, red grease fading to blue rubber through every shade of violet. Temperature is a dial on how sharp that colour stays. Turn it toward zero, and the blend tightens back toward pure red or pure blue — barely any violet left. The coin has almost no real choice: it follows the strongest colour every time, and the tallest pile wins almost without exception. Turn temperature up, and the opposite happens — the colours bleed further into violet, the contrast softens, and a path that used to lose narrowly gets a genuine chance. The tallest pile is still tallest. It's just no longer such a sure thing.
That's the trade a higher temperature makes: more variety and less repetition, at the cost of exactly the kind of stray pick the last two sections just described. Low temperature reads as careful and a little repetitive. High temperature reads as more alive — and more likely to wander.
Put the same prompt through the same model twice, and you are not guaranteed to get the same answer back. That's not a glitch — it's a direct consequence of everything this lesson just covered. There's a name for it: non-deterministic.
Ordinary software doesn't work this way. A calculator given "2 + 2" returns "4" every single time, with certainty — that's deterministic. A model given the same prompt twice is, underneath, drawing from the same pile of runners-up each time — and a stray pick, however rare, is a real possibility on every single pour. Turn temperature down toward zero and the answer gets far more consistent, but even then, tiny differences in how the numbers get processed can occasionally still tip a genuine near-tie. No provider guarantees byte-for-byte identical output, every time, no exceptions.
This matters most the moment something depends on it: a report that's supposed to read the same way twice, a process that's supposed to behave identically on a rerun. The answer isn't "something's broken" — it's "design around variation," the same way you'd design around any step that involves a genuine, repeated choice.
One more thing worth knowing now, even without the how yet: at least part of an answer can be pinned down anyway. Lesson 9 gets to exactly that, once you've met the other door into a model.
A stray pick isn't equally likely everywhere. It's far more likely wherever the training left a thinner, less certain map.
A topic barely covered in its reading, a very specific number, a niche fact stated only once or twice across trillions of tokens — the floor there is barely tinted, closer to the blank board from Lesson 2 than the confident heat map. And remember: training is finished and frozen before the model ever reaches you. Ask about anything after that point — its knowledge cutoff — and it isn't being careless either. It genuinely was never shown it.
None of this makes the model useless — it makes it exactly what Lesson 1 already told you: a very well-read colleague's fast, confident first draft.
If a stray pick can tip a whole sentence, then what you type — the words the model sees before it starts leaning — matters enormously.
The next lesson is about exactly that: how to ask well, and how to tilt the board on purpose so the pile lands where you actually want it.
Everything from the lessons before this one, plus what this lesson added.
| The board | The word for it |
|---|---|
| The board and its pegs | The structure — the model |
| A single peg | A neuron |
| A single step from one peg to the next | A connection |
| How red or blue a patch of floor is | A weight — a parameter |
| The whole painted floor | The trained model |
| Boards stacked, one on the next | Layers — "deep" |
| Smearing the floor in, coin by coin | Training |
| Dropping one coin through a finished board | Using it — inference |
| Which slot you pour into | Your input — your prompt |
| The bins, relabelled with tokens | The vocabulary — every token it could pick |
| The tallest pile among the bins | The prediction — the likeliest next token |
| Feeding the growing line back in and running again | The loop — how one guess becomes a sentence |
| One bin's label, precisely | A token — not always a whole word |
| How many bins there are, in total | Vocabulary size — tens of thousands |
| The size of the mountain of text it was shown | Training data — measured in tokens |
| A bin's own address, plotted in space | An embedding — meaning turned into a list of numbers |
| How close two addresses sit | Similarity — how related two things are |
| The measurement's real name, out in the wild | Cosine similarity — same idea as similarity, different name |
| About meaning, not exact spelling | Semantic |
| A neighbourhood of addresses that share a topic | A domain — a cluster of related meaning |
| Scanning every address for whichever one sits closest to a new point | Nearest neighbor — the closest real match to a computed point |
| A small, separate machine that only plots words as points — it never writes a sentence | An embedding model — not the same as a language model |
| The total number of coloured patches, across every board | Parameters — the number people quote |
| How many tokens the funnel can hold in one pour | Input tokens — the context window |
| How many tokens come back out | Output tokens — what it writes back |
| Everything you've met so far, built at real-world size | A Large Language Model — an LLM |
| + New in this lesson | |
| A shorter pile winning, extended forward with full confidence | A hallucination |
| The point where the training reading stopped | The knowledge cutoff |
| How sharp or blended the colour stays before a coin is poured | Temperature — the dial on how often a stray pick wins |
| Pouring the same slot twice, not always the same pile | Non-deterministic — same input, no guaranteed identical output |
30 rows now. It keeps growing as the course goes on.