How Claude Magic Happens

A plain explanation of what actually goes on between the words you type and the words that come back. We follow one tiny example all the way through — "beautiful morning".

Pl. VII — Inside an LLM · seven steps, two bands Sheet 5 of 6 · MMXXVI

Running example↘

"beautiful morning"said to claude · in chat · as a greeting

i.Upstream — the model reads & thinks

where all the actual cognition happens

"beautiful morning"just letters on a screen · 17 characters

step 01 · you type A flat ribbon of letters. The model can't read it yet — it needs numbers. So we have to turn the words into numbers first.

tokenizer ~100,000-piece vocabulary

␣beautifulid #3618 · ␣morningid #2872

step 02 · cut it up The text is chopped into tokens. "beautiful morning" becomes two — one for each word. Common words usually get a whole token to themselves; rarer or longer ones get chopped into smaller pieces. Each piece is just a row number in a giant lookup table the model already knows.

embedding space · 3d projection a map of meaning, thousands of dimensions

drag to rotate

step 03 · look it up Each row number is traded for a vector — thousands of dimensions long — which is just a position on a vast inner map of meaning.

why it's beautiful The model learned, during training, to put related words near each other on this map. Greetings cluster here. Cities there. Royalty over there. Feelings somewhere else entirely.

a famous old result king − man + woman ≈ queen Even directions on the map have meaning. The path from man to woman is the same path from king to queen. This clean version of the math comes from older, smaller models. Today's are fuzzier about it — same picture, less tidy arithmetic.

Greetings Animals Cities Feelings Actions

A word, on the inside, is a place — and meaning is how close it sits to everything else.

transformer blocks · repeated we don't know how many layers — but many

drag to rotate

step 04 · think it over, again and again The two token-positions now flow upward through a tall stack of transformer blocks. Each block does two things, in order — first a looking-around step, then a thinking-in-private step.

attention · the looking-around step Inside this sublayer, many heads run in parallel — each one lets every token look at every token that came before it, picking up a different kind of pattern. The vector for morning notices that beautiful sits in front of it — and updates itself to include that. Tokens can only look backward, never forward — they can't peek at words that haven't been said yet.

feed-forward · the thinking-in-private step Then comes the block's second sublayer, where each token's vector is reshaped on its own — sharpening, abstracting, combining what it just learned from looking around.

it adds on top of, doesn't overwrite Each sublayer's contribution — both attention and feed-forward — is mostly added to the running vector, not swapped in for what was there. The starting place stays largely intact underneath. By the top, the vector is the original meaning plus every block's thought stacked on top.

an honest note The exact number of heads in frontier models is a trade secret. For comparison, open-source models in roughly this size class use somewhere between 32 and 128 heads per layer, stacked across dozens of layers. Frontier models are presumed bigger.

A final, fully-thought-through vectorthousands of dimensions — but now soaked in context

step 05 · the answer takes shape What comes out the top is the model's compressed guess at what should happen next. Not yet a word — a position in the meaning-map that says "something like this".

ii.Downstream — the model picks a word

where the inner thought becomes one specific token

odds for every possible next token · top 15 of ~100,000 P(next word)

temperature 1.00

balanced — drag to make the model more predictable or more chaotic

step 06 · score every word The final vector is compared against every word in the 100,000-word vocabulary. Each gets a score. Most are nearly zero. A handful really stand out.

why these words? Because you said "beautiful morning" to me, in chat, as a greeting — the top candidates are greeting-shaped: "To" (as in to you too), "Good", "Hey", "Beautiful". If the same two words appeared in a novel, the top candidates would be totally different — "," or "for".

temperature · the chaos dial Try the slider above. At low temperature the model is almost deterministic — it picks the obvious winner every time. At high temperature the distribution flattens out and unlikely words become real contenders. Think of it as how much randomness you allow in the dice roll.

top-p · the long-tail cutoff A second knob. The model sorts the candidates by probability, then keeps only the smallest group that together adds up to p of the total — say 90%. Everything past that cutoff is discarded before the roll. Stops the model from very-occasionally picking something absurd.

→ a weighted die is rolled · ␣To wins this round

step 07 · roll the dice One word is sampled. The model doesn't deliberate — it picks weighted-randomly from the contenders. Today's lucky winner: " To".

↻

Then " To" gets glued onto the end of the input, and the whole machine runs again to pick the next word. And again. And again.

One token at a time, the model writes back: " To" · " you" · " too" · "!" · " ☀" — and then stops. The whole loop runs again for every single word, though the model quietly reuses most of the thinking it already did on the earlier words.

autoregressive · token by token