Glossary — what each word means
Plain-English definitions. (The math derivations live in the “Show the math” panel on each slide above.)
the data
Token — the smallest unit the model reads/writes. Here one digit; in a chatbot, a chunk of a word.
Vector / embedding — a digit’s “name tag”: a private list of 48 numbers that describes that one digit. Each digit 0–9 has its own.
Weight / parameter — one adjustable number (knob) inside the model. Learning = finding good values for all ~80,000.
the network
Dot product — multiply two lists slot-by-slot and add; one number for how well they match.
Attention — the model looking at all the numbers at once and focusing on the ones it needs right now (built on dot products + softmax).
Softmax — turns scores into percentages that add to 1 (raise e to each, then divide by the total).
LayerNorm — rescales a row of numbers to a steady size so training stays smooth.
ReLU — keep positives, turn negatives into 0. Adds the bend that lets it learn curves.
Residual — add a layer’s input back onto its output, so nothing is lost.
learning
Loss / error — one number for how wrong the model is. Smaller is better.
Gradient — how fast the error changes when you nudge a weight (change-in-error ÷ change-in-weight) — a slope telling you which way is downhill.
Self-supervised — the training data is its own answer: the target at each spot is just the actual next token in the sequence. No human-written answer keys.
Backprop — the efficient method that computes every weight’s gradient in one backward sweep (the chain rule).
Learning rate — how big a step to take downhill each update.