Learn AI by playing with it
Five core ideas behind modern AI, each as a hands-on experiment. No paper reading, no math background needed — just curiosity.
How does AI understand which words belong together?
When you read “the dog chased its tail”, you instantly know “its” means the dog’s. AI learns the same trick: every word looks at every other word and decides who matters. That one idea powers ChatGPT.
From: Attention Is All You Need (2017)Try this
- Hover “it” in the animal sentence — watch it find “animal”
- Switch to 1 head, then 8 — see how more heads notice more things
- Type your own sentence and explore it
Self-Attention Lab
Every word attends to every other word — see it happen
👆 touch a word to see its attention
Note: these attention patterns are an illustration modeled on head behaviors observed in trained Transformers (previous-word heads, coreference heads, …) — not the output of a live model. The equations and mechanism shown are exactly those of the paper.
Hover or tap any word. The lines show where that word is "looking" — this is attention, the paper's big idea.
Analogy: A classroom where every student can listen to every other student at the same time — and each decides who is worth listening to.
How does a computer actually see a picture?
A computer sees only a grid of numbers. To find shapes, it slides a tiny stencil (a “filter”) across the image, asking at every spot: does this patch match my pattern? Stack thousands of these and you get computer vision.
Try this
- Press play and watch the filter scan the digit 7
- Switch to “Outline” — only the edges survive
- Set stride to 2 — the output image shrinks. Why?
CNN Convolution Lab
How computers see: a tiny magnifying glass scanning a picture
Input
Feature map (10×10)
Compares each pixel's left side to its right side. Big difference = a vertical edge lives here. Press play to watch the filter scan the image — this exact multiply-and-add is what every CNN layer does, millions of times.
Analogy: Sliding a small stencil over a photo and asking at every spot: how much does this patch look like my pattern?
How do self-driving cars spot people in real time?
YOLO looks at an image once and shouts out every object it might see, each with a confidence score. Then two filters clean up: ignore weak guesses, and merge duplicate boxes pointing at the same object.
From: You Only Look Once (2015)Try this
- Drop confidence to 20% — a cat appears that isn’t there!
- Raise confidence to 70% — the dog disappears
- Push IoU to 0.9 — the car gets detected three times
YOLO Detection Lab
One glance, every object — then keep only the best guesses
Note: the scene and candidate boxes are illustrative, but the filtering is YOLO's real post-processing math — confidence thresholding and Non-Maximum Suppression with true Intersection-over-Union geometry.
Clean result: 3 objects. NMS quietly removed 3 duplicate boxes — for each object, only the most confident box survives.
Analogy: The model shouts out everything it thinks it sees, each with a confidence score. Your sliders decide which shouts to trust and which duplicate shouts to merge.
How does a neural network actually learn?
Learning = rolling a ball downhill in the fog. The ball is the model, the hill height is “how wrong am I?”, and every step moves toward less wrong. The learning rate — how big each hop is — makes or breaks everything.
Try this
- Get the ball to the flag with the default settings
- Set learning rate to 1.0 — watch training explode
- Start at −4.5 with momentum 0 — stuck in the trap! Now add momentum 0.8
Gradient Descent Lab
How neural networks actually learn
Press play to drop the ball! It always rolls downhill toward lower loss — that is how a neural network learns. Can you get it to the flag?
Analogy: A ball rolling down a foggy mountain: it can only feel the slope right under its feet, and the learning rate is the size of each hop.
How does the network know which connection to fix?
Watch an actual neural network learn, live in your browser. It guesses, measures its error, then sends the blame backward through every connection — each weight shifts a tiny real step. This lab is 100% real: real network, real gradients, real learning.
Try this
- Press play on XOR — watch the background snap into a checkerboard
- Click the map to quiz the trained network on any point
- Try the spiral with 3 neurons (it fails), then 8 (it works)
Backpropagation Lab
A real neural network, training live in your browser
What the network believes
Click anywhere on the map to quiz the network
Inside the network
Everything in this lab is real: a genuine neural network (2–4–4–1, 37 parameters) runs and trains in your browser with true backpropagation. Line thickness = actual weight values; neuron colors = actual activations; the background map = the network's live predictions.
Untrained: the background is the network's current guess — random nonsense. Press play to train it with real backpropagation on the 90 dots.
Analogy: Teaching by correcting mistakes: the network guesses, sees how wrong it was, and passes the blame backward so every connection knows how to improve a little.
Why did deeper networks get WORSE — and what fixed it?
In 2015, adding layers made networks dumber — a 56-layer net lost to a 20-layer one. ResNet fixed it with one addition: let every layer add its input back to its output. Watch two real networks of identical depth race — the only difference is that one wire.
From: Deep Residual Learning (2015)Try this
- Press play on the spiral — the plain net flatlines at ~50% while ResNet hits 100%
- Check the gradient bars: the skips carry ~40× more signal to layer 1
- Drop depth to 8 — shallow enough that even the plain net survives
ResNet Lab: the skip-connection race
Two real deep networks train live — one plain, one residual
Plain · 12 layers
loss — · acc —
ResNet · 12 layers + skips
loss — · acc —
The race — loss over time
100% real: both networks genuinely train in your browser with exact backpropagation (gradients verified against finite differences to ~1e-10). The residual network computes literally y = F(x) + x, the equation from the paper. Same initialization luck, same data, same learning rate — the skips are the only difference.
Two identical 12-layer networks, same data, same starting luck. The ONLY difference: the teal one adds its input back to every layer's output — y = F(x) + x, the paper's one-line idea. Press play and watch them race.
Analogy: Passing a message through 12 rooms of people. Plain network: each room must re-tell everything from scratch. ResNet: each room gets the original note and just adds a correction to it.
How does ChatGPT choose every single word?
A language model never “knows” the next word — it holds a raffle. Every possible word gets tickets based on how likely it is, and one is drawn. Temperature controls how fair the raffle is.
Related: Attention Is All You Need (2017)Try this
- Set temperature to 0.1 — you almost always get “Paris”
- Set it to 2.0 and sample five times — chaos!
- Set Top-K to 1 — now there’s no raffle at all
Next-Token Playground
How GPT-style models choose every single word
Next-token probabilities
Note: the candidate tokens and their starting scores are curated examples, but the temperature, top-k, top-p, softmax, and sampling math applied to them is exactly the algorithm production LLMs use to pick every word.
Temperature 0.80 keeps a balanced distribution — mostly confident, with some variety.
Analogy: The model builds a weighted raffle for the next word — temperature decides how fair the raffle is, Top-K and Top-P decide who's even allowed to enter.
Can you watch a neural network think?
The grand finale, honoring 3Blue1Brown's famous video: a real 100→16→16→10 network trains in your browser, then you draw digits and literally watch the neurons light up as it reads your handwriting. Hover any neuron to see the stencil it learned.
Try this
- Press play and watch accuracy climb from 10% (guessing) to ~100%
- Draw a 7 — then slowly erase its top bar and watch the prediction waver
- Hover layer-1 neurons: their stencils are real learned stroke detectors
“But what is a neural network?” Lab
Draw a digit, watch 1,860 real weights read it — an homage to 3Blue1Brown
Draw here
10×10 pixels = the network's 100 input neurons
100 → 16 → 16 → 10
The network says
·
hover any neuron to see inside it
100% real, in homage to 3Blue1Brown's “But what is a neural network?”: this is a genuine 100→16→16→10 network (the video's 16-16 hidden architecture on a 10×10 canvas) trained in your browser with real mini-batch gradient descent. Neuron glow = actual activations from your ink; edge colors = actual signed weights; the hover stencils are the network's true learned first-layer weights.
This 100→16→16→10 network is untrained — draw something and every digit gets ~10%, a pure guess. Press play to train it live on 400 noisy digits.
Analogy: Each neuron is a tiny inspector holding a stencil. Layer 1 inspectors look for strokes and edges; later layers combine their reports: “loop on top + line below… that's a 9.”
Can two AIs teach each other by fighting?
A forger network invents data; a detective network calls real-or-fake; each one's mistakes train the other. Both networks are real and train live — including the famous failure, mode collapse, which you can trigger on purpose.
From: Generative Adversarial Networks (2014)Try this
- Press play on 'two hills' — watch the teal fakes fill the amber outline
- Watch the white detective curve flatten to 0.5 (its surrender)
- Drop the detective's learning rate to 0.03 — real mode collapse!
GAN Lab: forger vs detective
Two real networks in a live adversarial duel
100% real: both networks train live with true backpropagation — generator gradients genuinely flow through the discriminator. The instabilities you can trigger (oscillation, mode collapse) are the same ones that made GANs famously hard to train.
Two networks, one duel: the forger (teal) turns random noise into samples; the detective (white curve) rates every point — 1 means 'looks real'. Press play to start the arms race.
Analogy: A counterfeiter and an art detective training each other: every caught fake teaches the counterfeiter, every convincing fake sharpens the detective — until the fakes are indistinguishable.
How can fine-tuning get 10,000× cheaper?
Fine-tuning a giant model changes billions of weights — but the CHANGE itself is secretly simple. A real SVD slices a weight update into layers: slide the rank and watch 25% of the numbers rebuild 93% of the update.
From: LoRA (2021)Try this
- Slide rank from 1 to 3 — the reconstruction snaps into focus
- Check the singular value bars: after 3, they crash to a noise floor
- Compare the parameter counts at rank 3 vs the full matrix
LoRA Lab: how much of a weight update is really there?
A real SVD, sliced rank by rank
Full fine-tune ΔW
LoRA rank 2 (B·A)
Difference (lost)
The matrix's real singular values — its "directions of change", strongest first
Teal bars = directions your rank-2 adapter keeps. After the first three, the values crash to a noise floor — the update was low-rank all along. That is the paper's "low intrinsic rank" hypothesis, visible.
The decomposition, reconstructions, error percentages, and singular values are all real (SVD via power iteration, computed live). The ΔW matrix itself is synthetic, built with the low intrinsic rank the paper measured in genuine fine-tuning updates.
Rank 2 uses only 96 numbers (17% of the full update) and already captures 78% of the change. Look at the "difference" panel — mostly faint noise.
Analogy: A photo that's secretly just 3 layered gradients: store the 3 layers instead of every pixel and you keep almost the whole picture at a fraction of the cost.
Why does reading BOTH directions matter?
BERT learns language by fill-in-the-blank — with X-ray vision. Move the mask around a sentence, then toggle between left-only reading (how GPT sees) and both-sides reading (how BERT sees), and watch the confidence jump when the clue sits after the blank.
From: BERT (2018)Try this
- Mask “guitar”, read left-only — it's a coin toss
- Switch to both sides — “on stage” gives it away
- Try each maskable word and compare the confidence gap
Masked Word Lab
BERT's training game: fill in the blank — with X-ray vision
Blue-bordered words are maskable — click one to move the blank.
The model's guesses for [MASK]
Note: candidate words and their scores are curated illustrations of real BERT behavior (the softmax over them is computed live); the left-vs-both comparison is the paper's actual case for bidirectional pre-training.
Seeing BOTH sides (BERT-style), the words after the blank give it away — “chef” jumps to 61%, versus 27% confidence with left context alone. That confidence gap is the entire argument for bidirectional pre-training.
Analogy: A crossword clue you can read from both ends: knowing the words after the blank is often worth more than everything before it.
How do you teach AI what humans actually want — without it cheating?
A robot judge scores summaries; the writer chases high scores on a leash (the KL penalty). Drag the leash slider and feel all three regimes: too tight (nothing improves), just right (genuinely better answers), and cut loose — real reward hacking, where gibberish wins.
From: Learning to Summarize from Human Feedback (2020)Try this
- Set β to 1.0 — the good summary (A) takes the lead
- Drop β below 0.4 — watch the gibberish hack take over
- Check the KL number: that's the real 'distance from normal' cost
RLHF Lab: the reward vs leash tug-of-war
Why aligned models need a KL penalty — feel the trade-off
Post to summarize
My neighbor's cat sneaks into my house every day through the window, eats my cat's food, and then naps on my couch. My own cat just sits there and watches it happen. Should I say something to my neighbor about it?
Candidate summaries — policy probability
○ = supervised baseline's probability
A.Neighbor's cat sneaks in daily, eats my cat's food and naps on my couch — my cat doesn't mind. Should I talk to the neighbor?
r = 2.6
46.7%
B.A neighbor's cat keeps coming into my house and eating food. Asking if I should tell the neighbor.
r = 1.7
42.3%
C.I have a cat and a neighbor who also has a cat.
r = -0.5
8.5%
D.Cats! There are cats everywhere in my house, all day long.
r = -1.2
0.4%
E.Excellent heartwarming summary!!! 10/10 amazing best post reward reward reward
r = 6.5
2.1%
Note: the summaries and their scores are curated; the policy distribution, KL divergence, and the emergence of reward hacking come from the real KL-regularized-RL optimum π ∝ π_SFT·exp(r/β) — the exact trade-off the paper tunes with its KL coefficient.
Sweet spot: at β = 1.00 the policy shifts toward genuinely better summaries — “A” leads — while the KL penalty (0.70 nats) keeps it from wandering into the reward model's blind spots.
Analogy: Training a dog with treats (the reward model) while keeping it on a leash (the KL penalty). Drop the leash entirely and the dog stops herding sheep and starts mugging you for treats.