The Big Bang of modern AI: a GPU-trained convolutional network crushed the world's hardest image contest by 10 points — and convinced the world that deep learning works.
Five animated scenes — everything you need to understand this paper, no reading required. Then scroll on to play with the ideas yourself.
ImageNet Classification with Deep Convolutional Neural Networks (AlexNet) — the picture-book version
No paper-reading needed — the pictures tell the whole story
To a computer, a photo is just a giant grid of numbers.
No dog, no colors, no shapes — only millions of brightness values. So how can it ever SEE?
The same ideas at three altitudes — start where you're comfortable, climb when you're curious.
ImageNet was the Olympics of computer vision: 1.2 million photos, 1,000 categories — from 'Siberian husky' to 'espresso'. In 2012, the best traditional methods got about a quarter of their guesses wrong. Then a neural network built by three researchers cut that error nearly in half: 15.3% versus 26.2% for second place. In competitions, you win by decimals. Winning by ten points meant something fundamental had changed.
AlexNet looks at pictures the way you built up reading: first tiny patterns, then combinations. Its early layers slide little pattern-detectors across the image — edges, colors, textures (exactly what the CNN Lab below shows). Middle layers combine those into eyes, wheels, fur. The final layers vote: 'husky: 43%, wolf: 31%…'. Nobody programmed any of those detectors — the network invented them all from examples.
The ideas were old — convolutional networks existed since the 1980s. What was new was the horsepower: two consumer gaming GPUs (the kind made for rendering video games) trained the network for about a week. Deep learning didn't win because someone had a new theory; it won because someone finally gave the old theory enough fuel. Everything since — including ChatGPT — runs on that same discovery.
Every great paper is a story: what was broken, the idea that fixed it, and what it unlocked.
Trying to describe every animal to a blind sorter with hand-written rules — the rules never end, and every new animal breaks them.
Stop writing rules; show the sorter a million labeled photos and let it invent its own rules — with enough coffee (GPUs) to get through them.
The four-minute mile: once broken, everyone broke it — because now they knew it could be done.
AlexNet is five layers of exactly this: a filter sliding across the image, building a feature map. Watch it happen, then imagine 96 filters, five layers deep.
How computers see: a tiny magnifying glass scanning a picture
Input
Feature map (10×10)
Compares each pixel's left side to its right side. Big difference = a vertical edge lives here. Press play to watch the filter scan the image — this exact multiply-and-add is what every CNN layer does, millions of times.
Analogy: Sliding a small stencil over a photo and asking at every spot: how much does this patch look like my pattern?
Follow a sentence through the model. Every block is clickable — purpose, example, and the deeper detail.
Click any block to inspect it
96 filters, 11×11, stride 4
Ninety-six large filters sweep the image (the exact operation in the CNN Lab below). After training, they self-organize into oriented edge detectors and color-blob detectors — nobody designed them.
Example
One filter fires along the husky's back edge; another responds to its blue-grey fur color.
Go deeper
The paper's Figure 3 — the learned filters — became iconic: Gabor-like edges on one GPU, color blobs on the other, emerging purely from gradient descent.
Tip: follow the animated edges top-to-bottom — that is the exact journey a sentence takes through the model.
Formula → plain meaning → real-world analogy. Hover the symbols to decode them one by one.
Hover a symbol to decode it.
What it means
Pass positives through unchanged, zero out negatives. Because the slope is exactly 1 for any positive input, gradients never shrink through active units — unlike tanh/sigmoid, which flatten out and choke learning in deep stacks.
Real-world analogy
Tanh neurons are dimmer switches that stick near the ends; ReLU is a simple on/off valve that never sticks — signals (and blame) flow straight through.
Hover a symbol to decode it.
What it means
Slide a small filter K over the image I; at each stop, multiply overlapping values and sum. One filter, reused everywhere — so a pattern learned in one corner is recognized in every corner, with a tiny number of parameters.
Real-world analogy
One stencil checked against every window of the photo — exactly what you can watch, step by step, in the CNN Lab on this page.
Hover a symbol to decode it.
What it means
During training, each neuron in the big FC layers is switched off with probability 0.5 — a fresh random mask every step. Neurons can't co-adapt into brittle conspiracies, so the network learns redundant, robust features. At test time all neurons run (outputs scaled).
Real-world analogy
A sports team where a random half of the players sit out every practice — no play can depend on one star, so everyone becomes genuinely good.
Hover a symbol to decode it.
What it means
Turn 1,000 raw class scores into probabilities, then penalize the network by how little probability it gave the true class. Confidently wrong = huge loss; confidently right = near zero.
Real-world analogy
A bookmaker forced to publish odds on all 1,000 outcomes, then fined by how badly they priced the actual winner.
The same recipe at pocket scale: a real network learns to read your handwriting, live — ReLU, softmax, gradient descent and all.
Draw a digit, watch 1,860 real weights read it — an homage to 3Blue1Brown
Draw here
10×10 pixels = the network's 100 input neurons
100 → 16 → 16 → 10
The network says
·
hover any neuron to see inside it
100% real, in homage to 3Blue1Brown's “But what is a neural network?”: this is a genuine 100→16→16→10 network (the video's 16-16 hidden architecture on a 10×10 canvas) trained in your browser with real mini-batch gradient descent. Neuron glow = actual activations from your ink; edge colors = actual signed weights; the hover stencils are the network's true learned first-layer weights.
This 100→16→16→10 network is untrained — draw something and every digit gets ~10%, a pure guess. Press play to train it live on 400 noisy digits.
Analogy: Each neuron is a tiny inspector holding a stencil. Layer 1 inspectors look for strokes and edges; later layers combine their reports: “loop on top + line below… that's a 9.”
Eight pages from 2017 that you now interact with every single day.
Face unlock, photo search, medical imaging, satellite analysis — the CNN pipeline AlexNet validated became the substrate of every vision product of the 2010s.
AlexNet proved gaming GPUs were AI hardware. The CUDA/cuDNN/framework ecosystem — and NVIDIA's central role in AI — trace to this demonstration.
AlexNet features, frozen and reused, turned out to beat hand-crafted features on almost any vision task — creating the pretrain-then-adapt pattern BERT later brought to language.
The 2012 result re-legitimized neural networks across speech, translation, and games — the funding, talent, and belief that produced the modern AI era started here.
Cell counting, tumor detection, protein localization microscopy — CNN feature hierarchies became standard tools of quantitative biology and medicine.
ZFNet, VGG, GoogLeNet, ResNet, and eventually Vision Transformers — each ILSVRC generation refined AlexNet's recipe until the contest itself was solved.
Reading real code is the final test of understanding. This is the paper's core mechanism, minimally.
The 2012 architecture, line by line — five convs, three FC layers, ReLU everywhere, dropout in the classifier. What took a custom two-GPU CUDA implementation then is ~30 declarative lines today.
1import torch2import torch.nn as nn345class AlexNet(nn.Module):6 """Krizhevsky, Sutskever & Hinton (NeurIPS 2012)."""78 def __init__(self, num_classes: int = 1000):9 super().__init__()10 self.features = nn.Sequential(11 # Conv1: 96 big filters hunting edges and color blobs12 nn.Conv2d(3, 96, kernel_size=11, stride=4, padding=2),13 nn.ReLU(inplace=True), # the 6x speed unlock14 nn.MaxPool2d(kernel_size=3, stride=2), # overlapping pooling15 # Conv216 nn.Conv2d(96, 256, kernel_size=5, padding=2),17 nn.ReLU(inplace=True),18 nn.MaxPool2d(kernel_size=3, stride=2),19 # Conv3-5: composing edges -> textures -> object parts20 nn.Conv2d(256, 384, kernel_size=3, padding=1),21 nn.ReLU(inplace=True),22 nn.Conv2d(384, 384, kernel_size=3, padding=1),23 nn.ReLU(inplace=True),24 nn.Conv2d(384, 256, kernel_size=3, padding=1),25 nn.ReLU(inplace=True),26 nn.MaxPool2d(kernel_size=3, stride=2),27 )28 self.classifier = nn.Sequential(29 nn.Dropout(p=0.5), # tame the 60M params30 nn.Linear(256 * 6 * 6, 4096),31 nn.ReLU(inplace=True),32 nn.Dropout(p=0.5),33 nn.Linear(4096, 4096),34 nn.ReLU(inplace=True),35 nn.Linear(4096, num_classes), # 1000-way verdict36 )3738 def forward(self, x: torch.Tensor) -> torch.Tensor:39 x = self.features(x)40 x = torch.flatten(x, 1)41 return self.classifier(x)424344net = AlexNet()45out = net(torch.randn(1, 3, 224, 224))46print(out.shape) # torch.Size([1, 1000])47print(sum(p.numel() for p in net.parameters()) / 1e6, "M params") # ~61M
Four questions, each targeting a concept people commonly get wrong. Mistakes come with explanations.
Question 1 of 4
What was the dominant approach to image recognition before AlexNet?