AlexNet: the week that started modern AI
A 4-lesson mini course: the hand-crafted era, learned features, the training tricks that stuck, and the avalanche that followed.
Before: vision by hand
Features by committee
Pre-2012 vision worked like this: experts designed feature extractors by hand — SIFT keypoints, HOG gradients, color histograms — then fed those features to shallow classifiers like SVMs. Every improvement was a new hand-crafted descriptor. On ImageNet's 1,000 classes, this machinery plateaued around 26% top-5 error, improving by whiskers each year.
The neural network winter
Convolutional networks existed — LeCun's LeNet read bank checks in the 90s — but scaling them seemed hopeless: deep nets with tanh/sigmoid neurons trained agonizingly slowly (saturating gradients), overfit wildly, and CPUs of the era would have needed months per experiment. 'Neural nets don't scale' was respectable consensus.
The three missing pieces
What changed by 2012: ImageNet provided 1.2 million labeled images (enough data to feed a big model), gaming GPUs provided the raw FLOPs, and a handful of tricks — ReLU, dropout — fixed the training pathologies. AlexNet is the paper that assembled all three and pulled the trigger.
Key takeaway
Vision was hand-crafted features + shallow classifiers, plateaued at ~26% error. Deep CNNs existed but lacked data, compute, and training tricks — until 2012.
Hands-on — experiment before the quiz
CNN Convolution Lab
How computers see: a tiny magnifying glass scanning a picture
Input
Feature map (10×10)
Compares each pixel's left side to its right side. Big difference = a vertical edge lives here. Press play to watch the filter scan the image — this exact multiply-and-add is what every CNN layer does, millions of times.
Analogy: Sliding a small stencil over a photo and asking at every spot: how much does this patch look like my pattern?
Checkpoint
What was the dominant approach to image recognition before AlexNet?