This paper solved deep learning's strangest bug — deeper networks were getting WORSE — with one line of math: let every layer add its input back to its output.
Five animated scenes — everything you need to understand this paper, no reading required. Then scroll on to play with the ideas yourself.
Deep Residual Learning for Image Recognition — the picture-book version
No paper-reading needed — the pictures tell the whole story
Everyone believed: more layers = a smarter network.
8 layers won in 2012, 19 layers won in 2014… so just keep stacking, right?
The same ideas at three altitudes — start where you're comfortable, climb when you're curious.
Everyone believed more layers = smarter network. But when researchers stacked past ~20 layers, something bizarre happened: a 56-layer network performed WORSE than a 20-layer one — even on its own training data. That's not overfitting (memorizing too much); the deep network couldn't even learn what the shallow one learned. Something about depth itself was broken.
ResNet's idea fits in one sentence: let each layer add its input back to its output. Instead of every layer re-describing the whole picture from scratch, each layer just suggests a small edit: 'keep everything you got, plus this little correction.' The 'shortcut' or 'skip connection' carries the original signal straight through, untouched.
If a layer has nothing useful to add, it can simply do nothing — output zero — and the shortcut passes the input along unharmed. A deep network can never be worse than a shallow one, because extra layers can always act invisible. With that guarantee, depth became free: the authors trained a 152-layer network, won the 2015 ImageNet competition, and made 'very deep' the default forever.
Every great paper is a story: what was broken, the idea that fixed it, and what it unlocked.
A message whispered through 56 people arrives more garbled than through 20 — even though 36 of them could have just repeated it exactly.
Editing a document with tracked changes instead of retyping it from memory at every step.
Like the invention of the elevator: suddenly buildings — and networks — could be as tall as you wanted.
Two real networks of identical depth train side by side in your browser. The only difference: one has skip connections. This is the paper's Figure 1, running in front of you.
Two real deep networks train live — one plain, one residual
Plain · 12 layers
loss — · acc —
ResNet · 12 layers + skips
loss — · acc —
The race — loss over time
100% real: both networks genuinely train in your browser with exact backpropagation (gradients verified against finite differences to ~1e-10). The residual network computes literally y = F(x) + x, the equation from the paper. Same initialization luck, same data, same learning rate — the skips are the only difference.
Two identical 12-layer networks, same data, same starting luck. The ONLY difference: the teal one adds its input back to every layer's output — y = F(x) + x, the paper's one-line idea. Press play and watch them race.
Analogy: Passing a message through 12 rooms of people. Plain network: each room must re-tell everything from scratch. ResNet: each room gets the original note and just adds a correction to it.
Follow a sentence through the model. Every block is clickable — purpose, example, and the deeper detail.
Click any block to inspect it
conv → BN → ReLU → conv → BN
The 'working' path of a residual block: two 3×3 convolutions (with batch normalization) compute a small correction to the input — not a replacement for it.
Example
For the dog photo, F(x) might sharpen 'floppy ear' evidence — a small edit to an already-rich signal.
Go deeper
In ResNet-50/101/152 this becomes a bottleneck: 1×1 conv reduces channels, 3×3 works in the small space, 1×1 restores — deep but cheap. ResNet-152: 11.3 GFLOPs vs VGG-19's 19.6.
Tip: follow the animated edges top-to-bottom — that is the exact journey a sentence takes through the model.
Formula → plain meaning → real-world analogy. Hover the symbols to decode them one by one.
Hover a symbol to decode it.
What it means
A block's output is its input, plus a learned correction. The layers inside F no longer carry the whole signal — they only compute what should CHANGE.
Real-world analogy
Editing with tracked changes: the document (x) flows through untouched, and each editor (F) only marks their suggested edits on top.
Hover a symbol to decode it.
What it means
If H(x) is what the block ideally should compute, ResNet asks the layers to learn the difference between that and the input. When the ideal mapping is close to identity — common in deep nets — the target F is close to zero, and learning 'nothing' is the easiest task a network has.
Real-world analogy
Asking 'what should I change about this drawing?' instead of 'redraw this from memory'. If the drawing is already right, the first question answers itself.
Hover a symbol to decode it.
What it means
Differentiate the block and the shortcut contributes an identity term. During backpropagation, the error signal always has a path that multiplies by 1 — never shrinking — no matter how many blocks it crosses. Plain networks only have the F path, which fades layer after layer.
Real-world analogy
A whisper chain where every person ALSO holds a copy of the original note. However garbled the whispers get, the note arrives intact at the first layer.
Hover a symbol to decode it.
What it means
Between stages, the feature map halves in resolution and doubles in channels, so x and F(x) have different shapes. A single 1×1 convolution Wₛ reshapes x to match — used only where needed.
Real-world analogy
The shortcut is a moving walkway; when the corridor changes width, a small ramp (Wₛ) adjusts your lane — but you never leave the walkway.
Every residual block is built from this operation. Watch a real filter slide across an image — then imagine 152 layers of it, made trainable by one addition.
How computers see: a tiny magnifying glass scanning a picture
Input
Feature map (10×10)
Compares each pixel's left side to its right side. Big difference = a vertical edge lives here. Press play to watch the filter scan the image — this exact multiply-and-add is what every CNN layer does, millions of times.
Analogy: Sliding a small stencil over a photo and asking at every spot: how much does this patch look like my pattern?
Eight pages from 2017 that you now interact with every single day.
For years, 'use a ResNet-50' was step one of every computer vision project — classification, detection (Faster/Mask R-CNN), segmentation — and it remains a standard baseline today.
Every 'Add & Norm' in the Transformer — and therefore inside GPT, Claude, and Gemini — is a residual connection. Without ResNet's idea, 96-layer language models would not train.
DeepMind's game-playing and protein-folding systems are built on deep residual towers — dozens of skip-connected blocks processing board states and molecular features.
Diabetic retinopathy screening, tumor detection, and X-ray triage systems run on residual backbones — depth captures subtle pathology that shallow nets miss.
DenseNet, EfficientNet, ConvNeXt, Vision Transformers — every successful architecture since 2015 keeps residual paths. The skip connection outlived every other design choice of its era.
The U-Nets inside image generators like Stable Diffusion are packed with residual blocks — the same y = F(x) + x, now generating images instead of classifying them.
Reading real code is the final test of understanding. This is the paper's core mechanism, minimally.
The paper's entire idea in code. Notice the two lines that matter: saving the input, and adding it back. Delete `out += identity` and you have a plain network — the one that loses the race in the lab above.
1import torch2import torch.nn as nn345class ResidualBlock(nn.Module):6 """Basic block from 'Deep Residual Learning' (ResNet-18/34)."""78 def __init__(self, channels: int):9 super().__init__()10 # The residual function F(x): conv -> BN -> ReLU -> conv -> BN11 self.conv1 = nn.Conv2d(channels, channels, kernel_size=3,12 padding=1, bias=False)13 self.bn1 = nn.BatchNorm2d(channels)14 self.conv2 = nn.Conv2d(channels, channels, kernel_size=3,15 padding=1, bias=False)16 self.bn2 = nn.BatchNorm2d(channels)17 self.relu = nn.ReLU(inplace=True)1819 def forward(self, x: torch.Tensor) -> torch.Tensor:20 identity = x # <-- save the input2122 out = self.relu(self.bn1(self.conv1(x)))23 out = self.bn2(self.conv2(out)) # this is F(x)2425 out += identity # <-- y = F(x) + x (the whole paper)26 return self.relu(out) # ReLU after the addition272829# --- A deep network is now just a stack of these ---30class TinyResNet(nn.Module):31 def __init__(self, num_blocks: int = 18, channels: int = 64):32 super().__init__()33 self.stem = nn.Conv2d(3, channels, kernel_size=7,34 stride=2, padding=3, bias=False)35 self.blocks = nn.Sequential(36 *[ResidualBlock(channels) for _ in range(num_blocks)]37 )38 self.head = nn.Linear(channels, 1000)3940 def forward(self, x):41 x = self.stem(x)42 x = self.blocks(x) # 18 blocks deep, trains happily43 x = x.mean(dim=(2, 3)) # global average pooling44 return self.head(x)454647net = TinyResNet()48out = net(torch.randn(1, 3, 224, 224))49print(out.shape) # torch.Size([1, 1000])
Four questions, each targeting a concept people commonly get wrong. Mistakes come with explanations.
Question 1 of 4
Why did the 56-layer plain network's HIGHER TRAINING error matter so much?