CIFAR-100 ResNet-18

ResNet-18 is built from one repeated unit: the basic residual block. Each block runs the input through two 3×3 convolutions (each followed by batch norm and a ReLU), then adds the block's input back to that result before a final ReLU. In symbols, $h_{l+1} = \text{ReLU}\bigl(h_l + F(h_l)\bigr)$ where $F$ is the two-conv branch — exactly the form the Vanishing Gradients page calls the residual fix. When a block also needs to change the number of channels or halve the spatial size, the shortcut itself becomes a 1×1 convolution with stride 2 — a projection shortcut — so the addition still has matching shapes on both sides.

Eighteen layers count as: 1 stem convolution, 4 stages of 2 basic blocks each (2 convs per block = $4 \times 2 \times 2 = 16$ convs), and 1 final linear layer — $1 + 16 + 1 = 18$. The four stages widen channels and shrink space in lockstep:

Why this shape? Each halving of spatial size is paired with a doubling of channels so the amount of computation per stage stays roughly constant — fewer spatial positions, but each one now costs roughly 4× as much (a 3×3 conv's cost scales with $C_{\text{in}} \cdot C_{\text{out}}$, and both doubled). This is the same convolution parameter-count formula, $\text{params} = C_{\text{out}}(k \cdot k \cdot C_{\text{in}} + 1)$, applied four times at increasing width: a stage-1 3×3 conv at 64→64 channels costs $64 \times (9 \times 64) \approx 36.9$K parameters, while the corresponding stage-4 conv at 512→512 costs $512 \times (9 \times 512) \approx 2.36$M — about 64× more, tracking the $8 \times 8$ growth in $C_{\text{in}} \cdot C_{\text{out}}$. Stack all four stages' convolutions, the stem, and the 512→100 classifier and the total lands at about 11 million parameters — small enough to train from scratch in hours, large enough to reach real accuracy on a 100-class problem.

Each halving also doubles the receptive field relative to the input, on top of the growth each 3×3 conv already contributes within a stage — by the receptive-field arithmetic, a stride-2 layer doesn't just shrink the map, it doubles the input-pixel footprint of every neuron downstream. By stage 4, a single 4×4 output cell already depends on a large fraction of the original 32×32 image — which is exactly what global average pooling needs: by the time the spatial map is collapsed to one vector, each of its 16 positions has already integrated information from most of the image.

Trained from scratch, this network reached 76.61% validation top-1 at epoch 99 of 100 — 100 epochs of SGD with an initial learning rate of 0.1 on a cosine schedule decaying smoothly toward zero. The curve below is the real per-epoch validation accuracy from that run, not a schematic: notice the late-training jump from the high-60s to the mid-70s as the cosine schedule's low-LR tail lets the model settle into a sharper minimum.

Every basic block in this network computes $h_{l+1} = h_l + F(h_l)$ — an addition, not a substitution. Differentiate it and the chain rule produces a local Jacobian that is the identity matrix plus whatever the two-conv branch contributes:

Chain that across all 16 residual blocks and the gradient reaching the stem is a product of 16 terms that each look like "identity plus something" — never a product of 16 terms each smaller than 1. Even in the degenerate case where every $F$ branch has gone to sleep ($\partial F/\partial h_l \approx 0$ everywhere), the product of 16 copies of $I$ is still $I$: the gradient arrives at the first layer completely undiminished.

This is the gradient highway — an additive path with multiplier 1 running underneath the entire 18-layer network, with the two-conv branches as side streets adding detail on top. Without it, an 18-layer plain (non-residual) ReLU network is already deep enough for the per-layer attenuation to compound noticeably; with it, depth stops being the thing that limits how well the network trains.

None of that is hypothetical here — it's measurable on this exact model's trained weights. The panel below runs a real backward pass: upload any image, and it records the L2 norm of the gradient at the first convolution of every residual block, from the block closest to the loss back to the block closest to the input. The Vanishing Gradients page shows a schematic version of this chart for a plain network vs. a residual one — this is the live, non-schematic version, computed on the same 76.61%-accuracy weights used everywhere else on this page.

What to look for: a roughly flat profile across all 8 blocks means every block — including the one nearest the input — is still receiving a useful training signal after 100 epochs. A profile that decays sharply toward block 1 would mean the earliest layers had effectively stopped learning, the failure mode this entire architecture exists to avoid.

Grad-CAM answers the question "which pixels mattered for this prediction?" by running a backward pass from the predicted class's score down to the last convolutional feature map, then weighting each channel of that map by how much it influenced the score on average. The result is a coarse heatmap — one value per spatial position of the final feature map, upsampled back to the input's resolution — that highlights the regions the network's last convolutional stage was looking at when it made its decision.

For this model, that final feature map is stage 4's 4×4×512 output — the same tensor described in the Architecture tab, right before global average pooling. Sixteen spatial positions is coarse, but for a 32×32 CIFAR-100 image it is still enough to distinguish "the network is looking at the animal's head" from "the network is looking at the background texture," which is often exactly the distinction that separates a correct prediction from a confident mistake.

Grad-CAM on this model is live in the workspace — upload any image and the overlay renders on top of it alongside the top-5 class predictions. Two things are worth specifically checking there: whether the heatmap concentrates on the object at all (vs. spreading across the whole frame, a sign the prediction may be relying on background correlations), and whether a wrong top-1 prediction still has a heatmap that makes sense — the network often "looked at the right thing" and simply confused two visually similar classes among CIFAR-100's 100 fine-grained categories.

A model graded only on clean validation images can hide a brittle decision boundary. Common-corruptions testing applies a fixed family of perturbations — Gaussian noise, Gaussian blur, JPEG compression artifacts, and brightness shifts — at increasing severity, and asks: how much does accuracy degrade as each corruption strengthens? A model that is robust loses accuracy gradually and predictably; a model that has overfit to the exact pixel statistics of its training set can fall off a cliff at moderate severity even when the corruption looks mild to a human.

The workspace runs this model under all four corruption types with a severity slider, so you can watch top-5 predictions and confidence shift in real time as noise, blur, JPEG quality, or brightness change. Blur and low-resolution noise are particularly informative for this model: CIFAR-100 images are already only 32×32, so a corruption that would be barely noticeable on a 224×224 ImageNet photo can remove a large fraction of the total information available at this resolution.

Out-of-distribution (OOD) detection asks a different question: not "is the prediction correct," but "does the network know when it's looking at something it wasn't trained on?" A network trained on CIFAR-100's 100 fine-grained classes will still emit a confident-looking softmax distribution over those 100 classes for any input — including images of things that aren't in any of those classes at all. The standard signals for catching this are the maximum softmax confidence (how high is the top class's probability) and the entropy of the full distribution (how spread out is the probability mass across all 100 classes) — a genuinely in-distribution image typically produces high confidence and low entropy, while an OOD image more often produces a flatter, higher-entropy distribution, though neither signal is a guarantee on its own.

The workspace surfaces both numbers for every prediction, so feeding it an image far outside CIFAR-100's 100 categories — a photo of a screenshot, a hand-drawn sketch, pure noise — is a direct way to see whether this model's confidence and entropy actually track "how much does this look like something I was trained on," or whether it confidently mis-files everything into its 100-class comfort zone regardless of input.