Dilated CNN

This model was built against a specific constraint: reach a receptive field greater than 44 on a $32\times32$ input, with no max-pooling and no stride-2 convolution anywhere. Pooling and stride are the usual way to grow the receptive field fast (the receptive-field page's $j$, the jump between adjacent output positions, doubles with every stride-2 step), but both spend resolution to buy reach. Three stride-2 layers on a $32\times32$ input already shrinks it to $4\times4$, leaving almost nothing for later layers to refine. The alternative this model bets on is dilation: widen what each kernel can see without touching the feature-map size or the weight count at all.

The result is a four-block stack, C1 through C4, that never changes spatial resolution, every feature map in the network is $32\times32$, yet the receptive field grows from 3 pixels after the first layer to 65 pixels by the last. That is more than double the input, with margin to spare.

The mechanism behind every row is the same recurrence from the receptive-field page, with dilation folded in as a change to the effective kernel size rather than a new formula. A dilated kernel with rate $d$ spaces its $k$ taps $d$ pixels apart, so it spans the same area as an ordinary kernel of size $k_{\text{eff}}$ while keeping the same $k^2$ weights:

Run the numbers for the C4 d=16 layer: $k_{\text{eff}} = 3 + (3-1)(16-1) = 3 + 30 = 33$. Plugging into the recurrence, $r_{\text{out}} = 33 + (33-1)\cdot 1 = 65$. A $3\times3$ kernel, still only 9 weights, behaves, for receptive-field purposes, like a $33\times33$ kernel. That single layer alone takes the receptive field from 33 to 65, doubling it without adding a single parameter.

The escalating rates are deliberate, not arbitrary. Stack dilated layers at the same rate and their sampling gaps line up into a checkerboard that nothing ever reads, a failure mode called gridding. Doubling the rate at every stage ($2 \to 4 \to 8 \to 16$) means each layer's gaps fall on positions an earlier layer's taps already covered, so the union of everything sampled stays dense even though no single layer is.

The chart below is this table drawn as a ladder. Each rung is one block; the bar length is that block's receptive field relative to the $32\times32$ input (the dashed line). Notice the C1 and C2 rungs are short and the same color, both are plain $3\times3$ ($d=1$), while the C3/C4 rungs are dilated and cross the input line by the fifth block.

C2 is where the parameter budget gets paid down. Splitting a $3\times3$ convolution into a depthwise pass (one filter per channel) plus a $1\times1$ pointwise pass (channel mixing only) cuts the weight count roughly $k^2$-fold for the same channel widths, the convolution page works the $64{\to}64$ case out fully: $576 + 4{,}096 = 4{,}672$ weights instead of $36{,}864$, about an 8x reduction. Crucially, this factorisation costs nothing in the receptive-field column: the depthwise pass contributes $+2$ to $r$ exactly like an ordinary $3\times3$ kernel ($k_{\text{eff}}=3$), while the pointwise $1\times1$ pass has $k_{\text{eff}}=1$ and adds zero. C2's row in the table above ($r: 3 \to 5$) is identical to what a plain $3\times3$ layer would produce, the parameter saving is free on the receptive-field side.

That saving is what keeps the whole six-layer stack (C1's plain conv, C2's depthwise-separable block, and four dilated convolutions in C3/C4) under the 200K-parameter assignment budget. The full network totals 98,730 parameters, under half that budget, while reaching a receptive field more than double the input. Spending the budget on dilation (which is free in parameters) and a factorised block (which is cheap in parameters) rather than on wider plain convolutions is the core design bet of this model.

Why no pooling at all? On a $32\times32$ input, every pooling or stride-2 layer halves the feature-map size, two of them and you are down to $8\times8$, three and you are at $4\times4$, with little spatial structure left for the dilated blocks to work on. This model's bet is that dilation alone can supply all the receptive-field growth a small image needs, so resolution is preserved for the entire forward pass and the final $32\times32\times C$ feature map is reduced only at the very end, by global average pooling into the 10 CIFAR-10 class logits.

Training used Albumentations for augmentation: horizontal flips, shift-scale-rotate, and coarse dropout, which matters here specifically because a network with this much spatial coverage per neuron benefits from seeing the same object shifted, rotated, and partially occluded. Every one of those transformations falls well within a single neuron's $65\times65$ field of view by the final layer.

Grad-CAM answers a simple question: which pixels did the network actually rely on for this prediction? It runs a backward pass from the predicted class back to the last convolutional feature map, weights each channel by how strongly it influenced that class, and overlays the result as a heatmap on the input image. Warm regions are pixels the network's decision is sensitive to; cool regions could change freely without moving the prediction.

This model is an unusually clean subject for Grad-CAM, for exactly the reason the architecture tab spent so much time on: the final convolutional feature map is still $32\times32$, the same resolution as the input. Most CNNs run Grad-CAM on a heavily downsampled map (ResNet-50's last stage is $7\times7$ before any interpolation), so the heatmap has to be upsampled and gets blurry. Here the heatmap is already at input resolution, so it can localise to individual object parts rather than coarse $32\times32$-to-$224\times224$ blocks.

There is a second, more interesting reading available on this specific model: because every neuron in that final feature map has a $65\times65$ receptive field on a $32\times32$ input, every single one of them can already see the entire image, there is no position whose Grad-CAM weight is structurally blind to any pixel. Differences in the heatmap are therefore differences in what the network chose to weight, not differences in what it could see. If you compare this model's Grad-CAM output against the MNIST CNN (a much smaller receptive field relative to its input), any sharper localisation here is a learned preference, not a geometric constraint.

Open the workspace to run this on a real image: upload or pick a CIFAR-10-style photo, get the top-5 prediction, and overlay the Grad-CAM heatmap for any of the five classes. Try a borderline image, something between two visually similar classes like cat and dog, and watch whether the heatmap shifts to different regions of the same image depending on which class's gradient you trace back.

An adversarial attack asks: how little can you change an image, while keeping it visually identical, to make the model misclassify it? FGSM (Fast Gradient Sign Method) answers this in one step. It computes the gradient of the loss with respect to the input pixels (the same backward pass Grad-CAM uses, but stopped one layer earlier) and nudges every pixel by a fixed amount $\varepsilon$ in whichever direction increases the loss. The result is the sign of the gradient, scaled by $\varepsilon$:

The $\varepsilon$ here is an $L_\infty$ budget: a cap on the largest change to any single pixel channel, with pixels on a $[0, 1]$ scale. $\varepsilon = 0.03$ means no channel of no pixel moves by more than 3% of the full range, visually that is faint static, often imperceptible at normal viewing size. The slider below builds intuition for that budget directly: drag $\varepsilon$ and watch the allowed perturbation band widen around a single example pixel.

The unsettling result, first shown by Goodfellow et al. (2014), is that small $\varepsilon$ values, well below the noise level a human would notice, are often enough to flip a confident prediction. The reason connects back to this model's receptive field: a $65\times65$-RF neuron is a function of every pixel in the image, so an attack does not need to find one critical pixel. It can spread a tiny, coordinated nudge across all of them, and the network's own sensitivity (the gradient) tells the attacker exactly which direction each pixel should move.

A second robustness question is orthogonal to attacks: does the model know what it doesn't know? Feed it an image from a class it was never trained on, an out-of-distribution (OOD) input, and a well-calibrated model should respond with low confidence and high uncertainty rather than a confident wrong answer. This model exposes that signal through two numbers computed from its output logits: the softmax confidence (the probability mass on the top predicted class) and the predictive entropy (how spread out the probability mass is across all 10 classes). A clean, in-distribution CIFAR-10 image typically produces high confidence and low entropy, the distribution is peaked on one class. An OOD image more often produces lower confidence and higher entropy, the distribution is closer to uniform, the model is effectively shrugging.

Both probes are live in the workspace: drag the FGSM $\varepsilon$ slider on a real image and watch the prediction flip as confidence collapses, then feed in an out-of-distribution image and compare its confidence/entropy readout against a clean CIFAR-10 photo. Together they answer two different questions, FGSM asks can this model be fooled on purpose?, OOD asks does this model notice when it's seeing something new?, and a small, fully-resolution-preserving network like this one is informative on both.