Section 13.3: Multiresolution and dilated features

"A footstep is milliseconds, a walk is seconds, a workout is minutes. Give me one kernel and I will miss two of them."
A Multi-Scale AI Agent

The Big Picture

Physical processes rarely live at a single time scale. An accelerometer sees the sharp impulse of a heel strike, the roughly one-second cadence of a stride, and the slow drift of posture across a minute, all superimposed in one channel. A small convolution kernel (Section 13.2) sees only a narrow window of samples, so a naive stack must go very deep before any neuron can relate a fast event to its slow context. This section is about seeing many scales at once, cheaply. Two ideas do most of the work: dilation, which spreads a fixed-size kernel over a wide span so the receptive field grows exponentially with depth, and parallel multiresolution branches, which run several kernel widths side by side and let the network fuse them. Together they are the deep-learning answer to the classical multiresolution idea you met with wavelets in Chapter 7.

This section assumes you are comfortable with the 1D convolution and pooling machinery of Section 13.2 and the window-as-tensor layout of Section 13.1. If the phrase "receptive field" is unfamiliar, it simply means the span of input samples that can influence one output value; enlarging it efficiently is the whole game here.

Why a single scale is never enough

Consider a window of $L$ samples from a wrist accelerometer at 50 Hz. A kernel of width $k=5$ touches 100 ms of signal. That is plenty to detect the shape of one impact but far too little to know whether the wearer is jogging or climbing stairs, distinctions that only appear over one or two full gait cycles (a second or more, roughly 50 to 100 samples). Stacking plain convolutions does grow the receptive field, but only linearly: with $D$ layers of stride-1 kernels of width $k$, the receptive field is

$$ R = 1 + D\,(k-1). $$

To cover 128 samples with $k=3$ you would need 64 layers. That is expensive, hard to train, and wasteful, because most of those layers add resolution the task does not need. The multiresolution question is: how do we reach far-apart samples with few layers while keeping fine detail available where it matters?

Dilated convolutions: exponential reach for free

What. A dilated (or atrous) convolution inserts gaps of $d-1$ zeros between the kernel taps, so a width-$k$ kernel with dilation $d$ spans $(k-1)d + 1$ input samples while still using only $k$ weights and $k$ multiply-adds. Formally, for input $x$ and kernel $w$,

$$ y[t] = \sum_{i=0}^{k-1} w[i]\, x\big[t - i\,d\big]. $$

Why. Stack layers whose dilation doubles, $d = 1, 2, 4, 8, \dots$, and the receptive field grows exponentially rather than linearly:

$$ R = 1 + (k-1)\sum_{\ell=0}^{D-1} 2^{\ell} = 1 + (k-1)\,(2^{D}-1). $$

With $k=3$, just seven stacked layers reach $R = 255$ samples, over five seconds of that 50 Hz stream, using seven layers instead of 127. This is the WaveNet construction, and it is why dilation became the default for long-context sensor and audio models.

How. In every mainstream framework dilation is a single argument on the convolution layer; you do not build the zero-stuffed kernel yourself. The one subtlety is padding: to keep the length $L$ fixed you pad each layer by $(k-1)d/2$ on each side (symmetric), or pad only on the left when the model must be causal (no peeking into the future), which matters for the streaming inference you will meet in Chapter 14.

Key Insight

Dilation buys reach, not resolution. A single deep dilated stack can see far, but a pure doubling schedule leaves gridding artifacts: some output positions never attend to certain input phases, because the sampled taps interleave badly. The fix is to combine scales rather than rely on one ladder. Either interleave dilation cycles (1, 2, 4, 1, 2, 4) or run several widths in parallel and fuse them, which is the multiresolution-branch idea below. Reach and resolution are different resources; a good design spends on both.

Parallel multiresolution branches

The complementary strategy runs several kernel sizes on the same input in parallel, then concatenates their outputs along the channel axis. A branch with $k=3$ captures sharp transients; a branch with $k=39$ captures slow envelope shape; the concatenation hands the next layer a feature vector that already mixes scales. This is the InceptionTime pattern for time series, itself borrowed from Inception in vision. A common trick keeps it cheap: a $1\times1$ "bottleneck" convolution reduces the channel count before the wide kernels run, so the wide branches cost far less than their width suggests.

Why prefer parallel branches over a deeper dilated stack? Because the network chooses, per feature, which scale is informative, instead of committing to one growth schedule. Two-second bearing vibration and 100 ms fault clicks can coexist as separate learned channels rather than fighting for the same kernel. The cost is more parameters and memory at each layer, so branches suit richer models while dilation suits long, thin, streaming ones.

Practical Example: catching a bearing fault before it screams

A wind-turbine gearbox is monitored by a 20 kHz accelerometer. An incipient outer-race bearing fault shows up as short impulse trains (each impact lasts under a millisecond) that repeat at the bearing's characteristic frequency (tens of hertz), all riding on the turbine's slow one-per-revolution modulation (about 0.3 Hz). A single-scale CNN tuned to the impulses missed the periodic structure and fired on isolated electrical spikes. Swapping the first block for three parallel branches (widths 3, 11, 31) plus a dilated tail reaching about 0.5 s let one set of channels lock onto the impulse shape while others tracked the repetition envelope. False alarms on the validation fleet dropped noticeably, and the model flagged two units days before the vibration RMS crossed the classical threshold used in Chapter 37.

Designing and reasoning about the receptive field

Before training anything, compute the receptive field and confirm it covers the shortest task-relevant pattern and its context. Under-reaching silently caps accuracy; over-reaching wastes compute and invites overfitting to window edges. The snippet below builds a small dilated stack and reports both the receptive field and the parameter count, so you can size the model on paper.

import torch, torch.nn as nn

def dilated_block(ch, k=3):
    layers, rf, d = [], 1, 1
    for _ in range(6):                      # dilations 1,2,4,8,16,32
        pad = (k - 1) * d // 2              # keep length fixed
        layers += [nn.Conv1d(ch, ch, k, padding=pad, dilation=d),
                   nn.BatchNorm1d(ch), nn.ReLU()]
        rf += (k - 1) * d                   # linear per layer, d doubles
        d *= 2
    return nn.Sequential(*layers), rf

net, rf = dilated_block(ch=16, k=3)
x = torch.randn(8, 16, 256)                 # batch, channels, time
print("receptive field:", rf, "samples")   # -> 127
print("output shape:", tuple(net(x).shape)) # -> (8, 16, 256), length preserved
print("params:", sum(p.numel() for p in net.parameters()))

A six-layer dilation-doubling stack over a 16-channel, 256-sample window. The loop accumulates the receptive field analytically (reaching 127 samples) while symmetric padding preserves length, so you can verify coverage before spending a single training step.

Notice that the receptive field is a property of the architecture, computed without data, whereas the effective receptive field (how much each input actually influences the output) is usually smaller and concentrated near the center; you inspect it empirically with the filter-visualization tools of Section 13.7. Sizing on paper first, then verifying, is the disciplined order.

Library Shortcut

Zero-stuffing a kernel and re-deriving the convolution by hand is dozens of lines and an easy source of off-by-one bugs. In PyTorch the entire mechanism is one keyword: nn.Conv1d(ch, ch, kernel_size=3, dilation=8, padding=8). The framework handles the sparse tap placement, the padding arithmetic, and a fused CUDA kernel that never materializes the zeros. That replaces roughly 30 lines of manual striding and indexing with one, and it stays correct when you switch to causal padding for streaming.

Exercise

You have a 100 Hz IMU stream and must classify gestures whose longest meaningful pattern is about 1.5 seconds. (a) Using $k=3$, find the smallest number of dilation-doubling layers whose receptive field covers 150 samples. (b) Compare the layer count and total multiply-adds against a plain (dilation-1) stack reaching the same span. (c) Redesign the first block as three parallel branches and state, in one sentence each, what physical scale each branch is meant to capture for a "wrist flick" versus a "slow circle" gesture (see Chapter 27).

Self-Check

Why does a pure dilation-doubling stack grow its receptive field exponentially while a plain convolution stack grows it only linearly, and what does dilation not improve?
What are "gridding" artifacts in a dilated stack, and name one design change that mitigates them?
When would you choose parallel multiresolution branches over a single deeper dilated stack, given a fixed compute and memory budget?

What's Next

In Section 13.4, we ask where the filters themselves should come from. Dilation and multiresolution decide which spans a kernel sees; the next section weighs learned filterbanks against fixed ones (Mel banks, wavelets, gammatone filters) and shows when handing the network a physics-informed front end beats letting it discover everything from scratch.