"They gave me a hundred filters and let me choose the passbands. I chose badly for the first ten thousand steps, then I chose exactly what the textbook already knew."
A Chastened AI Agent
Why this choice sits at the front of every sensor network
The very first layer of a convolutional network for sensor streams is a filterbank: a set of short filters that each respond to a band of frequencies and slide along the signal. You can hand that filterbank to the network as a fixed, hand-designed object (a mel bank, a wavelet family, a Gabor set) or you can let gradient descent invent the filters from data. This is not a cosmetic decision. It sets how much data you need, whether a regulator can read the front end, how robust the model is when the hardware drifts, and how many multiply-accumulates the first layer costs on a microcontroller. This section makes the tradeoff precise and hands you a decision rule, plus the middle ground almost everyone actually ships.
Section 13.2 established that a 1D convolution is a bank of learned FIR filters, and Section 13.3 stacked them across scales. This section asks the question those two deferred: should the first bank be learned at all? The alternative is a filterbank you inherit from classical time-frequency analysis, the machinery of Chapter 7. Prerequisites: the short-time Fourier transform and wavelets from Chapter 7, the convolution-as-filter view from Section 13.2, and the notion of hand-crafted spectral features from Chapter 8. If any are unfamiliar, read those first; this section sits on top of them.
What a filterbank is, and the two ways to get one
A filterbank is a collection of \(K\) filters \(\{g_k[n]\}\), each tuned to a different frequency band, applied to the same input so the signal is decomposed into \(K\) subband streams. In a fixed bank the coefficients come from a formula: a mel filterbank spaces triangular windows on a perceptual frequency axis, a wavelet bank dilates one mother wavelet, a Gabor bank multiplies sinusoids by Gaussians. In a learned bank the coefficients are the weights of the first convolution layer, initialized at random and moved by backpropagation to minimize the task loss. Mechanically both produce the same object, a set of FIR kernels convolved with the signal; the only difference is where the numbers come from, a designer's equation or the gradient.
That single difference propagates into everything downstream. A fixed bank encodes a strong inductive bias: the model is told, before it sees any data, that frequency content organized into these particular bands is what matters. A learned bank encodes almost none: it must discover from labels alone which bands carry signal, which is powerful when the useful bands are unusual and wasteful when they are exactly the ones a physicist could have named.
The fixed filterbank: physics and interpretability for free
A fixed bank buys three things that are hard to get any other way. First, data efficiency: because the filters are already correct, no training samples are spent learning them, so the network can be small and converge on a few hundred labeled windows rather than a few hundred thousand. Second, interpretability: every channel has a known center frequency and bandwidth, so a clinician or a safety auditor can read "this feature is energy in the 8 to 12 Hz band" instead of squinting at an opaque kernel. Third, robustness to leakage and drift: a data-independent front end cannot overfit an idiosyncrasy of one recording session, a real hazard we treat in Chapter 5, and it degrades gracefully when the hardware ages because its bands do not depend on any particular sensor's quirks.
The cost is rigidity. If the discriminative information lives between the bands the designer chose, or in a phase relationship a magnitude filterbank discards, a fixed bank simply cannot represent it, and no amount of downstream capacity fully recovers what the front end threw away. Fixed banks also carry human priors that may be wrong for a machine: the mel scale was tuned to human hearing, which has nothing to say about the resonance of a failing gearbox.
Fixed banks trade capacity for prior knowledge
A fixed filterbank is a claim about where the information is, made before training. When the claim is right, you get accuracy for almost no data. When it is wrong, you have hard-coded a bottleneck the rest of the network cannot undo. The learned bank makes the opposite bet: no prior, but the capacity to be right about bands no human would have guessed, paid for in labeled data.
The learned filterbank: the first conv layer as an adaptive analyzer
Letting the first layer learn its filters means the network can discover task-specific bands that no standard bank contains: the exact sideband a specific bearing defect excites, the narrow rhythm a particular gesture produces, a cross-channel pattern that a per-channel spectral transform never sees. With enough labeled data this consistently matches or beats a fixed bank, and it removes the guesswork of picking a wavelet family. The price is everything the fixed bank gave away: more data, more compute, weaker interpretability, and a front end that can quietly overfit the calibration of the sensors in your training set.
A useful habit is to inspect what the learned bank became, rather than trust it blindly. Because each first-layer kernel is an FIR filter, its magnitude response is just the magnitude of its discrete Fourier transform. The snippet below builds a small learnable-style bank and reads off its passbands, the same lens we sharpen for debugging in Section 13.7.
import numpy as np
fs, K, L = 200.0, 4, 65 # 200 Hz signal, K filters, length-L kernels
def bandpass_sinc(f_lo, f_hi, L, fs):
n = np.arange(L) - (L - 1) / 2
def sinc_lp(fc): # ideal low-pass, cutoff fc (Hz)
return 2 * fc / fs * np.sinc(2 * fc / fs * n)
h = sinc_lp(f_hi) - sinc_lp(f_lo) # band-pass = difference of low-passes
return h * np.hamming(L) # window to tame ripple
# a constrained, "learnable" bank: 2 numbers per filter (its two cutoffs)
cutoffs = [(4, 8), (8, 12), (12, 20), (20, 40)]
bank = np.stack([bandpass_sinc(lo, hi, L, fs) for lo, hi in cutoffs])
freqs = np.fft.rfftfreq(256, d=1/fs)
resp = np.abs(np.fft.rfft(bank, n=256, axis=1))
peak_hz = freqs[resp.argmax(axis=1)]
print("free-bank params :", K * L) # every tap is a weight
print("sinc-bank params :", K * 2) # only two cutoffs per filter
print("peak response Hz :", np.round(peak_hz, 1))print lines contrast a free learned bank (every one of the \(K\times L\) taps is a trainable weight) against a constrained sinc bank (only two cutoff numbers per filter), the parameter gap that motivates the next section.Running it confirms the four channels peak near 6, 10, 16, and 30 Hz, and that the free bank carries \(4\times65=260\) parameters against the sinc bank's \(4\times2=8\). That 30-fold gap is the whole argument for constraining the learning.
The middle ground: constrained learnable filterbanks
The dichotomy is false in practice. The dominant modern pattern is a parameterized filterbank: still learned by gradient descent, but constrained to a shape that guarantees it stays a sensible filter. SincNet (Ravanelli and Bengio, 2018) learns only the two cutoff frequencies of each bandpass sinc filter, so a 65-tap filter has 2 trainable numbers instead of 65. LEAF (Zeghidour et al., 2021) learns Gabor filter centers and bandwidths plus a learnable pooling and normalization. Both keep the interpretability of a fixed bank (you can still print a center frequency) while letting the data nudge the bands, and both need far less data than a free bank because the hypothesis space is tiny.
A second, even cheaper middle ground is initialization: start a free conv layer from a mel, gammatone, or wavelet bank instead of random noise, then fine-tune. The network begins already competent and only spends gradient on the corrections that actually help, which combines much of the data efficiency of the fixed bank with the ceiling of the learned one.
A trainable filterbank in one layer, not forty
Hand-rolling a fixed mel bank, wiring it into a differentiable front end, and adding learnable cutoffs is easily 40+ lines of window math and buffer bookkeeping. Modern libraries expose it as a single module. A fixed differentiable mel front end is torchaudio.transforms.MelSpectrogram(sample_rate=fs, n_mels=K); a fully trainable STFT/Chroma front end is one nnAudio.features.STFT(trainable=True); a SincNet bank is speechbrain.nnet.CNN.SincConv(out_channels=K, kernel_size=L, sample_rate=fs). Each replaces roughly 40 lines with 1, and the library handles windowing, framing, edge padding, and gradient flow through the transform.
Bearing faults: where a learned band beat the mel scale
A predictive-maintenance team monitoring rolling-element bearings on factory motors (the subject of Chapter 36) started with a mel filterbank on 20 kHz vibration, because it was the default in their audio toolkit. It plateaued: the mel scale packs resolution into low frequencies, but the diagnostic signature of an outer-race fault is a train of high-frequency resonant bursts modulated at a defect frequency the mel bands smeared together. Swapping to a SincNet front end, initialized from a linear-frequency bank, let the filters migrate to the resonance and sharpen there. Accuracy rose several points, and because each learned filter still reported a center frequency, the maintenance engineers could see the network had rediscovered the bearing's known resonant band, which turned a black box into something their reliability team would sign off on. A fully free bank reached similar accuracy but needed roughly ten times the labeled fault examples, which for a rarely-failing machine they simply did not have.
A decision rule you can apply today
Little labeled data, or a regulator who must read the front end, or a microcontroller counting every multiply (Chapter 59): use a fixed bank or a constrained learnable one (SincNet, LEAF). Abundant labeled data and a task whose useful bands are unusual or cross-channel: use a free learned bank, but initialize it from a fixed bank and inspect the result. When in doubt, the constrained learnable bank is the low-regret default: it rarely loses to either extreme by much and often wins.
Exercise: make the data-efficiency gap visible
Take a public accelerometer activity-recognition set (for example UCI HAR at 50 Hz). Train three identical small 1D CNNs that differ only in the first layer: (a) a fixed mel/linear bandpass bank with frozen weights, (b) a SincNet-style bank learning only per-filter cutoffs, (c) a free conv layer learned from random init. Sweep the training-set size from 5 percent up to 100 percent and plot validation accuracy against data budget for all three, using a subject-disjoint split so there is no leakage across people. Confirm that (a) and (b) dominate in the low-data regime while (c) catches up only once data is plentiful, and report the crossover point.
Self-check
- In what precise sense is the first convolution layer of a sensor CNN already a filterbank, and what is the only thing that distinguishes a "learned" bank from a "fixed" one?
- Name two concrete advantages a fixed mel bank has over a free learned bank, and one situation where each advantage becomes a liability.
- A SincNet filter of length 65 has how many trainable parameters, and why does that small number simultaneously improve data efficiency and preserve interpretability?
What's Next
In Section 13.5, we turn from the filters to the samples that flow through them: how to normalize sensor windows so a drifting baseline does not swamp the bands your filterbank just learned, and how to augment them so the network generalizes across mounting angles, users, and devices without ever seeing the future.