"They told me the wheels were spinning backwards. The wheels were fine. My eyes were just sampling too slowly, and no amount of model capacity fixes a frame that was never captured."
A Chastened AI Agent
Prerequisites
This section assumes you know that a sensor produces a continuous voltage that an analog-to-digital converter turns into numbers, the measurement model \(x = h(s) + \eta\) developed in Chapter 2, and the idea that a signal carries energy at various frequencies. Nothing more. The frequency-domain machinery used lightly here (the Fourier view of a signal) is built out in full in Chapter 7, and the filters that enforce the rule we derive are the subject of Chapter 6. A short trigonometric-identity refresher lives in Appendix A.
Why this section is the hinge of the chapter
Every digital sensor makes the same silent decision millions of times per second: it looks at the world, writes down a number, and then closes its eyes until the next tick. What happens between the ticks is gone forever. This section establishes the one law that governs whether the numbers you kept are enough to reconstruct what you missed, or whether they are a convincing lie. The Nyquist-Shannon sampling theorem is not a piece of trivia; it is the reason a 100 Hz accelerometer cannot see a 1 kHz bearing fault, the reason a cheap camera turns helicopter blades into a slow crawl, and the reason no downstream model, however large, can recover information that sampling threw away. Get the sample rate wrong at acquisition and the rest of the book cannot save you.
We work with the notation from Chapter 1: the continuous signal at the transducer is \(x(t)\), and sampling produces the discrete sequence \(x[n] = x(nT_s)\), where \(T_s\) is the sampling period and \(f_s = 1/T_s\) is the sampling rate in samples per second (hertz). The whole section answers one question: given \(x[n]\), what can we truthfully claim about \(x(t)\)?
Sampling: turning a function of time into a list of numbers
What sampling does is deceptively simple. It multiplies the continuous signal by an infinite train of impulses spaced \(T_s\) apart and keeps only the values that land on the ticks. Why it is subtle is that this operation is not lossless in general, yet under a precise condition it loses nothing at all. How we know is the sampling theorem, and the fastest route to it runs through the frequency domain.
Multiplying a signal by an impulse train in the time domain convolves its spectrum with an impulse train in the frequency domain. The practical consequence is that sampling takes the original spectrum \(X(f)\) and copies it, endlessly, centered at every integer multiple of \(f_s\):
$$X_s(f) = f_s \sum_{k=-\infty}^{\infty} X(f - k f_s).$$Picture the true spectrum as a single hump of content occupying frequencies up to some highest frequency \(f_{\max}\). After sampling, that hump is stamped out at \(0, \pm f_s, \pm 2f_s,\) and so on. If the copies do not touch, you can isolate the original with an ideal low-pass filter and recover \(x(t)\) perfectly. If they overlap, the overlapping energy adds together and the original is unrecoverable. That overlap is aliasing, and avoiding it is the entire game.
The Nyquist limit: the one inequality to memorize
The copies of the spectrum stay clear of each other exactly when the sampling rate is more than twice the highest frequency present in the signal:
$$f_s > 2 f_{\max}.$$The quantity \(f_s/2\) is the Nyquist frequency: the highest frequency a given sample rate can represent without ambiguity. The threshold \(2 f_{\max}\) is the Nyquist rate: the slowest you are allowed to sample a signal band-limited to \(f_{\max}\). The Nyquist-Shannon sampling theorem states the payoff precisely: a signal containing no energy at or above \(f_s/2\) is completely determined by its samples, and can be reconstructed exactly by sinc interpolation (summing shifted, scaled \(\sin(\pi t)/(\pi t)\) kernels, one per sample). This is a remarkable claim. A continuous object with uncountably many values is pinned down, with zero error, by a countable list of numbers, provided it was band-limited first.
The insight worth carrying to every deployment
Bandwidth, not sample rate, is the real budget. Doubling your sample rate is worthless if the signal already fits below the old Nyquist frequency, and it is insufficient if content lives above it. The design question is never "is 1 kHz fast enough?" in the abstract; it is "what is the highest frequency I care about, and have I guaranteed nothing lives above it before the converter fires?" That guarantee comes from a physical anti-alias filter in front of the ADC, not from software, because once aliasing has folded two frequencies onto the same sample, no algorithm can tell them apart again.
Aliasing: what a violated inequality actually looks like
When \(f_s \le 2 f_{\max}\), a frequency \(f\) above the Nyquist limit does not simply disappear. It folds back and masquerades as a lower frequency. The impostor frequency is
$$f_{\text{alias}} = \lvert f - k f_s \rvert,$$where \(k\) is the integer that lands the result in \([0, f_s/2]\). A 90 Hz vibration sampled at 100 Hz reappears as a \(\lvert 90 - 100 \rvert = 10\) Hz vibration, indistinguishable from a genuine 10 Hz oscillation. The famous "wagon-wheel" effect in film is the same arithmetic: a wheel rotating just under the frame rate appears to creep backward because each frame catches it a little short of a full turn. The code below makes the fold concrete by sampling a pure 90 Hz tone at 100 Hz and showing that its samples coincide, value for value, with a 10 Hz tone.
import numpy as np
fs = 100.0 # sampling rate (Hz): Nyquist = 50 Hz
n = np.arange(20) # 20 sample indices
t = n / fs # the sampling instants
true_signal = np.cos(2 * np.pi * 90 * t) # 90 Hz: above Nyquist
alias = np.cos(2 * np.pi * 10 * t) # 10 Hz: the impostor
# The two continuous signals are completely different, yet on the grid...
print("max |difference| on the sample grid:",
np.max(np.abs(true_signal - alias))) # -> ~1e-14, i.e. identical
f_alias = abs(90 - round(90 / fs) * fs)
print(f"90 Hz sampled at {fs:.0f} Hz masquerades as {f_alias:.0f} Hz")Run it and the printed difference is around \(10^{-14}\). The two tones are physically distinct sounds, yet the sampler wrote down the same twenty numbers for both. This is why aliasing is dangerous rather than merely lossy: it does not blur the signal, it fabricates a plausible false one that looks like clean low-frequency data your model will trust.
The bearing fault the monitor could not see
An industrial team instrumented a gearbox on a conveyor line with an accelerometer logging at 200 Hz, reasoning that the shaft turned at only 25 Hz so 200 Hz was "eight times faster than anything interesting." For months the vibration trace looked healthy. Then a bearing failed catastrophically. The post-mortem found the smoking gun: the failing bearing's characteristic defect frequency sat near 1.3 kHz, and its harmonics folded down through the 100 Hz Nyquist limit into a lazy 40 Hz wobble that the team had dismissed as ordinary machine hum. The energy that screamed "replace me" was present at the sensor and destroyed at the converter. The fix cost almost nothing: an analog low-pass filter cutting above 90 Hz plus a second high-rate channel at 20 kHz for the bearing band. The lesson is the one this chapter keeps returning to, and it reappears when we build condition monitors in Chapter 37: you cannot detect what you never sampled fast enough to record.
Practical sampling: guard bands, oversampling, and when the rule bends
Real signals are never perfectly band-limited, and real anti-alias filters do not cut off vertically; they roll off over a finite transition band. So engineers do not sample at exactly \(2 f_{\max}\); they leave a guard band and oversample, commonly at 2.5 to 5 times the highest frequency of interest, so the filter has room to attenuate everything above Nyquist before it folds. This is why consumer audio, chasing a 20 kHz hearing limit, settled on 44.1 kHz rather than 40 kHz. When you cannot raise \(f_s\) (a battery-powered wearable streaming at 50 Hz, say), you must instead lower \(f_{\max}\) with an aggressive analog filter, accepting that you have chosen to be blind above 25 Hz rather than deceived by it. There is also a deliberate exception worth naming: bandpass sampling intentionally aliases a narrow high-frequency band down to baseband when you know the signal occupies only a thin slice of spectrum, a trick used heavily in the radar and software-defined-radio front ends of Chapter 44. Aliasing controlled is a feature; aliasing unnoticed is a bug.
Right tool: resampling without hand-rolling the anti-alias filter
Suppose you must downsample a 2 kHz stream to 100 Hz. Doing it correctly by hand means designing an FIR low-pass filter below the new 50 Hz Nyquist, convolving, then decimating: perhaps 25 to 40 lines to get the filter order, window, and edge handling right, and easy to get subtly wrong. SciPy folds the anti-alias filtering and the decimation into one call:
from scipy.signal import decimate
# Applies an anti-alias low-pass filter, THEN keeps every 20th sample.
x_100hz = decimate(x_2khz, q=20, ftype="fir") # 2000 Hz -> 100 Hz, safelyscipy.signal.decimate call replaces the roughly 30-line design-filter-then-drop-samples routine, and it will not let you skip the filter, which is the step people forget when they naively slice x[::20] and quietly alias their data.The library handles filter design, the delay compensation, and the decimation together. The dangerous alternative, plain array slicing x[::20], skips the filter entirely and folds every frequency above 50 Hz straight into your "downsampled" signal. The line-count saving is real, but the correctness saving is the point.
Why no downstream model can undo it
It is tempting to hope that a sufficiently powerful network could learn to invert aliasing. It cannot, and the reason is information-theoretic rather than a matter of capacity. Aliasing is a many-to-one map: infinitely many continuous signals produce the identical sample sequence, so the inverse is not a function. A model handed the aliased samples is choosing among indistinguishable pre-images, and its "reconstruction" is really a prior over which continuous signal it guesses you meant. That can be useful (it is what super-resolution does) but it is imputation, not recovery, and it will confidently hallucinate structure that was never measured. This is the sampling-stage instance of a theme that runs through the whole book: fidelity lost at acquisition sets a ceiling no later stage can raise. It is also why the state estimators of Chapter 9 model measurement noise but assume the samples themselves are honest observations of the band the sensor was designed to see.
Exercise 3.1
A drone's motor spins at 6,000 rpm and you want to monitor its vibration, whose dominant energy is at the shaft frequency and its first three harmonics. (a) Convert 6,000 rpm to hertz and identify the highest frequency of interest. (b) State the minimum sampling rate the Nyquist theorem requires. (c) Recommend a practical rate including a guard band, and specify the cutoff of the anti-alias filter you would place before the ADC. (d) If instead you were forced to sample at 300 Hz, compute the alias frequency the third harmonic would fold to, and explain in one sentence why that is worse than simply missing the harmonic.
Self-check
1. A colleague says "our signal has no important content above 40 Hz, so 80 Hz sampling is safe." What physical component must exist for this claim to hold, and what happens without it?
2. Why does raising the sample rate after the ADC (upsampling in software) never recover aliased content, while raising it before the ADC can?
3. A tone at 130 Hz is sampled at 100 Hz. What single frequency will it appear as in the samples?
What's Next
In Section 3.2, we question the assumption baked into this entire section: that samples arrive on a fixed clock at all. Event-driven sensors fire only when something changes, trading the tidy Nyquist arithmetic for a stream whose timing is the signal, and that shift rewrites what "sample rate" even means.