"Every transform is a lens. The skill is not grinding a sharper lens; it is knowing which one to hold up to which signal."
A Discerning AI Agent
The big picture
The previous six sections handed you a toolbox: the DFT and its leakage in Section 7.1, the STFT and spectrogram in Section 7.2, wavelets in Section 7.3, the Hilbert-Huang transform in Section 7.4, cepstral and envelope analysis in Section 7.5, and power spectral density with coherence in Section 7.6. This closing section is not another transform. It is the meta-skill that decides which one to reach for, given a modality, a task, and a deployment budget. Picking a representation is the single highest-leverage decision in a classical pipeline: it sets the ceiling on accuracy before any model touches the data, it fixes your latency and memory cost, and it decides whether a downstream classifier sees the physics or fights it. This section gives you a small set of decision axes, a worked matching of common sensor modalities to representations, and an honest account of when to stop hand-picking a transform and let a learned front end do the choosing.
This section assumes you have met all six representations in this chapter and that you are comfortable with the sampling, aliasing, and stationarity vocabulary from Chapter 3. It also leans on the sensor-physics intuition from Chapter 2: the right transform is usually the one that matches the generative physics of the signal.
The four axes that actually decide the choice
Practitioners collect dozens of heuristics, but almost all of them project onto four axes. Naming them turns a vague "it depends" into a checklist you can answer in a minute.
Axis 1: stationarity. Is the signal's spectral content roughly constant over your analysis window, or does it drift within it? A steady 50 Hz power-line hum is stationary; a footstep, a heartbeat, or a car passing a microphone is not. Stationary signals are served well by a global spectrum (DFT, PSD). Nonstationary signals need a time-frequency view (STFT, wavelet, HHT). The formal test is whether the autocorrelation depends only on lag, but in practice you look at a spectrogram and ask whether the ridges move.
Axis 2: the shape of the information. Does the useful content live in narrowband tones, in broadband transients, in periodic impulse trains, or in slow envelopes? Tones favor high frequency resolution (long windows, PSD). Transients favor high time resolution and the multiresolution tiling of wavelets. Periodic impulse trains, the signature of gears and bearings, favor the cepstrum and envelope demodulation. This axis is where modality physics enters most directly.
Axis 3: the downstream consumer. A representation is an interface, not an end. A shallow classifier or a tree ensemble wants a compact, low-dimensional feature vector, so you summarize a spectrogram into band powers. A convolutional network wants a dense image-like map it can learn filters over, so you hand it the full log-mel spectrogram and let Chapter 13 take it from there. The same signal is best represented differently for different consumers.
Axis 4: the budget. Sample rate, window length, and transform choice set the compute and memory cost that must fit your platform. An FFT is \(O(N\log N)\); a continuous wavelet transform across many scales or an ensemble HHT is orders of magnitude heavier. On a microcontroller (Chapter 61) that gap decides feasibility, not elegance.
Key insight: match the transform to the generative physics
The representation that wins is almost always the one whose basis functions resemble the events that generate the signal. A bearing fault emits a train of sharp impacts modulated by a resonance; envelope analysis, whose implicit model is amplitude-modulated impulses, exposes it in one step while a raw FFT buries it under structural noise. Speech and most machine acoustics are sums of slowly varying tones, so a mel spectrogram (short-time sinusoids) fits. A seizure spike or a lidar return edge is a localized discontinuity, which is precisely what a wavelet is built to catch. When you find yourself fighting a representation, the usual cause is a physics mismatch: you are asking a sinusoid basis to describe an impulse, or an impulse basis to describe a drifting tone. Change the lens before you tune the model.
A modality-to-representation map
The four axes collapse into a short lookup table that covers most sensor work. Treat it as a strong prior to be validated, not a law. The right column names the default first choice and the reason, drawn from the modality's physics rather than fashion.
| Modality | Dominant structure | First-choice representation | Why |
|---|---|---|---|
| Machine vibration (bearings, gears) | Periodic impulse trains, resonances | Envelope spectrum + cepstrum | Demodulates repetitive impacts hidden under structural resonance |
| Audio / acoustic events | Slowly varying tones, harmonics | Log-mel spectrogram (STFT) | Perceptual bands, dense map for CNNs, cheap and standard |
| Inertial (IMU) for activity | Nonstationary, multi-band motion | Short-window STFT band powers or wavelets | Captures cadence and transients across scales at low cost |
| ECG / cardiac | Transient complexes on a slow baseline | Wavelet transform | Localizes QRS spikes while separating baseline wander |
| EEG / neural | Band-limited rhythms plus transients | STFT band powers (delta to gamma) | Clinically defined bands map directly to STFT bins |
| Rotating-shaft order tracking | Speed-varying harmonics | HHT or order-resampled spectrum | Follows instantaneous frequency as RPM changes |
| Structural / seismic coupling | Coupled stationary sources | PSD + coherence | Quantifies shared energy and transfer between channels |
Read the table above as a starting bet. The ECG row points forward to Chapter 29, where wavelet delineation of the QRS complex is standard; the vibration row is the backbone of predictive maintenance in Chapter 36. Two rows can be right at once: a bearing under variable speed often wants order tracking to stabilize the harmonics and then envelope analysis on the resampled signal.
Practical example: two teams, one gearbox
A wind-farm operator runs two condition-monitoring teams on the same 2 MW turbine gearbox. Team A, coming from an audio background, feeds raw accelerometer FFTs into a classifier and gets 71 percent fault-detection accuracy; the incipient bearing defects sit below the gear-mesh and structural peaks, invisible in a flat spectrum. Team B asks what generates the signal: a spalled bearing produces periodic impacts that ring the housing resonance near 4 kHz. They band-pass around that resonance, take the Hilbert envelope, and compute the envelope spectrum. The ball-pass frequency and its harmonics now stand up cleanly, and a simple threshold on their amplitude reaches 93 percent accuracy weeks earlier. No new sensor, no deeper model, no more data: the entire gain came from choosing the representation whose implicit model matched the fault physics. This is exactly the comparison Lab 7 asks you to reproduce.
A quick, quantitative pre-screen
You do not have to guess which axis dominates. A few cheap statistics computed on a handful of example windows will point you at the right family before you commit. Spectral flatness (the ratio of geometric to arithmetic mean of the power spectrum) separates tonal signals (flatness near 0) from noise-like ones (flatness near 1). Spectral kurtosis flags impulsive, transient bands that reward envelope analysis. A stationarity check compares spectra between the first and second half of a window. The helper below computes all three and prints a recommendation, referenced in the exercise.
import numpy as np
from scipy.signal import welch
def screen_representation(x, fs):
"""Cheap triage: suggest a representation family from three statistics."""
x = np.asarray(x, dtype=float)
x = x - x.mean()
# 1) Spectral flatness: tonal (->0) vs noise-like (->1)
f, pxx = welch(x, fs=fs, nperseg=min(1024, len(x)))
pxx = pxx + 1e-12
flatness = np.exp(np.mean(np.log(pxx))) / np.mean(pxx)
# 2) Spectral kurtosis proxy: impulsiveness across short frames
frame = 256
n = (len(x) // frame) * frame
frames = x[:n].reshape(-1, frame)
band_energy = (np.abs(np.fft.rfft(frames, axis=1)) ** 2).mean(axis=0)
sk = float(np.mean((band_energy / band_energy.mean() - 1) ** 4))
# 3) Nonstationarity: spectral distance between the two halves
half = len(x) // 2
_, p1 = welch(x[:half], fs=fs, nperseg=min(512, half))
_, p2 = welch(x[half:], fs=fs, nperseg=min(512, half))
drift = float(np.mean(np.abs(np.log(p1 + 1e-12) - np.log(p2 + 1e-12))))
if sk > 8:
rep = "envelope + cepstrum (impulsive/periodic-impact structure)"
elif drift > 1.0:
rep = "wavelet or STFT (nonstationary content)"
elif flatness < 0.2:
rep = "PSD / long-window FFT (stationary tonal content)"
else:
rep = "STFT spectrogram (broadband, mildly nonstationary)"
return {"flatness": round(float(flatness), 3),
"kurtosis": round(sk, 2),
"drift": round(drift, 3),
"suggested": rep}
if __name__ == "__main__":
fs = 20000
t = np.arange(0, 1, 1 / fs)
impacts = np.zeros_like(t)
impacts[::1800] = 1.0 # ~11 Hz impact train
ring = np.exp(-800 * (t % (1800 / fs))) * np.sin(2 * np.pi * 4000 * t)
bearing = impacts * ring + 0.05 * np.random.randn(t.size)
print(screen_representation(bearing, fs))
Library shortcut: librosa and scipy do the heavy lifting
Once the triage picks a family, the transform itself is rarely more than a call. A log-mel spectrogram that would be roughly 60 lines of framing, windowing, FFT, mel-filter construction, and log compression from scratch is librosa.feature.melspectrogram(...) followed by librosa.power_to_db(...), two lines, with the mel filterbank, centering, and padding handled for you. Welch PSD and coherence are single scipy.signal calls, and continuous wavelet transforms are one line in pywt or ssqueezepy. The value you add is choosing the representation and its parameters (window length, band edges, mel count), not reimplementing the FFT. Keep those parameters fixed across a dataset so the features stay comparable, echoing the leakage-safe discipline of Chapter 5.
When to stop choosing and start learning
Everything above assumes a human picks the lens. The running thread of this book is the classical-versus-learned tradeoff, and representation choice is where it first bites. A hand-picked transform embeds a strong, interpretable prior: it is data-efficient, cheap, auditable, and it degrades predictably. That is exactly what you want with small labeled sets, tight compute, regulatory scrutiny, or a well-understood physics (rotating machinery, ECG). Its ceiling is the quality of your prior. When you have abundant data and a task whose optimal features are not obvious, a learned front end, a 1D convolutional stack or a spectrogram-input network, can discover a representation that beats any fixed basis, at the cost of data hunger and opacity.
The two are not rivals so much as a spectrum, and the productive middle is common: feed a learned model a log-mel spectrogram rather than raw samples, and you keep the physics-matched prior while letting the network refine it. This hybrid is the default in modern audio and biosignal systems and is developed in Chapter 13 and, at scale, in the sensor foundation models of Chapter 20. A useful rule of thumb: start with the physics-matched classical representation, get an honest baseline, and only reach for a learned front end when that baseline plateaus and you have the data and compute to justify the move. The classical representation you chose here rarely goes to waste, because it becomes the input the learned model starts from.
Exercise
Take three signals: a steady 60 Hz tone with harmonics, a chirp sweeping 10 Hz to 200 Hz over one second, and the synthetic bearing signal from the code block. Run screen_representation on each and record the flatness, kurtosis, and drift scores. Do the recommendations match what you would pick by eye from a spectrogram? Now corrupt each with white noise at 5 dB SNR and rerun. Which statistic is most fragile to noise, and what does that tell you about trusting an automated triage without a human glance at the spectrogram?
Self-check
- Name the four decision axes and give one modality whose representation choice is driven mainly by each.
- Why does envelope analysis beat a raw FFT for an incipient bearing fault, in terms of the signal's generative physics?
- You have 300 labeled ECG windows and a regulatory audit ahead. Argue for a hand-picked wavelet representation over a learned front end, then state the single condition under which you would switch.
Lab 7
compare spectrogram, wavelet, and envelope features on machine-fault vibration data.
Bibliography
Foundational transforms and time-frequency analysis
The reference on window choice and leakage, which underlies every decision about frequency resolution versus sidelobe control when you pick a spectral representation.
Establishes multiresolution analysis, the formal reason wavelets suit transient-plus-baseline signals like ECG and seismic edges.
Introduces the data-driven alternative to fixed bases, the right lens when instantaneous frequency drifts within a cycle.
Modality-specific representation practice
The canonical case for envelope analysis and spectral kurtosis on bearing faults, the physics behind the wind-farm example in this section.
Origin of the mel scale that makes log-mel spectrograms the default acoustic representation for both classical and deep audio models.
Shows why wavelet delineation became standard for QRS detection, a concrete instance of matching the transform to transient physiology.
Tools and the classical-versus-learned boundary
The library that reduces spectrogram and feature extraction to a few lines, letting you spend effort on choosing rather than implementing a representation.
Supplies Welch PSD, coherence, and the FFT primitives behind the triage code, the workhorse backend for classical representation pipelines.
Demonstrates a learned front end operating on raw samples, the empirical marker for when a fixed spectral representation stops being the ceiling.
What's Next
In Chapter 8, we take the representation you have now learned to choose and turn it into features: compact, discriminative, leakage-safe descriptors that a classifier can actually use. You will see how the same spectrogram or envelope spectrum becomes a handful of band powers, shape statistics, and entropy measures, how automated libraries generate hundreds of candidates, and how to select the few that carry signal without overfitting.