Part II: Classical Signal Processing and Feature Engineering
Chapter 8: Feature Engineering and Dimensionality Reduction

Time-domain features

"My gait classifier scored 98 percent in the lab and collapsed in the field. It had learned the mean of the accelerometer, which was just how each volunteer happened to clip the sensor to their belt."

An Overfit AI Agent

Prerequisites. This section works on windowed sensor streams \(x[n] = x(nT_s)\) sampled at rate \(f_s = 1/T_s\), the discrete-time notation established in Chapter 3. It assumes the moving-average and windowing ideas of Chapter 6, and the basic estimators (mean, variance, quantiles) from the probability primer in Chapter 4. Summation and simple algebra are all the math you need; the frequency-domain counterparts of these features wait for Section 8.2. Detailed derivations of the statistical estimators live in Appendix A.

Why turn a window into a handful of numbers

A model rarely consumes raw samples. It consumes features: a fixed-length vector that summarizes a window of signal into quantities a classifier or regressor can compare across windows, subjects, and days. Time-domain features are the first and cheapest layer of that summary. They are computed directly from the samples with no transform, they cost a single pass over the window, and they run on a microcontroller with no floating-point unit. Before you reach for a spectrogram (Chapter 7) or a neural encoder (Chapter 13), you should know what a window's amplitude, spread, and shape already tell you. On many sensing problems, a dozen well-chosen time-domain features feed a shallow model that is smaller, faster, and more auditable than anything deeper. This section builds that vocabulary and, just as importantly, shows which of these features quietly leak identity or sensor placement instead of the thing you meant to measure.

Windowing: the unit of computation

What a time-domain feature summarizes is not the whole stream but a window: a contiguous block of \(N\) samples, typically 0.5 to 5 seconds long for human-scale motion, drawn with a fixed stride so consecutive windows overlap. Why windows exist is that most sensing tasks are locally stationary but globally not: a person is "walking" for a few seconds at a time, a bearing is "healthy" for hours, so the label lives at the window level, and the features must too. How you pick the window length is a genuine tradeoff. Too short and a feature is dominated by noise and cannot span a full gait cycle or breath; too long and a single window straddles two different activities, blurring the boundary the model must find. A common default for wearable motion is a 2 second window with 50 percent overlap, giving one feature vector every second. Overlap matters for a subtle reason we return to in Chapter 5: overlapping windows from the same recording are highly correlated, so if some land in training and their neighbors in test, your accuracy is inflated by leakage.

Amplitude and energy features

The simplest descriptors ask "how big is the signal in this window." The mean \(\mu = \frac{1}{N}\sum_n x[n]\) captures the DC level, which for an accelerometer axis encodes the projection of gravity, and therefore sensor orientation. The root-mean-square,

$$x_{\mathrm{rms}} = \sqrt{\frac{1}{N}\sum_{n=0}^{N-1} x[n]^2},$$

measures overall magnitude including any offset, and its square is proportional to signal energy or power. For a zero-mean vibration or EMG burst, RMS is the workhorse intensity feature: it rises with muscle force, with machine load, with impact severity. Closely related are the peak \(\max_n |x[n]|\), the peak-to-peak range \(\max_n x[n] - \min_n x[n]\), and the crest factor (peak divided by RMS), which is small for a smooth sinusoid and large for a spiky, impulsive signal. Crest factor is a classic early warning in rotating machinery: a healthy bearing hums smoothly (low crest factor) while a spalled one rings with sharp impacts on every revolution (rising crest factor) long before RMS energy itself climbs.

Absolute features encode the sensor, not just the phenomenon

The mean and the raw RMS are seductive because they are so informative, and that is exactly the danger. On a belt-worn accelerometer the per-axis mean is dominated by how the wearer clipped the device, and the RMS scales with each unit's gain calibration. A model that leans on these learns the volunteer and the hardware, then fails on a new person or a recalibrated sensor. The defenses are standard: compute magnitude \(\lVert \mathbf{a} \rVert = \sqrt{a_x^2 + a_y^2 + a_z^2}\) to cancel orientation, high-pass or mean-subtract to drop the DC term when the phenomenon is dynamic, and standardize features per subject or per device. Reach for absolute level only when you have controlled the sensor, as in a fixed industrial mount where the DC offset genuinely carries information.

Dispersion and distribution shape

Beyond size, the spread of the samples is diagnostic. The variance \(\sigma^2 = \frac{1}{N}\sum_n (x[n]-\mu)^2\) and its root, the standard deviation, measure how much the signal swings around its own level, and unlike RMS they are immune to a constant offset, which is often what you want. Two higher moments describe the shape of the amplitude distribution. Skewness measures asymmetry: a PPG pulse or an impact accelerometer has a fast rise and slow decay, so its sample distribution is skewed. Kurtosis measures tailedness, how peaky-and-heavy-tailed the distribution is relative to a Gaussian:

$$\mathrm{kurt} = \frac{\frac{1}{N}\sum_n (x[n]-\mu)^4}{\left(\frac{1}{N}\sum_n (x[n]-\mu)^2\right)^2}.$$

Kurtosis near 3 signals near-Gaussian, well-behaved noise; a value far above 3 signals rare, large excursions, the statistical fingerprint of impulsive faults. This is why kurtosis is a staple of vibration-based bearing diagnostics: it responds to the presence of sharp transients even when their total energy is small enough to leave RMS unmoved. Robust cousins, the median, the interquartile range, and quantiles, capture the same spread and shape while shrugging off the occasional saturated sample or dropout that would wreck a variance.

Temporal structure: the features raw statistics miss

Every feature so far ignores the order of the samples: shuffle the window and mean, RMS, variance, and kurtosis are unchanged. Yet order is where the signal dynamics live, and a second family of features reads it directly from the time axis. The zero-crossing rate, the number of times the signal changes sign per window, is a cheap proxy for dominant frequency and a mainstay of activity recognition and voice-activity detection. Mean absolute difference and the RMS of the first difference \(x[n]-x[n-1]\) measure how fast the signal moves, distinguishing a slow postural sway from a brisk shake at equal amplitude. Autocorrelation at lag \(\tau\),

$$r[\tau] = \sum_{n} (x[n]-\mu)\,(x[n+\tau]-\mu),$$

reveals periodicity: the lag of its first strong peak is the period, so on an accelerometer it recovers step cadence and on a PPG or ECG it recovers heart rate, entirely in the time domain and cheaper than an FFT for a single dominant period. Signal-magnitude area, the summed absolute value over the window, and simple peak counts round out a vocabulary that has driven human-activity classifiers for two decades. These order-aware features are the bridge to Chapter 23, where inertial motion signals are the whole subject.

A fall detector on a wrist wearable

A wearables team builds a fall detector for an elderly-care smartwatch running a 50 Hz accelerometer on a coin cell. A deep model is out: the budget is a few microjoules per inference. They window the magnitude signal at 2 seconds with 50 percent overlap and compute eight time-domain features per window. A real fall shows a signature the statistics catch cleanly: a brief free-fall dip (magnitude drops toward zero, so the window minimum plunges), then a sharp impact (a large peak and high crest factor and kurtosis), then unusual stillness (low post-impact variance). A vigorous hand wave, the classic false alarm, has high RMS but no free-fall dip and a modest crest factor, so the feature vector separates the two. Standardizing per device and using magnitude (not per-axis) keeps the detector working across watch models and however the user wears it. The whole feature extractor is a few dozen lines of integer-friendly C, and it runs continuously without waking the main processor. The same design pattern recurs for airbag triggers in Chapter 71.

Computing a feature vector in one pass

The code below turns a window into a labeled feature dictionary, so the earlier discussion becomes something you can run. It computes amplitude, dispersion, shape, and temporal-structure features together, and it deliberately keeps the raw mean available so you can see how large a per-window offset can be, and why you may want to drop it.

import numpy as np

def time_domain_features(x, fs):
    x = np.asarray(x, dtype=float)
    dx = np.diff(x)                       # first difference: how fast it moves
    mu, sd = x.mean(), x.std()
    z = (x - mu) / (sd + 1e-12)           # standardized, for stable moments
    feats = {
        "mean":        mu,                # DC level (often orientation/placement)
        "rms":         np.sqrt((x**2).mean()),
        "std":         sd,
        "ptp":         x.ptp(),           # peak-to-peak range
        "crest":       np.abs(x).max() / (np.sqrt((x**2).mean()) + 1e-12),
        "skew":        (z**3).mean(),
        "kurtosis":    (z**4).mean(),     # ~3 for Gaussian, high for impulsive
        "iqr":         np.subtract(*np.percentile(x, [75, 25])),
        "zcr":         np.mean(np.abs(np.diff(np.sign(x - mu)))) / 2 * fs,
        "mean_absdiff": np.abs(dx).mean(),
    }
    return feats

fs = 50
t  = np.arange(2 * fs) / fs
walk = 1.0 + 0.6*np.sin(2*np.pi*1.8*t)    # ~1.8 Hz cadence on a 1 g offset
print({k: round(v, 3) for k, v in time_domain_features(walk, fs).items()})
A single-pass time-domain feature extractor over one window. The synthetic "walk" signal is a 1.8 Hz cadence riding on a 1 g gravity offset; note how large mean is relative to std, which is exactly the placement-dependent term you would strip before training. zcr is scaled to crossings per second so it reads as an interpretable rate.

The printout makes the leakage argument concrete: the mean dwarfs the dynamic std, so a model handed the raw mean is largely reading the gravity projection, not the gait. The remaining features (crest factor, kurtosis, zero-crossing rate) describe the motion itself and travel far better across subjects.

Right tool: do not hand-roll a windowed feature matrix

The function above handles one window. A real pipeline must slide the window with overlap over long multi-channel recordings, batch the extraction, and name the columns, which is where hand-rolled code sprawls and off-by-one window bugs hide. Libraries collapse it:

import numpy as np
from numpy.lib.stride_tricks import sliding_window_view

W, hop = 100, 50                          # 2 s window, 50% overlap at 50 Hz
win = sliding_window_view(signal, W)[::hop]     # (n_windows, W)
feat_matrix = np.stack([np.column_stack([
    w.mean(1), w.std(1), np.sqrt((w**2).mean(1))]) for w in [win]])[0]
sliding_window_view replaces roughly 20 lines of manual index arithmetic and eliminates the classic last-partial-window bug. For the full catalog of hundreds of named time-domain features with tested edge-case handling, tsfresh and catch22 (Section 8.4) reduce this to a single call.

The vectorized window view turns a Python loop over slices into one strided array with no copy, and the dedicated feature libraries of Section 8.4 own the parts that are genuinely easy to get subtly wrong: consistent NaN handling, unbiased versus biased moment estimators, and reproducible column ordering.

Exercise

Take a 2 second, 50 Hz accelerometer-magnitude window and compare two candidate features for separating "walking" from "standing still": the raw mean and the standard deviation. (a) Which one will separate the two activities for a single fixed sensor placement, and which one will fail once you change where the device is worn, and why? (b) The zero-crossing rate of the raw magnitude signal is nearly useless here but becomes informative after one preprocessing step. What step, and what does the ZCR then estimate? (c) Propose one feature that would distinguish "walking" from "running" even though both have high standard deviation.

Self-check

1. Why are the mean, RMS, variance, and kurtosis all invariant to shuffling the samples in a window, and name one feature from this section that is not?

2. A vibration signal's RMS is unchanged but its kurtosis jumps from 3 to 9. What physically has likely happened, and why did RMS miss it?

3. You get 98 percent accuracy in the lab and 60 percent in the field on a wearable classifier that uses per-axis means as features. Give the most likely single cause and one fix.

What's Next

In Section 8.2, we cross from the time axis to the frequency axis, turning each window into spectral and time-frequency features: band powers, spectral centroid and bandwidth, dominant-frequency and its harmonics, and the spectrogram-derived descriptors that capture periodic structure a single autocorrelation lag cannot. Together with this section's amplitude, shape, and temporal-structure features, they form the raw material that the statistical and entropy features of Section 8.3 and the feature-selection methods of Section 8.5 then refine into a compact, leakage-safe representation.