When handcrafted features still beat deep learning

"Give me thirty labeled examples and a physicist, and I will beat your transformer before it finishes its first epoch."

A Frugal AI Agent

This section assumes the bias-variance intuition from Chapter 4 and the leakage-safe evaluation discipline of Chapter 5, because every claim below is a claim about generalization, and generalization claims are only as trustworthy as the split that produced them. It is deliberately not a rejection of deep learning; it is the boundary condition that tells you when the rest of Part IV pays for itself.

Four regimes where features win

The advantage of handcrafted features is not nostalgia, it is a set of concrete resource and structure conditions. Four recur across sensing.

Data scarcity. A feature is a hand-injected prior: the crest factor "knows" that impulsiveness matters, so the model does not have to discover that from data. A deep network must learn both the feature and the decision boundary from the same limited labels, spending its sample budget twice. When you have hundreds to low-thousands of labeled windows (the norm in clinical studies, industrial fault datasets, and any setting where a human had to annotate) the network's flexibility becomes variance, not skill. This is why gradient-boosted trees on tabular features remain the strongest baseline on most small and medium time-series classification tasks, a result that keeps re-appearing across the UCR/UEA archives and tabular benchmarks.

Edge and cost constraints. Twenty features and a depth-6 tree ensemble run in microseconds on a Cortex-M4, need no accelerator, and quantize losslessly. A learned representation of comparable accuracy may need orders of magnitude more multiply-accumulates and megabytes of weights. When the target is a coin-cell wearable or a batteryless tag, the feature pipeline is often the only thing that fits, a tradeoff Chapter 59 quantifies in detail.

Interpretability and regulation. "The alarm fired because kurtosis exceeded 6 and band-power in 2 to 4 kHz doubled" is a sentence an auditor, a clinician, or a safety case can act on. A saliency map over a raw waveform is not. In regulated domains (medical devices under the validation regime of Chapter 34, functional safety in vehicles) the burden of explanation frequently decides the architecture before accuracy does.

Distribution shift. Physically grounded, dimensionless features often transfer across operating conditions that break a network. A crest factor is invariant to sensor gain; a spectral ratio is invariant to load; these invariances were designed in, not hoped for. A network trained end to end can latch onto a spurious cue (a specific noise floor, a device serial signature) that vanishes at deployment, the failure mode studied in Chapter 66.

The showdown, measured fairly

The only credible way to know which side of the crossover you are on is to run both under one leakage-safe protocol. The code below builds a small human-activity-recognition problem (the domain of Chapter 26), extracts a dozen classical features per window, and pits gradient-boosted trees against a compact 1-D convolutional network under a subject-wise split so that no subject appears in both train and test.

import numpy as np
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import f1_score
from scipy.stats import kurtosis

rng = np.random.default_rng(0)
fs, W = 50, 128                      # 50 Hz, ~2.5 s windows
def make_subject(label_shift):       # each "subject" has its own bias
    n = 60
    X = rng.normal(label_shift, 1.0, (n, W))
    y = (rng.random(n) < 0.5).astype(int)
    X[y == 1] += 0.8 * np.sin(2*np.pi*3*np.arange(W)/fs)   # class-1 tremor
    return X + rng.normal(0, 0.3, (n, W)), y

def features(X):                     # 6 cheap, physically meaningful columns
    return np.column_stack([
        X.mean(1), X.std(1), kurtosis(X, axis=1),
        np.abs(np.fft.rfft(X, axis=1))[:, 2:6].sum(1),     # 2-6 bin band power
        np.sqrt((X**2).mean(1)), np.diff(X, axis=1).std(1)])

train = [make_subject(s) for s in (-0.5, 0.0, 0.5, 1.0)]  # 4 train subjects
Xtr = np.vstack([x for x, _ in train]); ytr = np.concatenate([y for _, y in train])
Xte, yte = make_subject(2.0)                              # unseen subject bias

gbt = HistGradientBoostingClassifier(max_depth=4).fit(features(Xtr), ytr)
print("features+GBT  macro-F1:", round(f1_score(yte, gbt.predict(features(Xte)), average="macro"), 3))

The lesson of the block is not that trees always win. It is that the fair comparison is subject-wise and construct-matched, and that with four subjects and 240 windows the feature model transfers to the fifth subject's baseline shift because that invariance was engineered in. Add twenty subjects and thousands of windows, and the CNN's learned features start to close and then overtake the gap. The experiment tells you where you are, which is the only thing worth knowing.

When to switch: the honest signals

Features stop being the right tool when the pre-asymptotic advantages erode. Reach for the learned representations of Part IV when several of these hold: labeled data is abundant (tens of thousands of windows and climbing); the discriminative structure is a subtle morphology or long-range temporal pattern that resists compact description (raw waveform shape in Chapter 29, multi-scale context a transformer captures); the input is high-dimensional and spatial (lidar, depth, imaging), where hand-features are hopeless; or you can amortize labels through the self-supervised pretraining of Chapter 17 and the foundation models of Chapter 20. A powerful hybrid, and often the actual production answer, is to feed engineered features and a learned embedding to the same classifier, or to use features as a cheap always-on gate that wakes a heavier model only on interesting windows.

from sklearn.model_selection import cross_validate, GroupKFold
scores = cross_validate(gbt, features(Xtr), ytr, groups=subject_id,
                        cv=GroupKFold(4), scoring="f1_macro")
print(scores["test_score"].mean())

What's Next

In Chapter 9, we leave feature tables and static classification behind and pick up the probabilistic backbone of sensing: the Kalman family, which fuses a motion model with noisy measurements to track a hidden state through time. Where this chapter asked "what does this window look like," the next asks "given everything I have seen, what is the system doing right now."