Statistical, shape, and entropy features

"The mean tells me where the signal lives. The kurtosis tells me whether it is hiding a knife behind its back."

A Distribution-Minded AI Agent

This section leans on the probability and estimation background from Chapter 4: moments, quantiles, and the idea that a finite window is a noisy estimator of an underlying distribution. If those terms feel shaky, skim that chapter first. Everything here is computed on a fixed analysis window, the same windowing discipline you have used throughout the chapter.

Statistical moments: the distribution as a fingerprint

Treat the \(N\) samples in a window as draws from a distribution and summarize that distribution with its moments. The first two are familiar: the mean \(\mu\) and the variance \(\sigma^2\). The interesting discrimination usually lives in the standardized third and fourth moments. Skewness measures asymmetry, and kurtosis measures how heavy the tails are relative to a Gaussian:

Why do these matter for sensors? Because a great many fault and event signatures are impulsive: they leave the bulk of the signal unchanged but inject rare, large excursions. Kurtosis is exquisitely sensitive to exactly those rare large values, since the deviations are raised to the fourth power. A healthy rotating machine produces near-Gaussian vibration with kurtosis close to 3; the first pits on a bearing race add sharp periodic impacts that push kurtosis to 5, 8, or higher long before the RMS energy moves. Skewness catches asymmetry: a photoplethysmography pulse has a steep upstroke and a slow decay, so its skew is a stable shape marker used in the wearable cardiovascular work of Chapter 30. Beyond moments, distribution-free quantiles (median, interquartile range, the 5th and 95th percentiles) give robust location and spread that a single outlier cannot wreck, which is why they pair so well with the heavy-tailed moments.

Shape features: the geometry of the waveform

Shape features describe the form of the waveform or of its value histogram, often as dimensionless ratios that survive changes in sensor gain and units. The crest factor, the ratio of peak amplitude to RMS, quantifies how "peaky" a signal is: a pure sine sits at \(\sqrt{2}\approx 1.41\), while an impulsive fault signal climbs well above it. Related dimensionless ratios (the shape factor, impulse factor, and clearance factor) are staples of vibration diagnostics because they cancel out the operating amplitude and isolate waveform form:

Other shape descriptors work on the trace itself rather than the histogram: the peak-to-peak span, the number and prominence of local maxima, the total length of the curve (its arc length, a proxy for roughness), and the area under the rectified signal. For a signal with a natural morphology, the biosignal world leans hard on these: an electrocardiogram beat is described by the width, amplitude, and slope of its QRS complex, features that carry straight into the cardiac models of Chapter 29. The common thread is that shape features encode what a cycle looks like, independent of how loud it is, which is exactly the invariance a robust classifier wants.

Entropy and complexity: how predictable is the signal?

Entropy features answer a question the moments cannot: given the recent past, how surprised are you by the next sample? A perfectly periodic signal is maximally predictable and scores low entropy; broadband noise is maximally surprising and scores high; and, importantly, many physiological and mechanical systems lose complexity as they fail or age, landing somewhere in between. Several definitions matter in practice. Spectral entropy treats the normalized power spectrum as a probability distribution and computes its Shannon entropy, measuring whether energy is concentrated in a few tones (low entropy) or spread across the band (high entropy). Sample entropy and approximate entropy measure temporal regularity by asking how often short patterns of length \(m\) that match within a tolerance \(r\) still match when extended to length \(m+1\). Permutation entropy is the cheap, robust favorite: it looks at the ordinal pattern (the rank order) of each short window of samples and computes the Shannon entropy of how often each pattern appears.

For a length-\(m\) ordinal pattern \(\pi\) occurring with relative frequency \(p(\pi)\), permutation entropy is simply

normalized by \(\log(m!)\) to land in \([0,1]\). Because it depends only on the order of values and not their magnitudes, permutation entropy is nearly immune to sensor drift, slow baseline wander, and monotone gain changes, the failure modes that plague raw amplitude features. Entropy features are the workhorses of the anomaly and change detectors in Chapter 12, where a sudden drop in regularity is the first sign that a monitored system has left its normal regime.

import numpy as np
from scipy.stats import skew, kurtosis
from math import factorial, log
from itertools import permutations

def permutation_entropy(x, m=3, tau=1):
    """Normalized permutation entropy of a 1-D signal."""
    patterns = {}
    for i in range(len(x) - (m - 1) * tau):
        window = x[i:i + m * tau:tau]
        order = tuple(np.argsort(window))     # the ordinal pattern
        patterns[order] = patterns.get(order, 0) + 1
    counts = np.array(list(patterns.values()), dtype=float)
    p = counts / counts.sum()
    H = -(p * np.log(p)).sum()
    return H / log(factorial(m))              # in [0, 1]

fs = 2000
t = np.arange(0, 1.0, 1 / fs)
healthy = np.sin(2 * np.pi * 50 * t) + 0.05 * np.random.randn(t.size)
impacts = 3.0 * (np.random.rand(t.size) < 0.01)      # rare sharp spikes
faulty  = healthy + impacts

for name, sig in [("healthy", healthy), ("faulty", faulty)]:
    print(f"{name:8s} kurtosis={kurtosis(sig)+3:5.2f} "
          f"skew={skew(sig):+5.2f} Hperm={permutation_entropy(sig):.3f}")

The code above makes the division of labor concrete. Kurtosis fires on the impulsive spikes because of the fourth-power weighting; permutation entropy responds to the loss of clean periodic structure; and the two disagree in informative ways on other fault types, which is exactly why you compute all three rather than betting on one.

import antropy as ant
h_perm = ant.perm_entropy(faulty, normalize=True)
h_spec = ant.spectral_entropy(faulty, sf=fs, method="welch", normalize=True)
h_samp = ant.sample_entropy(faulty)

Computing them without self-deception

Three cautions keep these features from lying. First, moments and entropies are estimators: on a short window, kurtosis and sample entropy have large variance, so a single window can look alarming purely by chance. Aggregate over several windows or lengthen the window when the phenomenon allows. Second, scale sensitivity cuts both ways. Amplitude-based features (variance, peak-to-peak) demand that you fix calibration and units, while ordinal features (permutation entropy) and dimensionless ratios (crest factor) shrug off gain changes; choose deliberately based on whether your sensor's gain is trustworthy. Third, and most important for machine learning, these features are computed per window and then fed to a model, so any normalization statistics (feature means and standard deviations) must be fit on the training split only. Fitting a scaler on the whole dataset leaks test information and inflates your accuracy, exactly the trap the leakage-safe pipeline of Chapter 5 is built to prevent.

What's Next

In Section 8.4, we stop computing features one function at a time and reach for curated libraries. Tools like tsfresh and catch22 compute hundreds of statistical, shape, spectral, and entropy features in a single call, complete with automated relevance filtering, turning the hand-built columns of this section into an industrial feature factory.