Section 6.1: Moving averages and exponential smoothing

"I averaged the last hundred readings and got a beautifully smooth curve. It was also half a second late, and by then the robot had already hit the wall."
A Smoothed-Over AI Agent

Prerequisites

This section assumes the discrete-time notation from Chapter 3: a sensor stream is a sequence $x[n] = x(nT_s)$ sampled every $T_s$ seconds, so a "window of $N$ samples" spans $N T_s$ seconds of real time. It also uses the additive measurement model $x[n] = s[n] + \eta[n]$, where $s$ is the true quantity and $\eta$ is zero-mean noise, and the idea that averaging independent noise shrinks its variance, both developed in Chapter 4. No frequency-domain machinery is required yet; the transfer-function view that makes these filters rigorous is built in Section 6.2. A summation refresher lives in Appendix A.

Why start a filtering chapter with an average

Before you reach for an FIR design tool or a Kalman filter, you almost always reach for the two smoothers in this section. The moving average and the exponential smoother are the most-deployed filters on Earth: they run inside your step counter, your thermostat, your car's fuel gauge, and nearly every microcontroller that has ever debounced a noisy pin. They are worth a section of their own for three reasons. They are the simplest possible answer to "how do I make this jittery signal less jittery," so they set the baseline every fancier method must beat. They expose, in the cleanest possible form, the one tradeoff that governs all of filtering: you buy smoothness with delay. And the exponential smoother, in particular, turns out to be a one-line, one-variable recursion that fits on the tiniest chip, yet it is the direct ancestor of the Kalman filter of Chapter 9. Master these two and you understand the skeleton of the whole chapter.

The simple moving average: averaging a sliding window

What a simple moving average (SMA) does is replace each sample with the mean of itself and its $N-1$ predecessors. Why it works is the noise-averaging fact from Chapter 4: if the underlying signal $s$ is roughly constant across the window and the noise samples are independent with variance $\sigma^2$, the average has variance $\sigma^2/N$, so the noise standard deviation drops by $\sqrt{N}$. How we write it, for a causal (past-only) window of length $N$, is

$$y[n] = \frac{1}{N}\sum_{k=0}^{N-1} x[n-k].$$

This is a convolution with a rectangular kernel of height $1/N$, which is why the SMA is also called a boxcar filter. Two properties matter for deployment. First, every sample in the window gets equal weight, so a spike that entered the window six samples ago still counts exactly as much as the newest reading, and it keeps counting until it falls off the far edge. Second, the filter is not free: because the output is the mean of samples centered, on average, $(N-1)/2$ steps in the past, a causal SMA delays the signal by roughly

$$\text{group delay} \approx \frac{N-1}{2}\, T_s \text{ seconds.}$$

A 50-sample average of a 100 Hz stream is smoother by a factor of about seven, but it also lags the truth by a quarter of a second. That number, not the smoothness, is what ends up mattering in a control loop.

Smoothing and delay are the same knob

There is exactly one dial on a moving average, the window length $N$, and it moves noise reduction and lag in lockstep: noise falls as $\sqrt{N}$ while delay grows as $N/2$. You cannot turn one without turning the other. Every "better" filter in this chapter is, at heart, a cleverer way to spend a fixed delay budget on more noise reduction, or a fixed noise budget on less delay. This tension is important enough that Section 6.7 is devoted entirely to quantifying it. When someone shows you a magically smooth sensor trace, your first question should be "how late is it?"

The running-sum trick and weighted variants

Computed naively, the SMA re-adds $N$ numbers every step, which is wasteful when $N$ is large. Because consecutive windows differ by only two samples, you can maintain a running sum and update it in constant time: add the arriving sample, subtract the one leaving the window. That makes the SMA an $O(1)$-per-sample filter regardless of window size, which is why it survives on hardware that streams thousands of samples per second.

The equal-weight kernel is not always what you want. If recent samples are more trustworthy, you can taper the weights: a triangular moving average weights the center of the window most (equivalent to averaging twice), and a linearly weighted average ramps the weights up toward the newest sample to cut the delay. All of these are still finite-window convolutions, the FIR filters that Section 6.2 treats systematically. The moment you start choosing weights deliberately to shape which frequencies pass, you have stopped hand-waving and started filter design.

Exponential smoothing: infinite memory in one number

The exponential moving average (EMA), also called exponentially weighted moving average or first-order exponential smoothing, throws away the window entirely. Instead of storing the last $N$ samples, it keeps a single running estimate and nudges it toward each new reading:

$$y[n] = \alpha\, x[n] + (1-\alpha)\, y[n-1], \qquad 0 < \alpha \le 1.$$

What this does is blend the new measurement (weight $\alpha$) with everything seen before (weight $1-\alpha$). Why it is remarkable is that unrolling the recursion shows every past sample still contributes, but with a weight that decays geometrically: sample $x[n-k]$ enters with weight $\alpha(1-\alpha)^k$. The filter has infinite memory yet stores exactly one number. How you choose $\alpha$ is the whole design problem. A small $\alpha$ (say 0.02) trusts history and smooths hard but reacts slowly; an $\alpha$ near 1 barely filters at all. The knob has an intuitive time interpretation: the smoother's effective memory, its time constant, is about

$$\tau \approx \frac{T_s}{\alpha} \quad\text{seconds,} \qquad\text{equivalently an SMA of about}\quad N \approx \frac{2-\alpha}{\alpha} \ \text{samples.}$$

So $\alpha = 0.1$ behaves roughly like a 19-sample moving average, but with a gentle exponential tail instead of a hard cutoff, and, crucially, with $O(1)$ memory rather than $O(N)$.

import numpy as np

def ema(x, alpha):
    """First-order exponential smoother. O(1) memory, one pass."""
    y = np.empty_like(x, dtype=float)
    y[0] = x[0]                       # seed with the first reading
    for n in range(1, len(x)):
        y[n] = alpha * x[n] + (1 - alpha) * y[n - 1]
    return y

rng = np.random.default_rng(0)
t = np.arange(500)
truth = np.sin(2 * np.pi * t / 200)          # slow underlying signal
noisy = truth + rng.normal(0, 0.4, t.size)   # heavy sensor noise

fast = ema(noisy, alpha=0.30)   # ~5-sample memory: responsive, still jittery
slow = ema(noisy, alpha=0.05)   # ~39-sample memory: smooth, but lags the sine

# Noise reduction vs lag, made numerical:
print("input noise std :", round(noisy.std() - truth.std(), 3))
print("residual (fast) :", round((fast - truth).std(), 3))
print("residual (slow) :", round((slow - truth).std(), 3))

A four-line exponential smoother applied at two settings. The alpha=0.05 curve is visibly cleaner than alpha=0.30, but printing the residuals against the known truth also reveals its cost: near the sine's peaks and troughs the slow smoother trails behind, because its long memory is still remembering the climb after the signal has turned. This is the smoothing-versus-delay tradeoff of the earlier callout, now measurable.

The code above makes the two settings concrete: alpha=0.05 produces the prettier trace, yet its residual is worst exactly where the signal moves fastest, because a long memory lags a turning signal. That lag is the same phenomenon as the SMA's group delay, wearing different clothes.

The wrist heart-rate monitor that had to forget quickly

A wearables team building an optical (PPG) heart-rate feature, the kind revisited in Chapter 30, smoothed the beat-to-beat rate with a heavy exponential filter ($\alpha \approx 0.03$) to kill motion jitter. On the bench, resting on a desk, the number was rock steady and looked premium. In the field it failed a simple test: a user sprinting for a bus saw their displayed heart rate crawl from 70 toward 150 over a full fifteen seconds, long after they were already winded. The time constant $\tau \approx T_s/\alpha$ had been tuned for cosmetics, not for physiology. The fix was an adaptive $\alpha$: keep the smoothing heavy while the accelerometer reports the wrist is still, and open $\alpha$ up the instant motion energy spikes, so the estimate can chase a real change. Same one-line recursion, one context-dependent knob, and the displayed rate finally tracked the body instead of flattering it. The general principle, letting the data decide how much to trust the newest measurement, is precisely what the Kalman filter formalizes.

When a plain smoother is not enough

Both filters share a blind spot: they assume the signal is locally flat, so on a signal with a steady trend (a slowly draining battery, a warming engine) they lag persistently, always reporting a value biased toward the recent past. Double exponential smoothing (Holt's method) fixes this by smoothing the level and the slope with two coupled recursions, so it can follow a ramp without a standing error. That is the same conceptual move, tracking a value and its rate of change, that produces the constant-velocity Kalman filter in Chapter 9. And because these smoothers treat a lone outlier as real data, worth $1/N$ of the window or $\alpha$ of the update, they smear spikes rather than rejecting them; the median-based robust filters of Section 6.5 exist for exactly that failure. Reach for a moving average or EMA first; reach past them when trend or outliers break their flat-and-clean assumption.

Right tool: skip the hand-rolled recursion

The teaching loop above is fine for one array, but production code should not re-implement smoothing, seeding, and edge handling by hand. Pandas expresses both filters as one-liners over any Series or grouped stream:

import pandas as pd
s = pd.Series(noisy)
sma = s.rolling(window=20, min_periods=1).mean()   # simple moving average
ema = s.ewm(alpha=0.05, adjust=False).mean()        # exponential smoother

Two calls replace roughly 15 to 20 lines of windowing, running-sum bookkeeping, seed handling, and boundary logic. rolling().mean() uses the constant-time running-sum internally, and ewm(adjust=False) is the exact recursion $y[n]=\alpha x[n]+(1-\alpha)y[n-1]$.

The library handles the constant-time running sum, the startup transient when the window is not yet full (min_periods), and NaN gaps from dropped samples, all of which are easy to get subtly wrong by hand. For an offline pass over a whole logged dataset this is the right tool; only when you are on a microcontroller with no pandas, streaming one sample at a time, do you write the four-line EMA yourself, and there its $O(1)$ memory is the reason it wins.

Exercise 6.1

A temperature sensor streams at $f_s = 20$ Hz and you want a smoother whose noise reduction matches a 40-sample moving average, but with $O(1)$ memory. (a) Estimate the $\alpha$ that gives an equivalent memory, using $N \approx (2-\alpha)/\alpha$. (b) Compute the resulting time constant $\tau \approx T_s/\alpha$ in seconds. (c) The process you are monitoring can ramp at up to $2^{\circ}$C per second; estimate the standing lag error your EMA will show during such a ramp, and state in one sentence why double exponential smoothing would remove it. (d) The 40-sample SMA has group delay $(N-1)/2 \cdot T_s$; compare it to your $\tau$ and comment on which filter feels "faster" to a step change.

Self-check

1. An exponential smoother stores a single number yet "remembers" every past sample. In what precise sense is that true, and how fast does an old sample's influence decay?

2. You double a moving average's window from $N$ to $2N$. By roughly what factor does the output noise drop, and by roughly what factor does the delay grow?

3. Why does a plain SMA or EMA report a persistently biased value on a signal that is steadily rising, and which named method fixes it?

What's Next

In Section 6.2, we stop treating these smoothers as ad-hoc recipes and name them for what they are: the moving average is a finite impulse response (FIR) filter, the exponential smoother is the simplest infinite impulse response (IIR) filter, and the difference between them, finite versus infinite memory, is the fault line that organizes all classical filter design.