Smoothing vs filtering

"I gave the analyst the same track the autopilot flew, and she found the wobble I never felt. She had one advantage I did not: she already knew how the flight ended."

A Hindsight-Rich AI Agent

Three questions, three information sets

What separates filtering from smoothing is purely which measurements the estimate is allowed to see. Filtering computes the posterior \(p(x_t \mid y_{1:t})\), conditioned on the past and present. Smoothing computes \(p(x_t \mid y_{1:T})\) for some \(T > t\), conditioned on future measurements as well. There is also a third relative, prediction, \(p(x_{t+k}\mid y_{1:t})\), which conditions on strictly less than filtering and is what a filter does internally on every time-update step.

Why the extra conditioning helps is that a future measurement is an indirect observation of a past state. If the tracker sees an object at position \(y_{t+3}\), that reading constrains where \(x_t\) could plausibly have been three steps earlier, because the dynamics couple them. Bayes' rule lets that later evidence flow backward. The result is never worse and usually strictly better: the smoothed covariance satisfies \(P_{t\mid T} \preceq P_{t\mid t}\) in the positive-semidefinite ordering, so every state's uncertainty shrinks or stays equal once the future is folded in.

How smoothing is deployed depends on how much future you are willing to wait for, which gives three named variants:

• Fixed-interval smoothing: you have the entire batch \(y_{1:T}\) and want the best estimate of every \(x_t\) for \(t = 1,\dots,T\). This is the offline, log-analysis case.
• Fixed-lag smoothing: you run online but deliberately delay the output by \(L\) steps, always reporting \(p(x_{t-L}\mid y_{1:t})\). You trade \(L\) samples of latency for accuracy on a live stream.
• Fixed-point smoothing: you care about one particular instant (the moment of an impact, a trigger, an event onset) and keep refining that single estimate as more data arrives.

The RTS smoother: one backward pass over the filter

The elegant fact that makes smoothing practical is that fixed-interval smoothing needs no new heavy machinery. You run the ordinary Kalman filter forward once, storing the filtered means \(\hat x_{t\mid t}\), the predicted means \(\hat x_{t+1\mid t}\), and both covariances at every step. Then a single backward sweep, the Rauch-Tung-Striebel (RTS) recursion, corrects each state using the smoothed estimate of the one after it. Starting from \(\hat x_{T\mid T}\) (the filter's last output already equals the smoothed value there) and running \(t = T-1\) down to \(1\):

The smoother gain \(C_t\) weighs how much the future correction, the bracketed difference between the smoothed and predicted next state, should be pulled back onto \(x_t\). Where the dynamics are confident (small predicted covariance), little is changed; where the filter was uncertain, the future reshapes the estimate substantially. The whole backward pass is linear in \(T\) and touches only quantities the forward filter already produced, so smoothing a batch costs about twice a single filtering run.

import numpy as np

def rts_smoother(xf, Pf, xp, Pp, F):
    """Rauch-Tung-Striebel fixed-interval smoother.
    xf, Pf: filtered means/covariances x_{t|t}, P_{t|t}   (length T)
    xp, Pp: predicted means/covariances x_{t+1|t}, P_{t+1|t} (length T)
    Returns smoothed means xs and covariances Ps."""
    T = len(xf)
    xs, Ps = xf.copy(), Pf.copy()
    for t in range(T - 2, -1, -1):
        C = Pf[t] @ F.T @ np.linalg.inv(Pp[t])     # smoother gain
        xs[t] = xf[t] + C @ (xs[t + 1] - xp[t])     # correct with the future
        Ps[t] = Pf[t] + C @ (Ps[t + 1] - Pp[t]) @ C.T
    return xs, Ps

Choosing filtering, fixed-lag, or full smoothing

When to smooth is a systems decision, not a statistical one, because the accuracy gain is always real but is only usable if you can afford the latency. Filter when an actuator, alarm, or control law consumes the estimate immediately: an anti-lock braking controller or a balancing robot cannot wait for the future, so the filter is the only honest choice. Use fixed-lag smoothing when the consumer tolerates a bounded delay, such as a gesture recognizer that can hold its decision for 100 milliseconds, or a lidar tracker feeding a planner that already looks a few frames ahead; pick \(L\) around the state's autocorrelation time and stop, because further lag buys almost nothing. Use full fixed-interval smoothing for any offline product: flight-data forensics, sleep-stage scoring from an overnight recording, sensor-fusion ground truth, or building the clean labels that a downstream learner will train on. That last use is why smoothing recurs throughout this book: the smoothed trajectory is frequently the best available reference signal, and the batch-smoothing view generalizes into the factor-graph formulation of Chapter 11, where all timestamps are optimized jointly rather than in a forward-then-backward sweep.

One caution carries over from Section 9.5: smoothing sharpens estimates only when the model is right. If \(Q\) and \(R\) are mistuned or the dynamics are wrong, the backward pass will confidently propagate that error across the whole batch, producing a trajectory that looks beautifully consistent and is uniformly biased. Smoothing narrows the reported uncertainty, which makes a miscalibrated model more dangerous, not less. Validate the filter first; only a trustworthy forward pass earns a backward one.

What's Next

In Section 9.7, we close the chapter by turning the estimator on itself: the failure modes that a smoothed, confident-looking trajectory can hide, and the online diagnostics (innovation whiteness, normalized estimation-error squared, covariance-consistency tests) that tell you whether the filter you just built deserves to be trusted before you ever run a backward pass over it.