Recursive Bayesian estimation

"I do not remember every measurement I have ever seen. I remember exactly one distribution, and I update it politely each time you speak."

A Well-Adjusted AI Agent

This section assumes you are comfortable with Bayes' rule, conditional probability, and the language of priors, likelihoods, and posteriors from Chapter 4, and with the hidden-state and observation-model notation introduced in Section 9.1. We stay representation-agnostic here: the closed-form Gaussian version arrives in Section 9.3, and the sample-based version in Chapter 10.

The belief, and why one number of it is enough

Write \(x_t\) for the hidden state at time \(t\) and \(z_{1:t} = \{z_1, \dots, z_t\}\) for every measurement seen so far. The quantity we actually want is the filtering distribution, also called the belief:

This is a full distribution, not a point estimate. It answers "where do I think the state is, and how sure am I?" in one object. A naive way to compute it would re-run inference over the whole growing history at every step, an amount of work that increases without bound as the sensor keeps streaming. Recursive estimation refuses to pay that bill. It exploits the Markov property of the state-space model: given \(x_{t-1}\), the next state \(x_t\) is independent of everything earlier, and given \(x_t\), the measurement \(z_t\) is independent of everything earlier. Under those two assumptions the previous belief \(\mathrm{bel}(x_{t-1})\) is a sufficient statistic for the entire past. You never need the raw history again. That single fact is what makes constant-time, constant-memory estimation on a live stream possible.

Predict: pushing the belief forward

The cycle begins by asking where the state will be before looking at the new measurement. We propagate the previous posterior through the transition model \(p(x_t \mid x_{t-1})\), summing over every place the state could have come from. This is the Chapman-Kolmogorov equation, the time update:

The bar denotes the predicted belief, our estimate after motion but before correction. Intuitively, prediction always spreads the distribution out: process noise and dynamic uncertainty add variance, so the predicted belief is broader and less confident than the posterior it came from. A tracker that has not received a measurement for a while grows steadily more uncertain, exactly as it should. When the transition is discrete the integral becomes a sum; when it is linear-Gaussian it becomes the matrix propagation of Section 9.3.

Correct: folding in the measurement

Now the sensor speaks. We sharpen the predicted belief using the new reading \(z_t\) and the observation model \(p(z_t \mid x_t)\), the likelihood. Bayes' rule does the folding:

Read it as a pointwise product: for each candidate state, multiply how plausible that state was after prediction by how well it explains the measurement, then renormalize with \(\eta\) so the result integrates to one. States the sensor rules out get suppressed; states consistent with the reading get reinforced. The measurement update always concentrates the belief, the mirror image of prediction. The whole filter is these two operations alternating forever: predict widens, correct narrows, and the balance between them is the estimator's entire personality. Tune it in Section 9.4.

The recursion, made concrete

The equations are easiest to trust once you watch them run on a distribution you can print. The classic teaching case is a discrete histogram filter: chop the state space into cells, store the belief as an array of probabilities, and implement predict and correct as array operations. Below, a robot lives on a 1-D loop of 20 cells, moves one cell per step with occasional slip, and carries a noisy sensor that reports "landmark" or "no landmark" at known positions.

import numpy as np

N = 20
landmarks = {3, 9, 14}                      # cells that emit "landmark"
bel = np.ones(N) / N                        # start fully uncertain

def predict(bel, p_slip=0.1):               # move +1 cell, sometimes slip
    moved   = np.roll(bel, 1)               # intended one-cell advance
    slipped = np.roll(bel, 2)               # over-shoot by an extra cell
    return (1 - p_slip) * moved + p_slip * slipped

def correct(bel, saw_landmark, hit=0.9, miss=0.1):
    like = np.array([hit if (c in landmarks) == saw_landmark else miss
                     for c in range(N)])
    post = like * bel                       # Bayes: likelihood x prior
    return post / post.sum()                # renormalize (this is eta)

measurements = [False, True, False, False, True]
for z in measurements:
    bel = predict(bel)                      # Chapman-Kolmogorov time update
    bel = correct(bel, z)                   # Bayes measurement update
    print(f"seen={z!s:5}  MAP cell={bel.argmax():2d}  "
          f"peak={bel.max():.2f}")

The array bel is the only state the loop keeps between iterations, the recursion in the flesh. Nothing about this code assumes a Gaussian, a linear model, or even a smooth state space, which is exactly why the histogram filter is the honest picture of what "recursive Bayesian estimation" means before any specialization narrows it.

When the belief has no closed form

The recursion is exact and universal, but the two integrals, the Chapman-Kolmogorov sum and the normalizer \(\eta\), are only tractable in special cases. If the transition and observation models are linear with Gaussian noise, every belief stays Gaussian and the integrals collapse to the finite matrix updates of the Kalman filter. If the state space is small and discrete, the histogram filter above computes them by brute force. Outside those cases, for nonlinear dynamics or non-Gaussian, multi-modal beliefs, the integrals have no analytic answer and the grid becomes exponentially expensive as dimensions grow. That is the entire motivation for Chapter 10: the extended and unscented Kalman filters linearize the models to keep the Gaussian machinery, while the particle filter represents the belief with a cloud of weighted samples and approximates both integrals by Monte Carlo. Each is a different answer to the same question posed here: how do I actually carry this belief forward when I cannot integrate it in closed form?

What's Next

In Section 9.3, we commit the abstract recursion to its most famous special case. Assume linear transition and observation models with Gaussian noise, and the belief stays Gaussian at every step, collapsing the two integrals of this section into a handful of matrix operations: the predict-correct cycle you just met, now with the Kalman gain deciding exactly how much to trust each new measurement.