Hidden state and observation models

"They asked me for the drone's velocity. My sensor measures acceleration and a noisy GPS fix. I confidently reported a number I had never once observed, and it was the right thing to do."

A State-Estimating AI Agent

State and observation: the two things that are never the same

What we mean by state is a vector \(\mathbf{x}_k\) that is sufficient: it packs everything about the system's past that matters for predicting its future. If you know \(\mathbf{x}_k\), no earlier history tells you anything more. For a moving object the natural state is position and velocity, \(\mathbf{x}_k = [p_x, p_y, v_x, v_y]^\top\), because Newton says tomorrow's position depends on today's position and velocity, not on where the object was last week. The state is hidden (also called latent): velocity, orientation, battery internal resistance, and lung compliance are all things you reason about but never read off a wire.

What we mean by an observation \(\mathbf{z}_k\) is whatever the sensors actually produce at time \(k\): a voltage, a pixel, a range, a GPS latitude. Why the distinction is not pedantic is that the observation is almost always a lossy, transformed, noisy shadow of the state. A GPS receiver reports position but says nothing direct about velocity. An accelerometer reports the second derivative of position but nothing about position itself. A camera reports a nonlinear projection of 3D geometry onto a 2D plane. In every case the map from state to measurement discards information and adds corruption, which is exactly why we cannot simply invert a sensor reading and read off the state.

The two models: process and measurement

A state-space model is exactly two equations. The process model (also transition or dynamics model) says how the hidden state evolves from one step to the next:

Here \(f\) encodes the physics or the assumed behavior, \(\mathbf{u}_k\) is any known control input (a commanded motor torque, a throttle setting), and \(\mathbf{w}_k\) is process noise: the honest admission that your model of the world is incomplete. Gusts of wind, unmodeled friction, and a pedestrian who changes their mind all live inside \(\mathbf{w}_k\). Its covariance \(\mathbf{Q}_k\) is how loudly you say "my dynamics are only approximately right," and tuning it is the subject of Section 9.4.

The measurement model (or observation model) says how the current hidden state produces the sensor reading:

The function \(h\) is the sensor's forward map, the same physics you characterized in Chapter 2: how a true position becomes a range return, how a true temperature becomes a thermistor voltage. The term \(\mathbf{v}_k\) is measurement noise with covariance \(\mathbf{R}_k\), the electrical and quantization jitter of the sensor itself. When \(f\) and \(h\) are linear, we write them as matrices \(\mathbf{F}\) and \(\mathbf{H}\), and the model becomes the linear-Gaussian system that the Kalman filter of Section 9.3 solves in closed form.

The Markov assumption and why it buys you everything

The two equations above quietly encode a powerful assumption: the future depends on the past only through the current state. Formally, \(p(\mathbf{x}_k \mid \mathbf{x}_{k-1}, \mathbf{x}_{k-2}, \ldots) = p(\mathbf{x}_k \mid \mathbf{x}_{k-1})\), and each measurement depends only on the state at its own instant, \(p(\mathbf{z}_k \mid \mathbf{x}_{0:k}, \mathbf{z}_{0:k-1}) = p(\mathbf{z}_k \mid \mathbf{x}_k)\). This is the first-order Markov property, and it is what makes recursive estimation possible. Because all relevant history is compressed into \(\mathbf{x}_k\), a filter can summarize an entire sensor stream by a single evolving belief and update it one measurement at a time, never revisiting the raw archive. This is the hinge on which Section 9.2 turns.

When the assumption holds is a design responsibility, not a gift. If your dynamics genuinely depend on a quantity you left out of the state (a slowly drifting sensor bias, an unmodeled acceleration), the process becomes non-Markov in your chosen coordinates and the filter will be biased no matter how you tune it. The standard cure is state augmentation: fold the offending quantity (a gyro bias, a clock offset, a wind term) into \(\mathbf{x}_k\) so the Markov property is restored. This diagram summarizes the flow of dependence that the whole chapter exploits.

Figure 9.1.1 makes the Markov structure literal: information flows left to right along the top chain and downward into the observations, and there are no long-range arrows. That absence is precisely what lets the recursion in the next section forget the past safely.

Building a model in practice: the constant-velocity template

The most-used starting point in all of tracking is the constant-velocity (CV) model. You do not believe objects truly move at constant velocity; you believe acceleration is small and unpredictable, and you shove it into \(\mathbf{Q}\). With state \(\mathbf{x} = [p, v]^\top\) in one dimension and step \(T_s\), the process is \(p_k = p_{k-1} + v_{k-1} T_s\) and \(v_k = v_{k-1}\), and a position-only sensor observes \(z_k = p_k + v_k^{\text{meas}}\). The snippet below assembles the four matrices \(\mathbf{F}, \mathbf{H}, \mathbf{Q}, \mathbf{R}\) that fully specify this linear-Gaussian model.

import numpy as np

def constant_velocity_model(dt, q_accel, r_pos):
    """Return (F, H, Q, R) for a 1-D constant-velocity state [position, velocity]."""
    # Process model: position integrates velocity; velocity is held (random walk).
    F = np.array([[1.0, dt],
                  [0.0, 1.0]])
    # Measurement model: we observe position only, never velocity.
    H = np.array([[1.0, 0.0]])
    # Process noise from an unknown constant-acceleration disturbance of PSD q_accel.
    Q = q_accel * np.array([[dt**3 / 3, dt**2 / 2],
                            [dt**2 / 2, dt      ]])
    # Measurement noise: variance of the position sensor.
    R = np.array([[r_pos]])
    return F, H, Q, R

F, H, Q, R = constant_velocity_model(dt=0.1, q_accel=2.0, r_pos=5.0)
x = np.array([0.0, 12.0])            # start at 0 m, moving 12 m/s
z_pred = H @ x                       # what the sensor should read
print("state:", x, " predicted measurement:", z_pred)   # -> [ 0. 12.]  [0.]

Code 9.1.1 is the entire model. Everything downstream, prediction, update, tuning, is a manipulation of these four objects. The structure of \(\mathbf{Q}\) here is not arbitrary: it is the exact covariance you get by integrating a white-noise acceleration over one step, which is why it couples position and velocity off-diagonally rather than treating them as independent.

Choosing the state: the decision that decides everything

How you pick the state vector is the highest-leverage decision in the whole pipeline, and it trades three pressures. Too small a state (position only, no velocity) violates sufficiency, breaks the Markov property, and biases the filter. Too large a state (every conceivable bias, every mode) is observable in principle but not in practice: some directions of it will never be pinned down by your sensors, and the filter will drift or diverge along them, a failure we dissect under observability in Section 9.5. The craft is to include exactly the hidden quantities that (a) matter for prediction and (b) are recoverable from the measurement stream. A good rule: add a state variable when you can name the sensor evidence that will constrain it. If you cannot, you are adding a free parameter for the filter to drift along, not a state for it to estimate.

What's Next

In Section 9.2, we turn these two static equations into motion. Given the process and measurement models defined here, and the Markov structure of Figure 9.1.1, we derive recursive Bayesian estimation: the predict-then-update loop that maintains a probability distribution over the hidden state and folds in each new observation exactly once. That recursion is the abstract engine; the linear-Gaussian Kalman filter of Section 9.3 is its most famous closed-form instance.