Section 9.5: Observability and divergence

"A filter that never doubts itself is not confident; it is lost, and proud of it."
A Self-Aware AI Agent

The big picture

Section 9.3 gave you a working Kalman filter and Section 9.4 taught you to tune its noise covariances. Both assumed the measurements actually carry enough information to pin the state down, and that the filter's own confidence tracks reality. This section confronts the two ways that assumption breaks. Observability asks a structural question: given the model, can any sequence of measurements recover the hidden state at all, or are some directions invisible? Divergence is the failure that follows when they cannot: the filter reports a tiny covariance while its estimate drifts far from the truth, confidently wrong. You will learn to test observability before deployment, to watch the innovation stream for divergence at run time, and to apply the small set of fixes (process-noise inflation, fading memory, and numerically stable updates) that keep a filter honest. Get this right and your estimator degrades gracefully; get it wrong and it lies to every consumer downstream.

This section assumes the linear-Gaussian filter of Section 9.3 and the covariance intuition from Section 9.4. It leans on basic linear algebra (matrix rank, positive-definiteness) and the chi-square distribution from Chapter 4. Following the convention fixed in Chapter 1, this chapter uses control notation: the hidden state is $x_k$, the measurement is $z_k$, the transition matrix is $F$, and the measurement matrix is $H$.

Observability: can the measurements pin the state down?

What it is. A linear time-invariant system with transition $F$ and measurement matrix $H$ is observable if, from a finite run of noiseless measurements, you can reconstruct the full state $x_0$. The test is algebraic. Stack the measurement map with its propagations through the dynamics into the observability matrix

$$ \mathcal{O} = \begin{bmatrix} H \\ HF \\ HF^2 \\ \vdots \\ HF^{n-1} \end{bmatrix}, $$

where $n$ is the state dimension. The system is observable if and only if $\mathcal{O}$ has full column rank $n$. Each block adds one more time step of measurement seen through the dynamics; if after $n$ steps some direction in state space has still produced zero measurement, that direction is invisible and no amount of data recovers it.

Why it matters more than it looks. Rank is binary, but reality is graded. A system can be observable on paper yet weakly observable in practice, meaning $\mathcal{O}$ is nearly rank-deficient and its smallest singular value is close to zero. Then the state is technically recoverable but only by amplifying measurement noise enormously, and the filter's covariance in that direction stays stubbornly large. Checking the rank tells you whether a state is knowable; checking the singular-value spectrum of $\mathcal{O}$ tells you how expensively.

How to read the failure. An unobservable direction is a null-space vector of $\mathcal{O}$. Perturb the true state along it and the entire future measurement stream is unchanged, so the filter has no lever to correct an error there. The classic sensor example is bias: an accelerometer bias and a genuine acceleration produce identical outputs while motion is constant, so the two are unobservable until the platform maneuvers and breaks the tie. This is exactly why orientation and dead-reckoning systems need excitation to converge, and why a stationary inertial unit can never learn its own gyro bias.

Key insight

Observability is a property of the model, not the tuning. No choice of $Q$ or $R$ can make an unobservable state observable; the information simply is not in the measurements. Tuning changes how fast an observable state converges and how a weakly observable one trades noise for responsiveness, but a rank-deficient $\mathcal{O}$ is a design defect you fix by adding a sensor, changing the motion, or removing the state, never by turning a knob.

Divergence: the confidently wrong filter

What it is. A Kalman filter diverges when its estimate error grows while its reported covariance $P_k$ shrinks or stays small. The two numbers that should move together, actual error and believed uncertainty, come apart. The estimate is not just wrong; it is wrong and sure of it, which is worse than a large honest error because every downstream consumer, from an alarm threshold to a fusion stage, treats the tight covariance as trustworthy.

Why it happens. Three mechanisms dominate. First, unobservable or weakly observable states: with no measurement pressure on a direction, the update step never corrects it, yet the predict step keeps folding in process noise and, if $Q$ is small there, $P$ can collapse even as the true error drifts. Second, model mismatch: a constant-velocity filter tracking an accelerating target, or an $R$ set smaller than the real measurement noise, makes the filter overtrust either its model or its sensor, and the Kalman gain drives the estimate into a systematic bias. Third, numerical loss of positive-definiteness: the standard covariance update $P_k = (I - K_k H)P_k^-$ subtracts two matrices and, in finite precision, can produce a $P$ that is asymmetric or has negative eigenvalues, at which point the filter is mathematically broken.

In practice: an automotive GNSS/IMU tracker enters a tunnel

A lane-keeping stack fuses GNSS position fixes with wheel odometry and an inertial unit in a Kalman filter. On the open highway the GNSS updates keep position fully observable and $P$ stays tight. Then the car enters a tunnel and the GNSS drops out for ninety seconds. Now position is observable only through dead reckoning, and any small scale error in the odometry or bias in the yaw rate integrates unchecked. The engineer who set $Q$ too small for the tunnel case sees the filter keep reporting a two-meter position covariance while the true error grows past fifteen meters. When the car exits and a fresh GNSS fix arrives fifteen meters from the filter's confident estimate, the innovation is enormous, the gain over-corrects, and the track snaps sideways across a lane boundary. The fix is not a better sensor; it is a filter that inflates its own uncertainty during the outage so the returning fix is trusted, and that watches its innovations to notice the outage in the first place.

Detecting divergence online: the innovation stream

What it is. You cannot compare against ground truth at run time, so you monitor the one quantity the filter produces that has a known distribution when everything is consistent: the innovation $y_k = z_k - H\hat{x}_k^-$, the gap between the measurement and its prediction. If the filter's model and covariances are correct, the innovation is zero-mean with covariance $S_k = H P_k^- H^\top + R$. The scalar summary is the normalized innovation squared (NIS),

$$ \epsilon_k = y_k^\top S_k^{-1} y_k, $$

which follows a chi-square distribution with degrees of freedom equal to the measurement dimension. A run of NIS values that consistently exceeds the chi-square upper bound means the innovations are larger than the filter believes possible: the model is wrong, the filter is overconfident, or it is diverging.

Why NIS and not raw error. The innovation is available without truth and it is self-normalizing: dividing by $S_k$ accounts for how confident the filter is at each step, so one threshold works across the whole trajectory. A raw residual of ten meters is fine when $S_k$ is large and alarming when it is tiny; NIS folds that judgment in automatically. Averaging NIS over a window and comparing against the two-sided chi-square interval catches both overconfidence (NIS too high) and the rarer overcautious case (NIS too low, meaning $Q$ or $R$ is inflated and the filter is throwing away good measurements).

import numpy as np
from scipy.stats import chi2

# Constant-velocity filter tracking a target that secretly accelerates.
dt = 1.0
F = np.array([[1, dt], [0, 1]])          # state = [position, velocity]
H = np.array([[1.0, 0.0]])               # we measure position only
Q = np.diag([1e-4, 1e-4])                # process noise: too small on purpose
R = np.array([[0.25]])                   # measurement variance

x = np.array([0.0, 1.0]); P = np.eye(2)
true = np.array([0.0, 1.0]); a = 0.05    # unmodeled acceleration
dof = H.shape[0]
hi = chi2.ppf(0.99, dof)                 # upper 99% bound on NIS

for k in range(60):
    true = F @ true + np.array([0.5 * a * dt**2, a * dt])   # real motion
    z = H @ true + np.random.normal(0, np.sqrt(R[0, 0]))
    x = F @ x; P = F @ P @ F.T + Q                          # predict
    y = z - H @ x                                           # innovation
    S = H @ P @ H.T + R
    nis = float(y.T @ np.linalg.solve(S, y))               # normalized innov.
    K = P @ H.T @ np.linalg.inv(S)                          # update
    x = x + (K @ y).ravel(); P = (np.eye(2) - K @ H) @ P
    if k > 5 and nis > hi:
        print(f"step {k:2d}: NIS={nis:6.1f} > {hi:.1f} : divergence flagged")

A constant-velocity Kalman filter is fed a target with a small unmodeled acceleration and a deliberately tiny process noise $Q$. The normalized innovation squared (NIS) is compared against the 99th-percentile chi-square bound for a one-dimensional measurement. As the position error accumulates and $P$ stays small, NIS climbs past the bound and the loop prints a divergence flag well before any external check could. Referenced in prose above and below.

The code above is the whole diagnostic in twenty lines: run the ordinary filter, form NIS at each step, and raise a flag when it breaches the chi-square bound. In a real system that flag would trigger a fallback (reset the covariance, switch to a more conservative model, or drop the offending measurement), which is the subject of Section 9.7.

Keeping the filter honest: the standard fixes

Once you can detect divergence, the remedies are few and well understood. Process-noise inflation is the blunt, reliable tool: raising $Q$ prevents $P$ from collapsing, so the filter never becomes overconfident about a poorly observed direction. The GNSS-tunnel filter above simply needed a larger $Q$ on position during the outage. Fading memory generalizes this by multiplying the predicted covariance by a factor $\alpha^2 > 1$, which exponentially discounts old data and keeps the filter responsive to model changes; it is process-noise inflation with a single interpretable knob. Numerically stable updates address the positive-definiteness failure. The Joseph form of the covariance update,

$$ P_k = (I - K_k H)\,P_k^-\,(I - K_k H)^\top + K_k R\, K_k^\top, $$

is algebraically identical to the short form but is a sum of two symmetric positive-semidefinite terms, so it preserves symmetry and positivity in finite precision. For long-running or high-dimensional filters, square-root and UD-factorized filters propagate a factor of $P$ instead of $P$ itself, guaranteeing a valid covariance at roughly double the arithmetic cost. These are the estimators of choice when a filter must run for hours without a reset, as in inertial dead reckoning.

The right tool: observability and stable updates for free

Hand-rolling an observability check plus a Joseph-form or square-root update is roughly thirty to forty lines of careful matrix code you must test against rank-deficient and ill-conditioned cases. Mature libraries collapse that to a few calls. Observability is a one-liner in python-control, and FilterPy ships numerically stable filters that manage the covariance factorization internally:

import numpy as np
from control import obsv                 # observability matrix + rank
O = obsv(F, H)
assert np.linalg.matrix_rank(O) == F.shape[0], "state not observable"

from filterpy.kalman import KalmanFilter, square_root   # stable update paths
# KalmanFilter.update uses the Joseph form; SquareRootKalmanFilter
# propagates a Cholesky factor so P can never lose positive-definiteness.

Observability testing with python-control's obsv and numerically stable filtering with FilterPy. The library handles the observability-matrix construction, the rank test, and the factorized covariance update (about forty lines of tested code) so you write three. Referenced in the fixes discussion above.

Exercise

Take the constant-velocity filter from the code block above.

Set the unmodeled acceleration a = 0.0 and confirm the NIS stays below the chi-square bound for the whole run. This is your consistent baseline.
Restore a = 0.05 and sweep the process noise $Q$ upward by factors of ten. Find the smallest $Q$ at which the divergence flag stops firing, and explain what that $Q$ is compensating for.
Build the observability matrix $\mathcal{O} = [H;\ HF]$ for this system by hand and confirm it has rank 2. Then change $H$ to measure velocity only (H = [[0, 1]]) and check whether position is still observable.

Hint

For part 3, with $H = [0,\ 1]$ the first block sees only velocity and the second block $HF$ still sees only velocity, because position never feeds back into the velocity row of $F$. The observability matrix loses rank, so absolute position becomes unobservable: you can know how fast you are going forever and never learn where you started.

Self-check

A filter reports a shrinking covariance while its true error grows. Which of the three divergence mechanisms could explain this, and what single quantity would you monitor to catch it without ground truth?
Why can no choice of $Q$ and $R$ rescue an unobservable state, and what three design moves actually can?
The Joseph-form update gives the same answer as the short form in exact arithmetic. What does it buy you in finite precision, and why?

What's Next

In Section 9.6, we relax the filter's strict causality. A filter uses only past and present measurements to estimate the current state; a smoother is allowed to look at the whole record and revise past estimates in hindsight. That extra information sharpens accuracy and, as you will see, quietly repairs some of the weakly observable directions that trouble a forward-only filter, at the cost of latency you cannot always afford.