"A number without its covariance is gossip. A number with its covariance is testimony."
A Well-Calibrated AI Agent
Prerequisites
This section assumes Bayes' rule, the multivariate Gaussian, and the meaning of a covariance matrix from the uncertainty primer in Chapter 4. It builds directly on the scalar inverse-variance weighting introduced in Chapter 48, promoting it from one number to a full matrix. The Gaussian product rule derived here is the measurement update you will meet again, wrapped in prediction, as the Kalman filter of Chapter 9. No filtering machinery is needed yet; we work with static estimates.
The Big Picture
Every sensor hands you two things, and most engineers keep only the first. The first is an estimate: a position, a velocity, a heart rate. The second is a covariance: how wrong that estimate is likely to be, and in which directions its errors are coupled. Bayesian fusion is the discipline of treating both as first-class. When two sensors disagree, the covariance decides who to believe and by how much, not as a hand-tuned gain but as the unique answer that Bayes' rule forces. Carry the covariance and fusion becomes a short, principled matrix computation. Drop it and you are left guessing weights, and your fused estimate will be overconfident precisely when it matters.
Fusion is a posterior, not an average
The what of Bayesian fusion is simple to state: each sensor reading is evidence, and combining sensors means computing the posterior over the world state given all of that evidence. Start from a prior belief \(p(s)\) over the state \(s\), collect a measurement \(z\) whose likelihood is \(p(z \mid s)\), and Bayes' rule delivers the fused belief,
\[ p(s \mid z) = \frac{p(z \mid s)\, p(s)}{p(z)} \propto p(z \mid s)\, p(s). \]The why this matters for fusion is that the rule composes. Two independent measurements \(z_1\) and \(z_2\) multiply their likelihoods, so the posterior after both is \(p(s \mid z_1, z_2) \propto p(z_1 \mid s)\, p(z_2 \mid s)\, p(s)\). Fusion is therefore not a weighted average chosen by taste; it is a product of probability densities, and the "weights" fall out of the shapes of those densities. A sensor that is sharply peaked (low uncertainty) pulls the product hard toward its own reading; a sensor that is broad and vague barely moves it. This is why the same framework handles a precise lidar and a vague radar without any special-casing: their uncertainties do the arbitration automatically, a point developed at the systems level in Chapter 48.
Covariance is the currency of belief
To turn the product rule into arithmetic we need a shape for the densities, and for sensor fusion the workhorse is the multivariate Gaussian \(\mathcal{N}(\mu, \Sigma)\). The what of the covariance matrix \(\Sigma\) is a full account of second-order uncertainty: its diagonal entries are the per-axis variances, and its off-diagonal entries \(\Sigma_{ij}\) record how errors in component \(i\) and component \(j\) move together. The why off-diagonals matter is that sensor errors are rarely axis-aligned. A stereo camera localizes a landmark tightly in bearing but loosely in range, so its position error is a long thin ellipse pointed along the line of sight, not a circle. That tilt lives entirely in the off-diagonal terms, and a fusion rule that ignores it will mix range error into bearing and corrupt both.
Geometrically, a Gaussian's covariance is an uncertainty ellipse (an ellipsoid in higher dimensions): its axes point along the eigenvectors of \(\Sigma\) and their half-lengths scale with the square roots of the eigenvalues. Fusing two Gaussians is then the geometric act of intersecting two ellipses, and the intersection is always at least as tight as either parent. The right way to measure how far a candidate state \(s\) sits from a reading, accounting for this shape, is the Mahalanobis distance \((s-\mu)^\top \Sigma^{-1} (s-\mu)\), which stretches space so that "one sigma in a confident direction" and "one sigma in a vague direction" count equally. That same quadratic form reappears as the gate for rejecting outliers and mismatched detections later in this chapter.
Key Insight
Work in the information form and fusion becomes addition. Define the information matrix \(Y = \Sigma^{-1}\) (the inverse covariance, also called precision) and the information vector \(y = \Sigma^{-1}\mu\). For independent Gaussian measurements the fused belief is simply \(Y_\text{fused} = \sum_i Y_i\) and \(y_\text{fused} = \sum_i y_i\). Precisions add. This is the exact matrix generalization of the scalar \(1/\sigma_\text{fused}^2 = 1/\sigma_1^2 + 1/\sigma_2^2\) from Chapter 48, and it explains why filters that carry the information form fuse many sensors in any order, cheaply, just by summing contributions.
The Gaussian product rule, in two lines
Concretely, fusing two Gaussian estimates \(\mathcal{N}(\mu_1, \Sigma_1)\) and \(\mathcal{N}(\mu_2, \Sigma_2)\) of the same state yields another Gaussian whose covariance and mean are
\[ \Sigma_f = \left( \Sigma_1^{-1} + \Sigma_2^{-1} \right)^{-1}, \qquad \mu_f = \Sigma_f \left( \Sigma_1^{-1}\mu_1 + \Sigma_2^{-1}\mu_2 \right). \]The how to read this: the fused mean is a covariance-weighted average of the inputs, where each input is weighted by its precision \(\Sigma_i^{-1}\) rather than a scalar. A sensor that is confident along one axis dominates the fused estimate along that axis while yielding to its partner along the axis where it is vague. That per-direction arbitration is the whole payoff of carrying the matrix. Listing 49.1 fuses two 2D position estimates with perpendicular error ellipses and shows the fused ellipse collapsing to the small region both sensors agree on.
import numpy as np
# Two 2D position estimates of the same landmark, in metres.
mu1 = np.array([10.0, 5.0]) # sensor A
S1 = np.array([[4.0, 0.0], [0.0, 0.25]]) # tight in y, loose in x
mu2 = np.array([10.6, 4.7]) # sensor B
S2 = np.array([[0.25, 0.0], [0.0, 4.0]]) # tight in x, loose in y
Y1, Y2 = np.linalg.inv(S1), np.linalg.inv(S2) # information (precision) matrices
Sf = np.linalg.inv(Y1 + Y2) # fused covariance: precisions add
mu_f = Sf @ (Y1 @ mu1 + Y2 @ mu2) # covariance-weighted mean
print("fused mean:", np.round(mu_f, 3)) # [10.32 4.82]
print("input stds (x,y):", np.sqrt(np.diag(S1)), np.sqrt(np.diag(S2)))
print("fused stds (x,y):", np.round(np.sqrt(np.diag(Sf)), 3)) # ~[0.485 0.485]As Listing 49.1 shows, the fused ellipse is smaller than either input along every direction, which is the vector version of the promise from Chapter 48 that fusion never makes you worse off. It is also, term for term, the measurement-update step of the Kalman filter in Chapter 9, met here without the time dynamics that surround it there.
In Practice: a warehouse robot fuses wheel odometry and a laser scan match
An autonomous forklift estimates its pose from two sources. Wheel odometry integrates encoder ticks: it is smooth and confident along the direction of travel but accumulates heading error, so its covariance is a long ellipse pointed across the aisle. A laser scan matched against the shelf racks fixes lateral position and heading tightly against the known walls but slides along a featureless corridor, giving an ellipse pointed down the aisle. Neither alone keeps the truck centered: trust odometry and it drifts into a rack; trust the scan match and it jitters along the corridor. Fused with the Gaussian product rule, the cross-aisle confidence of the laser and the down-aisle confidence of the odometry intersect into a tight pose in both directions. The covariance did the reasoning; no gains were tuned. This is the static core of the pose fusion that Chapter 24 runs continuously over time.
The trap: fusing correlated estimates
The product rule above assumed the two measurements were independent, and that assumption is where fusion quietly breaks. Adding information matrices double-counts any information the two estimates already share. In practice shared information is everywhere: two trackers that both consumed the same GPS fix, a network of nodes that have been exchanging and re-fusing each other's estimates, or two features derived from one camera frame. Feed such correlated estimates into \(\Sigma_1^{-1} + \Sigma_2^{-1}\) and the fused covariance shrinks as though you had two independent votes when you really had one and a half. The estimate becomes overconfident: its reported uncertainty is smaller than its actual error, the most dangerous failure a fusion system can have, because downstream logic trusts a number it should not. The blunt rule: never fuse two estimates with the naive information sum unless you can defend their independence.
Research Frontier: consistency without knowing the correlation
When the cross-correlation between two estimates is unknown, the standard robust answer is Covariance Intersection (Julier and Uhlmann), which fuses via \(Y_f = \omega\,\Sigma_1^{-1} + (1-\omega)\,\Sigma_2^{-1}\) with \(\omega \in [0,1]\) chosen to minimize the fused covariance. It is provably consistent (never overconfident) for any unknown correlation, at the cost of being conservative when the inputs are in fact independent. It underpins decentralized multi-robot mapping and vehicle-to-vehicle fusion, where estimates loop through the network and true correlations are intractable to track. Active work extends it with Split Covariance Intersection, which separates provably-independent from possibly-correlated parts to recover tightness, and with consistent fusion rules for the set-membership and particle representations used beyond the Gaussian world. Formal consistency checking is the subject of Section 49.7.
The Right Tool
Hand-rolled, the Gaussian product means inverting each covariance, summing the precisions, inverting again, and mapping the information vector back to a mean, roughly a dozen lines that invite a transpose bug or a singular-matrix crash when a covariance is near-degenerate. A filtering library that already carries the information form reduces a multi-sensor update to a loop of additions:
import numpy as np
# Fuse N Gaussian estimates in information form: one add per sensor.
estimates = [(mu1, S1), (mu2, S2)] # from Listing 49.1
Y = sum(np.linalg.inv(S) for _, S in estimates) # precisions add
y = sum(np.linalg.inv(S) @ mu for mu, S in estimates)
Sf, mu_f = np.linalg.inv(Y), np.linalg.inv(Y) @ yfilterpy and GTSAM wrap this in numerically stable factorizations and, crucially, do not check independence for you: guarding against the correlated-input trap above remains the engineer's judgment.As Listing 49.2 shows, the library removes the linear algebra, not the modeling. Whether these estimates are truly independent, and therefore whether the naive sum is even legal, is a decision no library will make for you.
Exercise
Take the two estimates from Listing 49.1 and deliberately make sensor B a near-copy of sensor A by setting \(\mu_2 = \mu_1\) and \(\Sigma_2 = \Sigma_1\). Fuse them with the naive information sum and report the fused covariance. By what factor did the reported variance shrink, and is that shrinkage justified if B is actually a re-broadcast of A's own estimate? Then refuse the fusion using Covariance Intersection with \(\omega = 0.5\) and compare. Write one sentence on what each answer would tell a robot about whether it is safe to move.
Self-Check
1. Two sensors estimate the same 2D position. Sensor A is tight along \(x\), sensor B along \(y\). Without computing, sketch the fused uncertainty ellipse and say why it beats both inputs on both axes.
2. Why does the information form make fusing ten sensors trivial, and what single property of the measurements does it silently assume?
3. A fused estimate reports a covariance smaller than the true error of the fused mean. Name the assumption that was violated and one concrete way it happens in a sensor network.
What's Next
In Section 49.2, we set the fused belief in motion. The Gaussian product rule you just met becomes the update half of the multi-sensor Kalman filter, paired with a predict step that pushes the covariance forward through the system's dynamics, so that estimates from sensors arriving at different rates and different times can all be folded into one running, uncertainty-aware state.