Sensors as imperfect windows into the world

"I have never once touched the world. I have only ever read its handwriting, and the handwriting is dreadful."

A Nearsighted AI Agent

A sensor projects the world onto a handful of numbers

Consider what actually happens when a device "measures temperature." A room contains something on the order of \(10^{25}\) air molecules, each with its own position and velocity, and temperature is a statistical summary of all that motion. A thermometer collapses that entire high-dimensional state into a single scalar, updated a few times per second. This is partial observability: the sensor projects a vast hidden state \(s\) down to a few numbers. Projection is lossy by construction. The moment you go from \(10^{25}\) degrees of freedom to one, you throw almost all of the world away. No amount of downstream cleverness recovers what was never captured.

What survives the projection is chosen by physics, not by you. A microphone integrates pressure over its diaphragm and reports a voltage; it cannot tell you which of two people in the room spoke. A three-axis accelerometer reports force along three directions and stays silent about color, smell, and the intent of whatever is shaking it. The what of a sensor is this: it maps the state you care about, plus a great deal you do not, onto a low-dimensional signal. The why it matters is that the map is many-to-one. Different worlds can produce identical readings, so a single number can never uniquely name the state that caused it. That ambiguity is not a defect of cheap hardware; it is the geometry of measurement.

Noise, bias, and drift are properties of measurement, not accidents

Ambiguity is only the first way the world resists clean measurement; a second family of imperfections corrupts even the numbers that do survive the projection. It is tempting to imagine a "perfect" sensor whose errors are just engineering laziness waiting to be fixed. That intuition is wrong, and unlearning it is the point of this section. Three distortions are intrinsic to the act of measuring, and it helps to name them precisely.

Noise is the random jitter riding on every reading. It has deep physical roots: thermal agitation of electrons (Johnson-Nyquist noise), the discreteness of photons and charge (shot noise), quantization when a continuous voltage is forced onto a finite number of digital levels. You can average noise down, but you can never delete it, because it is the world and the transducer (the component that converts a physical quantity into an electrical signal) being made of countable, jostling things.

A single noisy reading is why a fitness tracker can misjudge your resting heart rate by ten beats, or a furnace controller chatter around its setpoint and scorch the batch: the value is right on average yet wrong on any given sample. Averaging is the cheapest defense engineers deploy against exactly this. Averaging deserves a precise word, because it is the first tool everyone reaches for. It is the simplest possible estimator: take \(N\) repeated readings of a quantity you believe is roughly constant and report their mean. It matters because it buys accuracy almost for free, and it works because zero-mean random errors (as often too high as too low) partly cancel when summed, so the standard deviation of the mean shrinks like \(1/\sqrt{N}\): a hundred readings cut the noise tenfold, and ten thousand cut it a hundredfold. Reach for averaging when the underlying value holds still over your window and the error is random and zero-mean; reach for a filter instead when the signal itself is moving, and remember that averaging does nothing at all to bias or drift, which do not cancel. In short: a sensor hands you the world's handwriting, and skill is never mistaking one dreadful scrawl for the truth it distorts.

Bias is a systematic offset: the reading is consistently wrong in the same direction. A cheap accelerometer at rest rarely reports zero; it reports a small, stubborn nonzero vector. Averaging makes bias more visible, not less, because it does not cancel.

Drift is bias that will not hold still. As a component warms up, ages, or its supply voltage sags, its zero point wanders. The calibration you performed this morning is typically already slightly stale by afternoon. Drift is why "calibrate once and forget" is a fantasy and why Chapter 9 spends real effort tracking a slowly-moving bias as part of the hidden state rather than pretending it is constant. Figure 1.1.1 illustrates noise vs. bias vs. drift, and what averaging can and cannot fix.

Two quick vignettes make the point physical. Leave a thermometer in direct sunlight and it will happily report the temperature of a sunbaked plastic case, not the air; the instrument faithfully measured the wrong thing, and its reading is a confident lie. Set a smartphone flat on a table and its accelerometer reports roughly \(9.8\ \mathrm{m/s^2}\) pointing up, because an accelerometer cannot separate the specific force, which is the acceleration a sensor feels relative to free fall, of gravity from genuine motion. It is not broken. It is telling you the literal truth about force, which is not the question you asked. In both cases the number is real and the interpretation is a trap.

Perception is an inverse problem

Put the pieces together and a shape emerges. Physics runs forward: a hidden state \(s\) causes a signal through some sensor response \(h(\cdot)\), corrupted by measurement noise \(\eta\). Written compactly, and stripped of every subtlety that Chapter 2 will restore,

You never get \(s\). You get \(x\), and your job is to run the arrow backward: recover the cause \(\hat{s}\) from the effect \(x\). This is an inverse problem, hard for a specific reason. Because \(h(\cdot)\) is many-to-one and \(\eta\) is random, many states \(s\) fit the same \(x\), so inversion is never pure deduction; it is inference under uncertainty. The right output is not one confident answer but a distribution over states, the posterior \(p(s \mid x)\) that Chapter 4 builds from the ground up.

The diagram in Figure 1.1 traces this loop. The world holds a rich hidden state; the sensor squeezes it through \(h(\cdot)\) and adds \(\eta\), yielding a thin signal \(x\); and inference reaches back across that lossy channel to propose \(\hat{s}\), always with a margin of doubt attached.

No single sensor gives you ground truth

Here is the conclusion that reorganizes everything. Because measurement is a lossy, biased, drifting projection, and because inversion is ambiguous, a single reading is a claim, not a fact. The thermometer claims the air is warm; the PPG sensor claims the heart beats at 150; the accelerometer claims a force. Each claim is evidence, weighted by how much you trust the instrument that made it, and evidence can be wrong. Treating one sensor's output as ground truth is the original sin of applied sensing. In practice, a great many field failures trace back to it: a drone that flips because it trusted a single bad gyro reading, a monitor that pages a nurse over a motion artifact.

This is precisely why the rest of the book exists. If one sensor cannot be trusted, you characterize its error (calibration), you suppress what you can and quantify what remains (filtering and uncertainty), you combine independent instruments so their errors cancel rather than compound (fusion), and you keep tracking the drift while the system runs (state estimation). Every one of those chapters is a different answer to the same sentence: the instrument mediates everything, so knowing the instrument is the prerequisite to trusting the data.

from scipy.signal import detrend
x_clean = detrend(x)          # removes a fitted linear drift from x

import numpy as np
a = np.loadtxt("accel.csv", delimiter=",", skiprows=1)  # t, ax, ay, az
print("mean (bias):", a[:, 1:].mean(axis=0))
print("std  (noise):", a[:, 1:].std(axis=0))

What's Next

Section 1.2 turns the smudged-pane idea into a working distinction: why building AI on physical measurements differs so sharply from building AI on text and pixels, and why the presence of an instrument between the model and the world changes almost every design decision that follows.