"The cheapest measurement is the one I never had to take, because I already knew what it would say. The most valuable is the one that proved me wrong."
A Prescient AI Agent
Prerequisites
This section turns the world models of the previous five sections into a sensing tool, so you should be comfortable with the latent forward model \(z_{t+1}=f(z_t,a_t)\) from Section 53.2 and the representation-space prediction of Section 53.3. The anomaly machinery leans on classical change detection (Chapter 12) and on calibrated intervals from conformal prediction (Chapter 18). If "residual", "lead time", and "prediction interval" are familiar, you are ready.
The Big Picture
Every world model in this chapter was built to answer one question: given what I have seen and what I might do, what happens next? The earlier sections spent that answer on planning, choosing actions inside imagination. This section spends it on sensing itself. A model that can forecast its own future observations gives you three things that no purely reactive perception stack has. First, anticipation: it names events before they occur, converting a horizon of tens of milliseconds to several seconds into decision time. Second, surprise: the gap between what it predicted and what the sensor actually delivered is a self-supervised anomaly signal that needs no labeled faults. Third, economy: when the forecast is confident you can skip a measurement, delay a wake-up, or hide latency by acting on the prediction instead of waiting for the reading. Predictive sensing is the discipline of treating a good forecast as a first-class sensor output, cheaper than silicon and available before the physical event.
Anticipation is the world model's native output
The what: anticipation is producing a distribution over a future observation or a future event, \(p(o_{t+h}\mid o_{\le t}, a_{\le t})\), at a lead time \(h\) chosen to be long enough to act on and short enough to stay accurate. A world model already computes this. Rolling the latent dynamics \(f\) forward \(h\) steps from the current inferred state and decoding, or reading off an event head, is anticipation; nothing new needs to be trained if the model was fit to predict.
The why is latency budgets. Physical actuation, human reaction, and network round-trips all take time the world will not pause for. A collision-avoidance brake that fires on the frame in which contact is already unavoidable is useless; one that fires when the model's forecast of the pedestrian's latent trajectory crosses the vehicle path with high probability, 700 ms early, is a safety system. This is the same forecasting the recurrent and generative models did internally for control (Section 53.5), redirected outward so that perception reports the future, not just the present.
The how splits by what you predict. Predict the raw signal and you get a full anticipated observation you can feed to an existing detector, expensive but general. Predict in representation space (the JEPA route of Section 53.3) and you anticipate the embedding, cheaper and more robust because irreducible sensor detail was already discarded. Predict a scalar event probability directly and you get the leanest anticipatory alarm of all. Most deployed systems use the third for the alarm and the second for the reasoning behind it.
Key Insight: prediction error is a free, always-on anomaly detector
A world model trained only to forecast normal operation ships an anomaly detector for free. Define the surprise \(s_t = d\big(o_t, \hat{o}_{t\mid t-1}\big)\), the distance between the arriving observation and what the model, one step earlier, expected. On in-distribution data \(s_t\) stays small; when the physics the model learned no longer holds (a cracked bearing, a spoofed lidar return, an arrhythmia), the forecast fails and \(s_t\) spikes. This is precisely the residual-monitoring idea of Chapter 12, but the "expected value" now comes from a learned nonlinear world model rather than a linear filter, so it flags anomalies that are structural rather than merely large. The subtle failure mode: an over-powerful generative model can learn to reconstruct anomalies too, quietly reconstructing the fault and reporting no surprise. Predicting one step ahead rather than reconstructing the present, and predicting in a bottlenecked latent space, both guard against that.
Surprise, calibrated: from spike to trustworthy alarm
A raw surprise trace is noisy, so a bare threshold either misses faults or drowns operators in false alarms. Two ideas make it deployable. First, calibrate the threshold with conformal prediction (Chapter 18): compute surprise scores on a clean calibration set, take the empirical \((1-\alpha)\) quantile \(q\), and flag \(s_t>q\). Under exchangeability this gives a distribution-free guarantee that the normal-data false-alarm rate is about \(\alpha\), with no assumption on the shape of \(p(s)\). Second, accumulate evidence over time instead of trusting a single frame: a CUSUM or a short exceedance count on \(s_t\) turns a persistent small drift into a confident detection while ignoring one-frame glitches. The two combine cleanly, and the payoff, unique to a predictive detector, is lead time: because the model saw the near future, the alarm can precede the overt failure. That lead time is the metric a condition-monitoring team actually cares about (Chapter 37), and it is what separates anticipation from ordinary detection.
import numpy as np
def anticipatory_alarm(world_model, stream, calib_scores, alpha=0.01, k=5):
"""One-step-ahead surprise, conformal threshold, CUSUM lead-time alarm."""
q = np.quantile(calib_scores, 1 - alpha) # distribution-free threshold
z = world_model.infer_state(stream[0]) # latent belief
cusum, fired_at = 0.0, None
for t in range(1, len(stream)):
o_hat = world_model.decode(world_model.step(z)) # forecast o_t at t-1
s = np.linalg.norm(stream[t] - o_hat) # surprise / residual
cusum = max(0.0, cusum + (s - q)) # accumulate exceedance
if cusum > k * q and fired_at is None:
fired_at = t # anticipatory alarm
z = world_model.update(z, stream[t]) # assimilate reading
return fired_at
q sets a calibrated per-step false-alarm rate; the CUSUM accumulator converts a persistent forecast breakdown into an early, low-jitter detection. fired_at minus the true failure index is the lead time reported in the text. Only step, decode, and update are model-specific; the rest is domain-agnostic.Listing 53.6 is deliberately thin because the world model does the heavy lifting: step and decode are the forecast, update assimilates the real reading, and everything else is the calibration wrapper from Chapter 18. Swapping a DreamerV3 latent model for a V-JEPA embedding changes only what "surprise" is measured in.
Practical Example: an AR headset that renders the future
A mixed-reality headset has a brutal constraint called motion-to-photon latency: the roughly 40 to 60 ms between a head movement and the light that reflects it, spanning IMU read, pose fusion, render, and display scanout. Show the pose you measured and the world visibly lags the head, which induces nausea. The fix is anticipation. A lightweight motion world model, fed the fused orientation stream from Chapter 24, forecasts head pose one display frame ahead and the renderer draws for that predicted pose, so the photons arrive aligned with where the head actually is. The same forecast doubles as a sensor: when the measured pose disagrees sharply with the anticipated pose, the surprise flags a tracking dropout (a covered camera, a fast unmodeled jerk) and the compositor falls back to a conservative reprojection instead of showing a wrong frame. One predictive model buys both latency compensation and fault detection: it renders the future when confident, and confesses when it cannot.
Predicting to sense less: scheduling, gap-filling, latency compensation
The third use flips the economics of sensing. If a forecast is confident, the corresponding physical measurement carries little new information, so you can spend the sensor budget elsewhere. This is predictive, or attentive, sensing. When the anticipated observation and its uncertainty are within tolerance, skip or downsample the reading, wake the radio less often, or leave a camera region unread; when predictive uncertainty rises, or surprise breaches threshold, sense densely. The saved energy is decisive for battery-bound wearables and intermittently powered nodes (Chapter 63), and the value-of-information logic is the same one that makes an event camera transmit only on change (Chapter 46). The model also fills gaps: when a modality drops out or a packet is late, the forecast is the maximum-a-posteriori stand-in until real data returns, a graceful degradation that a reactive fusion stack cannot offer. And by running the state estimate a fixed horizon ahead, it compensates the pipeline's own latency, publishing a present-time estimate from delayed inputs.
Right Tool: let the probabilistic forecaster own the intervals
Hand-rolling a calibrated predictive-sensing loop means writing a forecaster, a proper uncertainty estimate, quantile calibration, and a re-fit schedule: a few hundred lines, and the uncertainty is the easy part to get quietly wrong. A modern probabilistic time-series library collapses the forecast-plus-interval step to a handful of lines. With darts, model.fit(series) then model.predict(h, num_samples=200) returns a sampled predictive distribution, and .quantile_df() hands you the very band that drives the skip-or-sense decision and the surprise threshold, roughly a 20-line replacement for a 300-line bespoke pipeline. The library owns the sampling, backtesting, and quantile bookkeeping; you own only the tolerance policy. Reach for the from-scratch world model of Listing 53.6 when the dynamics are action-conditioned or the state is high-dimensional latent, where a univariate forecaster no longer fits.
Research Frontier: intuitive-physics surprise as a sensing signal
The current frontier reads anticipation off large pretrained video and sensor world models rather than task-specific ones. V-JEPA 2 (Meta, 2025) shows that a model pretrained on internet-scale video develops an intuitive-physics prior: its representation-space prediction error spikes on physically impossible events (an object passing through a wall, a ball that vanishes), matching the "violation of expectation" paradigm from developmental psychology. That spike is exactly the surprise signal of this section, now emerging zero-shot from a foundation model that saw no anomaly labels. Open problems for sensing: calibrating this surprise so it is a trustworthy alarm rather than a confident artifact (the reconstruct-the-fault trap of the key insight), extending it beyond vision to fused IMU, tactile, and RF streams, and deciding the right lead-time horizon per task. Evaluating whether these models truly anticipate physics, rather than pattern-match appearance, is the subject of Section 53.7.
When anticipation pays, and when it misleads
The when follows from horizon economics. Predictive sensing pays when there is a real latency or energy cost to acting late, when the dynamics are learnable and locally smooth, and when the useful lead time \(h\) is shorter than the model's accuracy horizon. Head-pose forecasting over 40 ms, bearing-fault anticipation over minutes, and pedestrian-intent over a second all sit in that regime. It misleads in three cases worth naming. When the process is genuinely unpredictable at the horizon you need, the forecast is confident noise and acting on it is worse than waiting; the uncertainty band, not the point forecast, must gate the decision. When anomalies are the target but the model is expressive enough to reconstruct them, surprise vanishes exactly when you need it, so predict ahead and bottleneck the state. And when the world drifts (a new operating mode, seasonal change), a frozen model's rising baseline surprise masquerades as a fault; test-time adaptation and drift monitoring (Chapter 66) keep the anticipation honest. Anticipation buys decision time by trusting a forecast; the discipline is knowing, from calibrated uncertainty, exactly how far that trust extends.
Exercise
Build an anticipatory detector on a run-to-failure sensor set (for example a bearing vibration dataset). (1) Train a one-step latent forecaster on the healthy prefix only, leakage-safe, and instrument the surprise \(s_t\) of Listing 53.6. (2) Calibrate a conformal threshold at \(\alpha=0.01\) on held-out healthy data, add the CUSUM accumulator, and report both the false-alarm rate on healthy segments and the lead time (alarm index minus labeled failure index) across runs. (3) Sweep the forecast horizon \(h\in\{1,5,20\}\) steps and plot lead time against precision, showing where a longer horizon stops helping because the forecast degrades. (4) Repeat the surprise measurement with a high-capacity autoencoder that reconstructs the present instead of predicting the future, and demonstrate the reconstruct-the-fault failure from the key insight.
Self-Check
1. A world model was trained only on normal operation, with no fault labels. Explain how it nonetheless yields an anomaly detector, and state the one architectural choice that stops it from silently reconstructing the fault. 2. Why does conformal calibration on the surprise score give a false-alarm-rate guarantee without assuming the score's distribution, and what property must the calibration data satisfy? 3. Give two distinct ways a forecast reduces sensing cost (not just detects faults), and name the quantity that must gate the decision so you do not act on a confident-but-wrong prediction.
What's Next
In Section 53.7, we ask how to trust any of this: the physical-reasoning benchmarks that test whether a world model has really learned to anticipate physics rather than to memorize appearance. The surprise-on-impossible-events signal we just exploited becomes, on the evaluation side, the very probe that tells a genuine world model from a convincing mimic.