"They told me the patient had nine months of continuous data. What I received was a shoebox of timestamps: dense some nights, empty for whole weekends, never once landing on a tidy round second."
A Chronically Underslept AI Agent
Prerequisites
This section assumes the sampling and timing vocabulary of Chapter 3: uniform sampling, the Nyquist rate, and why a clock matters as much as a value. It leans on the biosignal engineering anchors from Chapter 28 and the leakage-safe splitting discipline of Chapter 5, which becomes subtle once records span months. No physiology beyond Chapter 28 is required. We treat the fact that a gap is itself informative only lightly here; that idea gets its own full treatment in Section 33.2.
The Big Picture
Almost everything in Parts III and IV quietly assumes a signal arrives as a neat vector, one sample per fixed tick, no holes. Continuous health monitoring violates that assumption on three axes at once. The horizon is long: a diagnosis emerges over weeks or months, not seconds. The clock is irregular: a smartwatch measures when it decides conditions are good, not on a metronome. And the record is sparse: the device is off the wrist during charging, showers, and half of daily life, so most of the timeline is simply absent. Ignore any one of these and your pipeline silently misbehaves: you alias a circadian rhythm, you impute a heart rate for a watch that was in a drawer, or you leak tomorrow into today. This section is about seeing the shape of real longitudinal health data clearly enough to choose a representation that respects it.
Three ways a health stream breaks the grid assumption
The comfortable object at the heart of classical signal processing is the uniform grid: values \(x_1, x_2, \dots, x_N\) at times \(t_k = k/f_s\), one after another, forever. Continuous monitoring breaks that object three distinct ways, and it helps to name them because the fixes differ.
Longitudinal means the horizon is enormous relative to the phenomenon. Resting heart rate is a per-beat quantity, roughly 1 Hz, but the clinically interesting signal, a slow upward drift that precedes an infection or a change in fitness, plays out over days to months. A ninety-day record sampled once per minute is already \(1.3\times10^5\) points per channel, and the label you care about attaches to the whole trajectory, not to any single sample. This is why recurrent, convolutional, and state-space models that scale linearly in sequence length, developed in Chapter 16, matter here far more than they do for a five-second window.
Irregular means the inter-sample interval \(\Delta t_k = t_k - t_{k-1}\) is a random variable, not a constant. A modern watch is opportunistic: it fires the optical sensor when the accelerometer says the wrist is still, when skin contact is good, and when the battery allows. The result is a timeline where \(\Delta t\) might be five seconds during quiet sitting and forty minutes during a busy commute. Treating those two gaps as equal, which any plain array indexing does, throws away exactly the timing that Chapter 3 insisted you protect.
Sparse means that over the long horizon, most of the clock has no observation at all. Wear time for consumer wearables in real studies commonly lands well under half of the calendar; a device worn twelve hours a day and charged overnight already covers only half the timeline, before you subtract showers and forgetful mornings. Sparsity is not the same as irregularity: a record can be irregular but dense (jittered samples that still tile the day) or regular but sparse (one clean reading every night and nothing between). Health streams are usually both irregular and sparse, which is the hardest combination.
Key Insight
Irregular and sparse are orthogonal properties, and conflating them is the root of most modeling mistakes here. Irregularity is about the spacing of the observations you have; sparsity is about the fraction of the timeline you are missing. A fix for one is not a fix for the other. Resampling onto a fine grid can regularize spacing, but it manufactures dense fake data across the sparse gaps, converting an honest absence into a confident fabrication. Always characterize a new dataset on both axes before choosing a model.
Why resampling to a grid quietly lies
The reflex fix for irregular, sparse data is to force it back onto a uniform grid: pick a target rate, then forward-fill, linearly interpolate, or average into bins. It is one line of pandas and it makes every downstream tool happy, which is precisely why it is dangerous. Three costs hide inside that one line.
First, interpolation invents structure. Linearly connecting a heart rate at 08:00 to the next reading at 08:40 draws a smooth ramp across forty minutes in which the user might have climbed two flights of stairs and sat down again. The model then trains on a fiction that is smoother, and more autocorrelated, than any real physiology. Second, binning aliases the rhythms you most want. Human physiology is dominated by a circadian cycle near \(1/86400\) Hz plus weekly structure; coarse or uneven binning folds that low-frequency structure in ways the aliasing analysis of Chapter 3 predicts, corrupting exactly the multi-day trend that carries the clinical signal. Third, and most insidiously, imputation erases the missingness itself. Whether the watch was off because the user was showering or because they were hospitalized is often diagnostic, and a forward-fill overwrites that fact with a plausible number. We hold that thread for Section 33.2, but flag it now as one more reason not to paper over gaps by default.
The better mental model is to keep the observations at their true timestamps and make the elapsed time a first-class input to the model, rather than pretending it away.
In Practice: the sepsis model that loved the night shift
A hospital team builds an early-warning model for clinical deterioration from vital-sign charts. Nurses record vitals irregularly: every fifteen minutes for an unstable patient, once a shift for a stable one. The team resamples everything to a clean hourly grid with forward-fill and gets excellent validation numbers. In deployment the model fires disproportionately at 04:00. The cause is not physiology; it is that stable patients are measured rarely overnight, so forward-fill stretches a single stale reading across hours, and the model learned that long flat runs, an artifact of low measurement frequency, correlate with the sicker patients who were, in truth, being measured more often. The fix was to feed the model the time since the last real measurement as an explicit feature and to stop imputing across long gaps. The measurement schedule, not the vital sign, had become the strongest predictor.
Modeling time as a first-class input
If you refuse to fake a grid, you owe the model the timing information directly. Three families of technique do this, in rising order of sophistication. The simplest is to append \(\Delta t\) as a channel: alongside each value, hand the network the elapsed time since the previous observation, so a recurrent or attention model can learn that a forty-minute gap means less certainty than a five-second one. The transformer positional machinery of Chapter 15 generalizes cleanly here by embedding continuous timestamps instead of integer positions.
The second family adds a principled decay of stale information. The insight, popularized by the GRU-D architecture, is that the older the last real reading, the more the model should trust a learned baseline over that reading. A standard choice is an exponential decay of the observed value toward a running mean,
$$ \gamma_k = \exp\!\big(-\tfrac{1}{\tau}\,\max(0,\,\Delta t_k)\big), \qquad \hat{x}_k = \gamma_k\, x_{\text{last}} + (1-\gamma_k)\,\bar{x}, $$where \(\tau\) is a learnable time constant per channel. A freshly observed value (\(\Delta t_k \to 0\)) passes through almost untouched; a value from many time constants ago decays gently toward the personal baseline \(\bar{x}\). This is a soft, uncertainty-aware alternative to a hard forward-fill, and it is close kin to the process-noise growth that inflates a Kalman filter's covariance between updates in Chapter 9. The third family models the trajectory in genuinely continuous time, evolving a latent state along a differential equation and only reading it out at the moments an observation actually landed; we return to its research status below.
import numpy as np
# Irregular, sparse heart-rate observations over one morning.
# Times are seconds since midnight; note the uneven, sometimes large gaps.
t = np.array([0, 5, 11, 18, 2400, 2405, 2412, 9000], dtype=float) # s
hr = np.array([58, 60, 59, 61, 72, 71, 73, 64], dtype=float) # bpm
baseline = hr.mean() # a crude personal baseline; 33.2 does this properly
tau = 600.0 # decay time constant: trust a reading for ~10 min
dt = np.diff(t, prepend=t[0]) # elapsed time since previous observation
gamma = np.exp(-np.maximum(dt, 0) / tau) # 1.0 when fresh, ->0 across long gaps
x_hat = gamma * hr + (1 - gamma) * baseline
for ti, g, xi in zip(t, gamma, x_hat):
print(f"t={ti:6.0f}s decay={g:4.2f} time-aware estimate={xi:5.1f} bpm")As Listing 33.1 shows, the elapsed-time decay does automatically what the sepsis team above had to discover the hard way: a value stops being trusted as it ages, and the estimate never pretends a nine-thousand-second-old reading is current. The same \(\Delta t\) that drives the decay is the feature you would also expose to a downstream classifier so it can calibrate its own confidence, a theme that connects to the conformal methods of Chapter 18.
The Right Tool
Hand-rolling a full irregular-series pipeline, masking, per-channel decay, a GRU-D or attention imputer, and batching sequences of unequal length, is comfortably 200-plus lines of careful PyTorch that is easy to get wrong at the mask boundaries. The pypots toolkit (Python toolbox for partially-observed time series) packages the standard models behind one API:
from pypots.imputation import SAITS
# X shape: (n_series, n_steps, n_features) with NaN where unobserved
model = SAITS(n_steps=48, n_features=6, n_layers=2, d_model=128, n_heads=4, d_k=32, d_v=32, d_ffn=128, epochs=10)
model.fit({"X": X_train})
X_filled = model.impute({"X": X_test}) # respects the observed maskpypots reduces a bespoke irregular-series imputation stack to a handful of lines and gives you SAITS, BRITS, and GRU-D behind one interface, collapsing roughly 200 lines to under ten. It handles the masking and unequal-length batching; deciding whether imputing at all is honest for your gaps, versus modeling the gap as signal, remains your call and is the subject of the next section.The convenience is genuine, and so is the responsibility: a toolkit will happily impute across a week-long gap where imputation is meaningless. Validate on held-out real observations, split by participant to avoid the leakage traps of Chapter 5, before trusting any filled value.
Multi-scale horizons and the observation clock as signal
Longitudinal data is not one time series; it is nested rhythms. A resting-heart-rate stream carries a per-minute jitter, a circadian dip during sleep, a weekly pattern tied to work and rest, and a multi-week trend that is often the actual clinical target. A model with a receptive field of a few hours cannot see the trend, and one that only looks at the monthly trend misses the acute event. Good longitudinal representations are explicitly multi-scale: they summarize each day into a small vector (nightly resting rate, daytime range, sleep-window features) and then model the sequence of daily summaries, so the long horizon becomes short again. This two-stage view, dense within a day, sparse across days, also sidesteps much of the sparsity problem, because a day with two hours of wear can still yield one honest daily summary rather than twenty-two imputed hours.
There is a final, subtler point. In opportunistic monitoring, when an observation exists is itself data. A watch samples more often when the wearer is active and still enough for a clean read; a patient is measured more often when clinicians are worried. The sequence of timestamps is therefore a marked point process whose intensity correlates with state, and models that treat the observation times as a random process to be explained, rather than a nuisance grid to be filled, can extract signal from the schedule. That is the bridge into Section 33.2, where the pattern of missingness graduates from nuisance to feature.
Research Frontier
Continuous-time deep models are the active front here. Latent ODE and ODE-RNN models (Rubanova and colleagues, 2019) evolve a hidden state along a learned differential equation and read it out at arbitrary observation times, sidestepping the grid entirely. Attention-based approaches such as the Multi-Time Attention Network (mTAND, Shukla and Marlin, 2021) and the imputation transformer SAITS (Du and colleagues, 2023) learn continuous time embeddings and reach strong results on irregular clinical benchmarks like PhysioNet and MIMIC. The linear-time state-space models of Chapter 16 are increasingly adapted to irregular inputs, and wearable foundation models (Chapter 20) are beginning to pre-train directly on months of sparse, real-world streams rather than curated clinical windows.
Exercise
You are given three months of wrist heart rate from one person, stored as (timestamp, value) pairs with roughly 35 percent wear time and highly uneven spacing. Sketch, in a short paragraph, a two-stage representation that (a) produces one honest feature vector per calendar day even on low-wear days, and (b) lets a downstream model detect a slow multi-week upward drift in resting rate. State one decision rule for when you would refuse to emit a daily vector at all, and explain why refusing is safer than imputing.
Self-Check
1. Irregular and sparse are orthogonal. Give a concrete health-monitoring example of a stream that is irregular but dense, and one that is regular but sparse.
2. Why can forward-filling across long gaps make the measurement schedule, rather than the physiology, the strongest predictor a model learns?
3. In the decay estimator \(\gamma_k = \exp(-\Delta t_k/\tau)\), what happens to the estimate as \(\tau \to \infty\), and as \(\tau \to 0\)? Which limit reproduces a naive forward-fill?
What's Next
In Section 33.2, we stop treating the gaps as a problem to be filled and start reading them as information. When a device stops recording, why it stopped often correlates with the very health state you are trying to infer, so the pattern of missingness becomes a feature in its own right, and the personal baseline we sketched crudely here gets built properly, per individual, so that "normal for you" replaces "normal on average."