Irregular and asynchronous sampling

"I was trained on a metronome and deployed at a jazz club. Nobody told the samples to arrive on the beat."

A Rhythmically Challenged AI Agent

This section assumes you understand sampling, jitter, and clock synchronization at the physical level from Chapter 3, and that you can build the recurrent and convolutional encoders from the earlier sections of this chapter. It also leans on the leakage-safe splitting discipline of Chapter 5, because irregular data offers new and creative ways to leak the future into the past.

Where irregularity comes from, and why you cannot ignore it

Irregular sampling is not one problem; it is a family. Event-driven sensors emit a reading only when something crosses a threshold, so quiet periods produce no samples at all, the extreme case being the event cameras of Chapter 46. Power-managed sensors duty-cycle: the device sleeps for a variable interval, so the gap itself encodes a scheduling decision. Lossy transport, typical of Bluetooth Low Energy wearables, drops packets under radio contention, leaving ragged holes in an otherwise uniform stream. Asynchronous multi-rate fusion combines a 100 Hz IMU, a 1 Hz GPS fix, and an event-triggered barometer, each on its own clock, so there is no single grid onto which all channels naturally fall. And human-triggered records, most clinical data, appear when a measurement was clinically warranted, which means the sampling pattern is itself a signal about patient state.

The tempting fix, resample everything to a uniform grid and pretend, is dangerous for two reasons. First, upsampling by interpolation fabricates measurements: a linear fill between two blood-pressure readings twelve hours apart invents a smooth trend the patient may never have had, and a forward-fill can carry a stale value across a state change. Second, the interpolation kernel can straddle the split boundary and leak future information backward, exactly the trap flagged in the forecasting discussion of Section 14.4. The gap length is often the most predictive feature you own, and resampling deletes it.

Three ways to make a model time-aware

The cheapest intervention is time as a feature: append the elapsed interval \(\Delta t_i = s_i - s_{i-1}\) (where \(s_i\) is the timestamp of sample \(i\)) as an extra input channel, and often the observation mask alongside it. A vanilla GRU or TCN can then learn to modulate its update on the gap. This is a one-line change and a strong baseline, though the model must discover the right functional form of the gap on its own.

The principled recurrent answer is a time-decayed state. The GRU-D and T-LSTM families make the hidden state forget as a function of elapsed time: between two observations the memory decays toward a learned baseline, fast for channels that go stale quickly (heart rate) and slow for those that persist (a chronic diagnosis). A common decay is

with the decayed state \(h_i^{-}\) fed into the usual gated update. When \(\Delta t_i\) is small, \(\gamma_i \approx 1\) and nothing changes; when the gap is long, the state relaxes toward its prior, which is exactly the behavior a Kalman prediction step exhibits between measurements in Chapter 9.

The most expressive answer models the hidden state as a continuous-time trajectory. ODE-RNNs and latent-ODE models evolve \(h(t)\) by a learned ordinary differential equation \(\mathrm{d}h/\mathrm{d}t = f_\theta(h)\) between observations, then jump the state at each arrival with a standard recurrent update. Because the dynamics are defined for every real \(t\), the model is genuinely grid-free: you can query it at any timestamp, interpolate and extrapolate within one framework, and handle wildly uneven spacing without a single imputed value. This continuous-time viewpoint connects directly to the linear-time state-space models of Chapter 16, several of which are discretizations of exactly this kind of continuous system.

import torch, torch.nn as nn

class GRUDecayCell(nn.Module):
    """Minimal GRU-D-style cell: hidden state decays with the elapsed gap."""
    def __init__(self, in_dim, hid):
        super().__init__()
        self.gru = nn.GRUCell(in_dim, hid)
        self.w = nn.Parameter(torch.zeros(hid))   # per-unit decay rate
        self.b = nn.Parameter(torch.zeros(hid))

    def forward(self, x_t, dt_t, h):
        # dt_t: (batch, 1) elapsed time since previous sample, in seconds
        gamma = torch.exp(-torch.relu(self.w * dt_t + self.b))  # (batch, hid)
        h_decayed = gamma * h                                   # forget by gap
        return self.gru(x_t, h_decayed)

cell = GRUDecayCell(in_dim=4, hid=16)
h = torch.zeros(2, 16)
# two irregular samples: a 0.1 s gap, then a 30 s gap
for x, dt in [(torch.randn(2, 4), torch.full((2, 1), 0.1)),
              (torch.randn(2, 4), torch.full((2, 1), 30.0))]:
    h = cell(x, dt, h)
print(h.shape)  # torch.Size([2, 16])

The code above shows the decay applied to two samples spaced 0.1 s and 30 s apart; the second update starts from a heavily faded memory, which is the whole point.

Asynchronous multi-sensor streams, and the leakage traps

When several channels each fire on their own clock, you have two structural choices. Union-of-timestamps builds one merged sequence over every arrival time across all sensors; at each row, exactly one channel (or a few) is real and the rest are masked, and per-channel \(\Delta t\) tells the model how stale each masked entry is. This preserves the true event order and is the natural input for a decay or ODE model. Set-based encoders drop the grid entirely and treat the record as an unordered set of (time, channel, value) tuples, which is the door into the attention-based sensor models of Chapter 15, where a learned positional encoding of the timestamp replaces the grid outright.

Two evaluation traps deserve a warning label. First, any statistic computed for imputation, a channel mean, a decay baseline, a normalization scale, must be estimated on the training split only, or the test set's future bleeds into training exactly as it would for the continuous-health monitors of Chapter 33. Second, and subtler: if the sampling times themselves are correlated with the label (a lab ordered because the clinician already suspected the outcome), then the timestamp pattern can be a shortcut feature that will not exist at prediction time. Audit whether \(\Delta t\) is a legitimate predictor or a leak of the very decision you are trying to anticipate.

What's Next

In Section 14.7, we assemble everything from this chapter, the recurrent and convolutional cores, the streaming state, and the irregular-time inputs, into practical training recipes: how to window, normalize, batch ragged sequences, schedule learning rates, and regularize sensor models so they actually train stably and survive contact with a live deployment.