Sequence models for degradation

"I fed it the last reading and it guessed the age of the machine. I fed it the last fifty readings and it guessed how the machine was aging. Only the second guess was worth having."

A Recurrent AI Agent

Framing RUL as a sequence-to-target problem

The core object is a sliding window. From a run-to-failure record of length \(T\), you extract fixed-length windows \(X_t = (x_{t-L+1}, \dots, x_t)\) of the last \(L\) timesteps, where each \(x\) is a vector of sensor channels, and you regress the scalar remaining useful life \(y_t\) at the window's end. The model learns a function \(f_\theta: \mathbb{R}^{L \times C} \rightarrow \mathbb{R}\). Why a window rather than a single reading? Because the informative content of degradation lives in the rate and shape of change, not the absolute level. Two engines can share an identical vibration amplitude while one is stable and the other is accelerating toward failure; only the trajectory distinguishes them. The window length \(L\) is the model's memory budget: too short and it cannot see the trend, too long and early windows straddle multiple operating regimes and dilute the signal. On the canonical C-MAPSS turbofan benchmark, windows of 30 to 50 cycles are standard.

The labeling choice is where most first attempts quietly break. The naive label is linear: if a unit fails at cycle \(T\), then \(y_t = T - t\). This asserts that a machine at 80 percent of life is measurably older than one at 79 percent, which is false. Early in life a machine is healthy and its sensors carry no degradation signal, so a linear label forces the network to hallucinate a countdown from noise. The fix is the piecewise-linear RUL target: clamp the label to a constant ceiling \(R_{\max}\) during the healthy phase, and let it decline linearly only once wear begins.

The ceiling \(R_{\max}\) (commonly 125 cycles on C-MAPSS) encodes a simple truth: beyond some horizon the machine is just healthy, and "very healthy" and "extremely healthy" are the same prediction. This one change routinely moves error more than any architecture swap.

Which sequence architecture, and when

Three families dominate degradation modeling, and the right pick follows from the physics of the signal. LSTMs and GRUs (Chapter 14) carry a hidden state that accumulates slowly, which matches monotone wear well and makes them the reliable default on modest datasets like C-MAPSS. Their weakness is throughput: the recurrence is sequential, so long windows train slowly. Temporal convolutional networks replace recurrence with dilated causal convolutions; they see a fixed receptive field in parallel, train fast, and excel when the degradation cue is a local morphological change (a growing sideband, a widening pulse) rather than a slow global drift. Transformers (Chapter 15) use attention to relate distant timesteps directly, which helps when the failure signature couples events far apart in the window, though on the small labeled fleets typical of prognostics they need careful regularization to beat a well-tuned GRU. For very long, high-rate windows the linear-time state-space models of Chapter 16 are increasingly the throughput-versus-accuracy sweet spot.

Whatever the backbone, two design touches matter more than the choice between them. First, an asymmetric loss: in maintenance, a late prediction (you thought there was more life than there was) is far costlier than an early one, because it means an unplanned failure. The C-MAPSS scoring function bakes this in with an exponential penalty that is steeper for overestimates. Second, a monotonicity or smoothness prior on the predicted health trajectory, because real degradation rarely reverses; penalizing upward jumps in successive RUL predictions removes the physically absurd "the engine got younger" artifact that raw per-window regression produces.

import torch, torch.nn as nn

def piecewise_rul(T, t, r_max=125):
    "Clamp remaining-useful-life label to a healthy-phase ceiling."
    return torch.clamp(T - t, max=r_max).float()

class GRUProg(nn.Module):
    def __init__(self, n_channels, hidden=64):
        super().__init__()
        self.gru = nn.GRU(n_channels, hidden, num_layers=2, batch_first=True)
        self.head = nn.Sequential(nn.Linear(hidden, 32), nn.ReLU(), nn.Linear(32, 1))

    def forward(self, x):            # x: (batch, L, n_channels)
        _, h = self.gru(x)           # h: (layers, batch, hidden)
        return self.head(h[-1]).squeeze(-1)   # RUL at end of window

def asymmetric_loss(pred, target, late=1.3):
    "Penalise under-prediction (late warning) harder than over-prediction."
    err = pred - target
    w = torch.where(err < 0, late, 1.0)       # err<0 means predicted too low = late
    return (w * err.pow(2)).mean()

model = GRUProg(n_channels=14)
x = torch.randn(32, 30, 14)                   # 32 windows, L=30, 14 sensors
rul = model(x)                                # one RUL estimate per window
loss = asymmetric_loss(rul, torch.full((32,), 40.0))

The listing above is the whole conceptual skeleton: a recurrent backbone reads the window, a small head emits one RUL value, and the loss encodes the business asymmetry. In practice you also normalize each sensor within its operating condition, because a turbofan at altitude and at sea level produce different baselines that would otherwise swamp the wear signal.

Learning a health index, not just a number

Regressing RUL directly works, but the more interpretable and often more robust route is to have the sequence model emit a scalar health index \(h_t \in [0, 1]\): a learned one-dimensional summary that starts near 1 on a healthy machine and decays toward 0 at failure. You then map \(h_t\) to RUL with a simple, auditable curve, or trigger maintenance when \(h_t\) crosses a threshold. The health index is attractive for three reasons. It is comparable across machines even when their absolute sensor levels differ; it exposes the degradation trajectory to an operator as a single trend line they can sanity-check; and it decouples the hard perception problem (read the sensors, place the machine on the degradation manifold) from the easier, changeable business problem (how much life does this health level imply for this asset class). When degradation is multi-modal, one bearing fault and one lubrication fault with different signatures, you can even emit a small vector of health indices, one per known failure mode, which connects directly to the interpretability and root-cause methods of Chapter 67.

Evaluating and trusting the prediction

A sequence prognostic model is only as good as its evaluation split. The non-negotiable rule is that no window from a given machine may appear in both training and test; you split by unit, never by window, or the model scores brilliantly by recognizing engines it has already seen dying. Report both a symmetric metric (RMSE, so results compare across papers) and the asymmetric operational score (which reflects real cost), because a model can win one and lose the other. Finally, inspect the predicted trajectory per unit, not just the aggregate error: a good model produces a health index that is flat during the healthy phase and then declines smoothly and monotonically, and the shape of that curve tells you whether the network learned degradation or learned the dataset. The measurement-first humility that opened this book applies at the end too: the prediction is a lossy inference about a physical process, and its value is set by whether a maintenance crew can act on it in time.

What's Next

In Section 36.5, we lift the ceiling on data hunger: instead of training a fresh sequence model on each fleet's scarce failure records, we use pretrained time-series and sensor foundation-model embeddings as the front end for RUL, transferring degradation structure learned across many machines into settings where you have only a handful of run-to-failure examples of your own.