Graph-based process modeling (GNNs)

"A per-sensor model asked whether each tag was surprising on its own. The graph asked a harder question: does this tag still agree with the neighbours it is wired to? Half my false alarms were tags that were merely lonely, not lying."

A Relationally-Minded AI Agent

Why the graph, and where the edges come from

The premise is that anomalies in a cyber-physical process are relational. When an adversary spoofs a level transmitter to a plausible constant, the reading itself is unremarkable; what betrays the attack is that the level no longer moves the way the inflow and the pump command say it should. To detect that, the model must represent inter-sensor dependence explicitly. Stacking all channels into one vector and feeding a dense network can in principle learn these couplings, but it wastes capacity modelling every pair equally and offers no handle on which pair broke. A graph makes the dependence structure a first-class object: node \(i\) is sensor \(i\), and a directed edge \(j \to i\) says "sensor \(j\) helps predict sensor \(i\)."

Where do the edges come from? Three sources, in increasing order of how much a modern detector leans on them. You can hand-build the graph from the piping-and-instrumentation diagram, wiring tags that share a physical pipe or control loop; this is faithful but laborious and brittle to undocumented couplings. You can derive it from data by correlation or mutual information, thresholding a similarity matrix. Or, in the approach that now dominates, you can learn the graph jointly with the detector. Each sensor gets a trainable embedding vector \(\mathbf{v}_i \in \mathbb{R}^d\), and the affinity between two sensors is their embedding similarity; for every node you keep only its top-\(k\) most similar candidates as incoming edges,

This is the core idea of the Graph Deviation Network (GDN), and it means the model discovers the plant's dependency structure from healthy data instead of demanding a schematic. The learned graph is also a deliverable in its own right: an engineer can inspect the top edges and confirm that stage-three level really does depend on the stage-two flow, a sanity check no black-box detector offers.

Message passing, forecasting, and the deviation score

With a graph in hand, a detection step runs in three moves. First, embed: each node's recent window \(\mathbf{x}^{(i)}_{t-w:t-1}\) is encoded into a feature vector. Second, pass messages: every node aggregates its neighbours' features, weighted by learned attention, producing a representation that fuses a sensor with the tags it depends on. A single graph-attention layer computes, for node \(i\),

where \(\mathbf{g}_j\) fuses sensor \(j\)'s window with its embedding \(\mathbf{v}_j\) and the attention weight \(\alpha_{ij}\) says how much node \(i\) trusts neighbour \(j\) right now. Third, predict: a small output head maps \(\mathbf{h}_i\) to the forecast \(\hat{x}^{(i)}_t\) for the next value of that sensor. Train the whole stack (embeddings, attention, head) to minimize forecasting error on a contiguous clean run, and the per-sensor deviation is the familiar residual \(r^{(i)}_t = \lvert x^{(i)}_t - \hat{x}^{(i)}_t\rvert\), normalized per channel by its healthy median and inter-quartile range so no single loud tag dominates. The plant-wide score at time \(t\) is the maximum normalized deviation across sensors, and the tag achieving that maximum is the model's first guess at the culprit. Thresholding that score reuses the calibration of Section 38.3 exactly: a high quantile on a held-out clean stream, smoothed to demand sustained rather than single-sample exceedance.

import torch, torch.nn.functional as F

class GraphAnomalyDetector(torch.nn.Module):
    def __init__(self, n_sensors, window, emb_dim=64, topk=15):
        super().__init__()
        self.topk = topk
        self.embed = torch.nn.Embedding(n_sensors, emb_dim)   # learned node embeddings
        self.enc   = torch.nn.Linear(window, emb_dim)          # per-sensor window encoder
        self.attn  = torch.nn.Linear(2 * emb_dim, 1)           # graph-attention scorer
        self.head  = torch.nn.Linear(emb_dim, 1)               # one-step forecast head

    def learned_edges(self):                                   # top-k cosine graph
        v = F.normalize(self.embed.weight, dim=1)
        sim = v @ v.t()
        sim.fill_diagonal_(-1e9)                               # no self-loop in the ranking
        return sim.topk(self.topk, dim=1).indices              # [n_sensors, topk]

    def forward(self, x):                                      # x: [batch, n_sensors, window]
        g = torch.relu(self.enc(x)) + self.embed.weight        # fuse window with node identity
        nbr = self.learned_edges()                             # who each node listens to
        msg = g[:, nbr]                                        # [batch, n_sensors, topk, emb]
        pair = torch.cat([g.unsqueeze(2).expand_as(msg), msg], dim=-1)
        a = torch.softmax(self.attn(pair).squeeze(-1), dim=-1) # attention over neighbours
        h = torch.relu((a.unsqueeze(-1) * msg).sum(dim=2))     # aggregated neighbour message
        return self.head(h).squeeze(-1)                        # x_hat: [batch, n_sensors]

Listing 38.4 shows why the graph is not just an architectural flourish. The learned_edges method returns, for every sensor, the handful of tags it consults, and that structure is what makes the detector both sharper on relational attacks and readable afterward: when the deviation on a level transmitter spikes, the neighbours in its row tell you which upstream tags it should have agreed with and did not.

from torch_geometric.nn import GATConv
gat = GATConv(in_channels=emb_dim, out_channels=emb_dim, heads=4)
h = gat(node_feats, edge_index)   # edge_index: [2, E] from the learned top-k graph

What the graph buys, and what it costs

The gain is concentrated where per-tag detectors are weakest. On a spoof pinned to a plausible value, the target channel forecasts fine on its own history but wildly given its neighbours, so its normalized deviation spikes even though a univariate model stays flat. Because the score localizes to the offending node, the graph detector feeds root-cause analysis (Section 38.5) directly, and its learned edges align with interpretability tooling for time series (Chapter 67). The costs are real and worth naming. A learned graph can overfit spurious correlations in a short clean run, wiring sensors that co-move by coincidence; a top-\(k\) that is too large dilutes attention, too small starves nodes of context. And the graph is still learned only from normal data, so a coordinated multi-point attack that moves several coupled sensors together in a way the graph considers normal can slip through, the relational cousin of the replay blind spot. The honest evaluation discipline from Section 38.3 carries over without change: report event-level detection at a fixed clean false-alarm budget and median detection delay on a chronological split, never point-adjusted F1 in isolation, and never tune the graph or threshold on the attack set.

What's Next

In Section 38.5, we turn the detector's finger-pointing into a discipline. A graph model already nominates the node with the largest deviation and the neighbours it disagreed with, but a single tamper propagates through the plant and lights up a cluster of downstream tags, so the loudest alarm is not always the origin. Root-cause localization walks the learned dependency structure backward, separates the primary fault from its physical consequences, and hands the operator not just "something is wrong" but "it started here."