Supervised vs unsupervised condition monitoring

"You asked me to learn what a broken machine looks like. I have seen exactly one, three years ago, and it was labeled 'maybe.'"

An Underlabeled AI Agent

Two framings of the same alarm

Fix the object of study: a stream of sensor windows \(x_1, x_2, \ldots\) from a machine, and a wish to raise an alarm when the machine is not healthy. There are two mathematically distinct ways to formalize that wish.

The supervised framing treats it as classification. You assume a labeled dataset \(\{(x_i, y_i)\}\) with \(y_i \in \{\text{healthy}, \text{fault}_1, \text{fault}_2, \ldots\}\) and learn a decision boundary \(p(y \mid x)\) that separates the classes. This is the familiar setting, and when it applies it is powerful: it can distinguish a bearing fault from a gear fault, name the failure mode, and exploit whatever subtle features discriminate them.

The unsupervised (more precisely, semi-supervised or one-class) framing treats it as novelty detection. You assume access only to normal data, estimate a model of the healthy operating distribution \(p_{\text{normal}}(x)\), and define an anomaly score \(s(x)\) that grows as \(x\) becomes less plausible under that model. An alarm fires when \(s(x)\) crosses a threshold \(\tau\). You never show the model a fault during training; you show it only what "well" looks like, and let anything unfamiliar trip the wire. The terminology is loose in the literature: purely unsupervised methods assume the training set is mostly-but-not-entirely normal (a small unknown contamination fraction), while semi-supervised methods assume the training set is certified clean. Both share the defining move of learning the normal class alone.

Why unsupervised usually wins in the field

Three structural facts about industrial data push the choice toward the one-class branch, and it is worth naming each because each has a different remedy if you try to fight it.

Label scarcity. A well-maintained pump may run for years between failures. To assemble a balanced supervised dataset you would need to observe, correctly diagnose, and precisely time-stamp many faults across many units, and industrial maintenance logs are notoriously coarse ("replaced bearing, Tuesday") when they exist at all. Healthy data, by contrast, is nearly free: it accumulates every second the machine runs. The natural dataset is enormous on one side and empty on the other.

The open world of faults. Even a complete catalog of past faults is not a catalog of future ones. New failure modes appear as machines age, as loads change, as a supplier quietly alters a component. A supervised classifier trained on modes \(\{1, 2, 3\}\) has no defined behavior on mode \(4\); it will confidently assign the novel fault to whichever known class it superficially resembles, often "healthy." Novelty detection has no such blind spot because it never enumerated the faults in the first place.

Extreme class imbalance and cost asymmetry. If faults are, say, one window in ten thousand, the base rate is \(10^{-4}\). A classifier that always predicts "healthy" scores 99.99% accuracy while catching nothing, so accuracy is a useless target and even a well-trained classifier is pulled hard toward the majority class. Worse, the costs are wildly asymmetric: a missed catastrophic failure can cost orders of magnitude more than a false alarm, yet the data volume rewards the opposite bias. This is the same base-rate reasoning that governs medical screening and fraud detection, and it is why raw accuracy is banned from this chapter's vocabulary; the honest metrics are developed in Chapter 65 and specialized to time series in Section 37.7.

The normal-only model and its Achilles heel: contamination

The one-class recipe is disarmingly simple to state. Collect windows you believe are healthy, fit a density or boundary that encloses them, and score new windows by how far outside they fall. The score can be a reconstruction error (Section 37.4), a distance in a learned embedding, an isolation depth, or an explicit likelihood. What all these share is a single fragile assumption: the training set really is normal.

It rarely is. Field data labeled "healthy" is healthy only in the sense that nobody logged a repair; slow degradation, transient faults, and sensor glitches leak in. This contamination is corrosive because the model learns to treat the contaminating anomalies as normal, raising the threshold and hiding exactly the events you care about. Two defenses matter. First, quantify and cap contamination: robust one-class methods take a nominal contamination fraction \(\nu\) and fit a boundary that deliberately excludes the worst \(\nu\) of the training points, so a few bad windows do not drag the boundary outward. Second, respect operating-condition structure. Much of what looks anomalous is simply a legitimate but under-sampled regime (cold start, high load, a different product being milled), so a "normal" model must either cover every regime or be conditioned on the regime, a distinction that becomes the central theme of domain shift in Section 37.6. Building the clean, regime-aware normal set is a data-engineering problem, and the leakage rules of Chapter 5 apply with full force: a healthy window from the same machine and hour must not straddle the train/test split.

When to reach back for supervision

None of this makes supervision useless; it makes it a second tool, deployed when its preconditions are actually met. Reach for a supervised or hybrid approach when (a) you have enough labeled examples of a specific, recurring, high-cost failure that naming it changes the maintenance action, (b) the failure modes are genuinely closed-world and safety-critical, so an unnamed anomaly is not an acceptable output, or (c) you can obtain even a handful of labeled faults and want to calibrate an otherwise unsupervised detector's threshold against real events rather than guesswork.

The productive middle ground is a spectrum, not a binary. One-class detection raises the alarm; a lightweight supervised classifier, trained on the fault modes you do have, runs only on flagged windows to attach a name and a likely action. Few-shot and first-shot methods (the subject of Section 37.3) push into the regime of one or zero fault examples per machine. And self-supervised representation learning (Chapter 17) sidesteps the label problem entirely by learning a feature space from unlabeled healthy data, on top of which even a simple distance becomes a strong anomaly score. The design question is never "supervised or not" in the abstract; it is "how many trustworthy fault labels do I have, of how many distinct modes, and does naming the mode change what the operator does."

The listing below makes the core argument concrete. It simulates a fleet dominated by healthy windows with two fault modes, trains a supervised classifier that sees only mode A during training, and compares it against a one-class detector on a test set that also contains an unseen mode B. The classifier's recall collapses on the novel mode; the detector's does not.

import numpy as np
from sklearn.ensemble import IsolationForest, RandomForestClassifier

rng = np.random.default_rng(0)

def sample(kind, n):                          # 8-dim health-indicator vectors
    if kind == "normal":  return rng.normal(0.0, 1.0, (n, 8))
    if kind == "faultA":  return rng.normal(3.0, 1.0, (n, 8))   # known mode
    if kind == "faultB":  return rng.normal([0,0,0,0,0,0,4,4], 1.0, (n, 8))  # novel mode

# Training: healthy-heavy, and the supervised model sees ONLY fault mode A.
Xn, Xa = sample("normal", 4000), sample("faultA", 40)
X_train = np.vstack([Xn, Xa])
y_train = np.r_[np.zeros(len(Xn)), np.ones(len(Xa))]           # 0 = healthy, 1 = fault

clf = RandomForestClassifier(class_weight="balanced", random_state=0).fit(X_train, y_train)
iso = IsolationForest(contamination=0.01, random_state=0).fit(Xn)   # one-class: normal only

# Test: healthy + the KNOWN mode A + an UNSEEN mode B.
X_test = np.vstack([sample("normal", 2000), sample("faultA", 100), sample("faultB", 100)])
is_fault = np.r_[np.zeros(2000), np.ones(200)].astype(bool)
is_novel = np.r_[np.zeros(2100), np.ones(100)].astype(bool)   # the mode-B slice

clf_fault  = clf.predict(X_test) == 1
iso_fault  = iso.predict(X_test) == -1                        # -1 = anomaly

recall = lambda flag, mask: flag[mask].mean()
print(f"supervised recall on KNOWN faultA : {recall(clf_fault, is_fault & ~is_novel):.2f}")
print(f"supervised recall on NOVEL faultB : {recall(clf_fault, is_novel):.2f}")
print(f"one-class  recall on NOVEL faultB : {recall(iso_fault, is_novel):.2f}")

As Listing 37.1 shows, the gap is not a tuning artifact; it is structural. The classifier cannot generalize to a fault mode absent from its training labels, while the detector never needed the fault at all. That single asymmetry, replayed across thousands of real deployments, is why condition monitoring leans unsupervised.

from pyod.models.ecod import ECOD          # parameter-free, calibrated tail scores
det = ECOD(contamination=0.01).fit(X_normal)
scores = det.decision_function(X_test)      # higher = more anomalous, ready to threshold