Isolation Forest, LOF, one-class SVM

"You spent a week teaching me what a fault looks like. I found it faster by learning only what normal looks like, and flinching at everything else."

An Unsupervised AI Agent

These three methods share one framing: they are trained on data assumed to be mostly or entirely normal, and at inference they emit a continuous anomaly score rather than a hard label. You still need a threshold, and choosing it well is its own discipline covered in Section 12.7. Throughout, \(x \in \mathbb{R}^d\) is a feature vector, usually the windowed statistical and spectral features of Chapter 8, and the training set \(\{x_i\}_{i=1}^n\) is drawn from routine operation. Because all three learn "normal" from data, they inherit every leakage hazard of Chapter 5: if a fault sneaks into the training window, the model learns to call it normal.

Isolation Forest: anomalies are cheap to isolate

Isolation Forest (Liu, Ting, and Zhou, 2008) inverts the usual density-estimation instinct. Instead of modeling where the data is dense, it asks how hard each point is to isolate by random partitioning. Build a tree by repeatedly picking a random feature and a random split value between that feature's current min and max, recursing until every point sits alone in a leaf. A point in a dense cluster needs many splits to be cornered; an outlier sitting far out on some axis gets carved off after only a few. The path length \(h(x)\), the number of edges from root to the leaf holding \(x\), is therefore short for anomalies and long for inliers. Average over an ensemble of such random trees and you get a stable estimate \(E[h(x)]\).

The score normalizes path length against \(c(n)\), the average path length of an unsuccessful search in a binary tree of \(n\) points, so that scores are comparable across sample sizes:

where \(H(i)\) is the \(i\)-th harmonic number. A score near 1 means "isolated almost immediately," a strong anomaly; near 0.5 means "as hard to isolate as a typical point." The method's appeal for sensor fleets is practical: it is near-linear in \(n\), trivially parallel across trees, needs no distance metric or feature scaling (splits are axis-aligned and scale-invariant), and handles the high-dimensional feature vectors that vibration and current signatures produce without choking. Its blind spot is the flip side of axis-aligned splits: it struggles with anomalies defined only by a correlation between features rather than an extreme value on any single axis. The Extended Isolation Forest (Hariri et al., 2019) uses random-slope hyperplane cuts to soften that limitation.

Local Outlier Factor: weird relative to your neighbors

Isolation Forest and most statistical detectors judge a point against the global distribution. That fails when normal operation has regions of genuinely different density: an HVAC chiller idling, ramping, and running at full load produce three legitimate clusters of very different tightness. A point on the sparse edge of the idle cluster might be perfectly healthy, while a point of identical global rarity inside the dense full-load cluster is a real fault. Local Outlier Factor (Breunig et al., 2000) makes rarity local. For each point it finds the \(k\) nearest neighbors, defines a smoothed local reachability density \(\mathrm{lrd}_k(x)\) (roughly the inverse of the average distance to those neighbors), and compares your density to theirs:

An \(\mathrm{LOF}\) near 1 means you are about as dense as your neighbors: an inlier. Substantially above 1 means your neighborhood is much denser than you are, so you sit in a relative void: an outlier. Because the comparison is only ever against nearby points, LOF adapts to varying density automatically, which is its whole reason to exist. The costs are equally characteristic. It is distance-based, so it demands careful feature scaling (a feature in millivolts will drown one in kilohertz) and it degrades in high dimensions where distances concentrate, both pushing you toward the dimensionality reduction of Chapter 8 first. Naive LOF is also \(O(n^2)\) to score all points, and the choice of \(k\) matters: too small and it chases noise, too large and local structure washes out.

One-class SVM: draw a boundary around normal

The third principle is geometric. A one-class SVM (Schölkopf et al., 2001) maps the data into a high-dimensional feature space through a kernel, usually the radial basis function \(k(x, x') = \exp(-\gamma \lVert x - x' \rVert^2)\), and finds the hyperplane that separates the data from the origin with maximum margin. Points falling on the origin side score as anomalies. An equivalent and often more intuitive formulation, Support Vector Data Description (Tax and Duin, 2004), fits the smallest hypersphere enclosing the normal data; anything outside is anomalous. The key knob is \(\nu \in (0, 1]\), which simultaneously upper-bounds the fraction of training points allowed outside the boundary and lower-bounds the fraction of support vectors. Set \(\nu = 0.05\) and you are asserting that at most about 5 percent of training data are contaminants, a direct handle on your assumed anomaly rate.

One-class SVM shines when normal operation occupies a compact, curved region that a good kernel can wrap tightly, and it produces a genuine decision boundary you can reason about. Its weaknesses are real: the RBF bandwidth \(\gamma\) is finicky and interacts with \(\nu\), training scales poorly past tens of thousands of points, and it is sensitive to feature scaling and to any contamination in the training set. On messy, high-volume fleet telemetry it is frequently the least robust of the three out of the box, which is why it is often kept as a boundary-aware complement rather than the primary detector.

The comparison code below fits all three on the same feature matrix and lines up their scores, which is exactly how you should start any new deployment: run all three, look at where they agree and disagree, and let that structure the investigation.

import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor
from sklearn.svm import OneClassSVM
from sklearn.preprocessing import StandardScaler

# X_train: healthy windows (n, d).  X_test: recent windows to score.
scaler = StandardScaler().fit(X_train)          # matters for LOF and OCSVM
Xtr, Xte = scaler.transform(X_train), scaler.transform(X_test)

iforest = IsolationForest(n_estimators=200, contamination=0.02,
                          random_state=0).fit(Xtr)
ocsvm   = OneClassSVM(kernel="rbf", gamma="scale", nu=0.02).fit(Xtr)
lof     = LocalOutlierFactor(n_neighbors=20, novelty=True).fit(Xtr)

# Higher = more anomalous for all three (sign-flipped so they align).
scores = {
    "iforest": -iforest.score_samples(Xte),
    "ocsvm":   -ocsvm.score_samples(Xte),
    "lof":     -lof.score_samples(Xte),
}
consensus = np.mean([(s > np.quantile(s, 0.98)) for s in scores.values()], axis=0)
flagged = np.where(consensus >= 2/3)[0]          # >=2 of 3 detectors agree

What's Next

In Section 12.6, we confront the uncomfortable truth lurking under every score in this section: with few or no labels, how do you even know your detector is any good? We will build honest evaluation protocols for partially labeled anomaly data and dismantle the point-adjust trap, a scoring convention that has quietly inflated a decade of time-series anomaly-detection results.