"I was ninety percent sure, so I answered. Nobody had told me that the other ten percent was where the patient lived."
A Chastened AI Agent
Prerequisites
This section turns the uncertainty numbers of the rest of the chapter into decisions, so you should be comfortable with the calibrated confidences of Section 18.2 and the coverage-guaranteed prediction sets of Section 18.4. It assumes the aleatoric/epistemic split from Section 18.1, because the two uncertainties trigger different responses here. Familiarity with expected-value decision rules from the probability primer in Chapter 4 is enough for the cost arithmetic; nothing here needs the drift machinery of Section 18.5, though it composes cleanly with it.
The Big Picture
Everything earlier in this chapter produced a better-calibrated number. This section is where the number finally does something. A sensor model that must answer on every input is a model with no brakes: on the one bad window in a thousand where it is confidently wrong, it hands that wrong answer straight to a controller, a clinician, or a safety interlock. Abstention gives the model a third option beyond "class A" and "class B": the option to say "I do not know, ask someone else." Fallback decides who that someone else is when the model steps aside: a slower classical estimator, a conservative default, a human. Safety margins decide how far to inflate the interval before you trust it to act. Together they convert an uncertainty estimate into an operating policy, and that policy, not the raw accuracy, is what a functional-safety reviewer signs off on. Get it right and you trade a small, controlled amount of coverage for a large, provable drop in the error rate on the answers you do give.
The reject option and the risk-coverage tradeoff
A selective predictor is a pair \((f, g)\): a base model \(f(x)\) that predicts, and a gating function \(g(x) \in \{0, 1\}\) that decides whether to answer. When \(g(x) = 0\) the system abstains. Two quantities describe its behavior. Coverage is the fraction of inputs it chooses to answer, \(\phi = \mathbb{E}[g(x)]\). Selective risk is the error rate on the answered subset only,
\[ R(f, g) = \frac{\mathbb{E}\big[\,\ell(f(x), y)\,g(x)\,\big]}{\mathbb{E}[g(x)]}. \]What the reject option buys you is a knob: as you abstain on more inputs (coverage drops), the errors concentrate in the rejected pile and selective risk falls. Why it works is that a good confidence score is monotone in error probability, so thresholding it removes the riskiest inputs first. How you tune it is the risk-coverage curve: sweep the threshold, plot selective risk against coverage, and read off the operating point your application can live with. The classical result behind this is the Chow rule (Chow, 1970): for a calibrated posterior, the risk-optimal gate rejects exactly when the maximum class probability falls below a threshold set by the cost of an error relative to the cost of abstaining. Everything modern in selective classification is a robust approximation to that rule when the posterior is not perfectly calibrated.
Key Insight
Abstention is only as trustworthy as the score you gate on, and softmax confidence is the wrong score for sensors. A network extrapolating onto an out-of-distribution window (a new user, an unlogged fault) is frequently confident and wrong at once, so a softmax gate keeps answering exactly when it should stop. This is why the aleatoric/epistemic split of Section 18.1 matters operationally: gate on the epistemic term, not the total. High aleatoric uncertainty means "the world is noisy here, widen the margin and proceed"; high epistemic uncertainty means "I have left the map, abstain." A gate that cannot tell these apart abstains on merely noisy inputs it could have handled and charges ahead on novel inputs it could not. The conformal prediction set of Section 18.4 is a natural gate here: abstain when the set is too large (ambiguous) or, for a distance-based nonconformity score, when the input is too far from the calibration data to trust at all.
From confidence to a cost-aware decision
A threshold is not a hyperparameter to grid-search blindly; it is the solution to a cost equation. Assign a cost to each outcome: \(c_{\text{err}}\) for a wrong answer, \(c_{\text{abstain}}\) for invoking the fallback, and (usually) \(0\) for a correct answer. Answering on input \(x\) beats abstaining exactly when the expected cost of answering is lower:
\[ \underbrace{(1 - p_{\max}(x))\,c_{\text{err}}}_{\text{expected cost of answering}} \;<\; \underbrace{c_{\text{abstain}}}_{\text{cost of fallback}}, \]which rearranges to the Chow threshold \(p_{\max}(x) > 1 - c_{\text{abstain}}/c_{\text{err}}\). Read it literally: the more a wrong answer costs relative to a fallback, the higher you set the confidence bar, and the more often you abstain. This is why a fall-detection wearable and a music-recommendation model with the same accuracy operate at completely different thresholds: the fall detector's \(c_{\text{err}}\) is a missed emergency, so it abstains readily into a "ask the user" fallback, while the recommender's error costs almost nothing and it answers on everything. The threshold encodes the application's values, and it belongs in a config reviewed by the safety owner, not buried in a notebook.
import numpy as np
def risk_coverage_curve(scores, correct):
"""scores: per-sample confidence (higher = more trustworthy).
correct: 1 if the base model was right, else 0.
Returns coverage and selective risk as the threshold sweeps."""
order = np.argsort(-scores) # answer highest-confidence first
correct = correct[order]
n = len(scores)
coverage = np.arange(1, n + 1) / n
cum_err = np.cumsum(1 - correct) # errors among the answered set
selective_risk = cum_err / np.arange(1, n + 1)
return coverage, selective_risk
def chow_threshold(c_err, c_abstain):
"""Risk-optimal confidence threshold for a calibrated posterior."""
return 1.0 - c_abstain / c_err # answer iff p_max > threshold
# Example: an error costs 50x a fallback -> abstain unless very confident
tau = chow_threshold(c_err=50.0, c_abstain=1.0) # -> 0.98
cov, risk = risk_coverage_curve(scores, correct)
# Pick the operating point where selective_risk first drops below the SLA.
target = 0.01
op = np.argmax(risk <= target) if (risk <= target).any() else -1
print(f"gate at p_max > {tau:.3f}; SLA-feasible coverage = {cov[op]:.2%}")Listing 18.6 makes the two ways of choosing a threshold concrete. If you know the costs, use chow_threshold directly. If instead you have a hard error budget (a regulator says "at most one percent error on delivered predictions"), sweep the risk-coverage curve and take the largest coverage whose selective risk still meets the budget. Either way, the number you ship is defensible, because it traces back to a stated requirement rather than a lucky validation run.
In Practice: an ICU arrhythmia monitor that knows when to page a human
A bedside monitor classifies each ten-second ECG window as normal, atrial fibrillation, or one of several ventricular arrhythmias, streaming from the cardiac models of Chapter 29. Alarm fatigue is the enemy: too many false alerts and nurses mute the device, so a confident wrong answer is genuinely dangerous. The team gates on a conformal nonconformity score rather than softmax. When the prediction set is a clean singleton the monitor acts autonomously. When the set contains two rhythms, or the input's nonconformity exceeds the calibration range (a lead has come loose, producing a waveform unlike anything in the training data), the monitor abstains and its fallback fires in tiers: first a classical threshold-based QRS detector (Chapter 12) that at least flags asystole reliably, and if that too is uncertain, a page to the on-call cardiologist. During a validation month the abstention rate sat near four percent, but selective error on the ninety-six percent it did classify fell below the alarm-fatigue threshold the hospital had set, and every dangerous miss in the old always-answer system landed in the abstain-and-page tier instead of a silent misclassification. The reject option did not make the model smarter; it made the system safe to deploy.
Fallback hierarchies and safety margins
Abstaining is only half a policy; you must say what happens next. A good fallback hierarchy is ordered by decreasing capability and increasing trust. The deep model is fastest and most capable but least trusted at the edges of its distribution. Below it sits a classical estimator (a Kalman filter from Chapter 9, a hand-tuned threshold, a physics model) that is less accurate on average but degrades gracefully and predictably out of distribution. Below that sits a fixed safe default (hold the last actuator command, enter a limp-home mode, freeze and alert) and, at the bottom, a human. Why this ordering: each rung's job is to be trustworthy precisely in the regime where the rung above it failed, so the epistemic uncertainty that triggered the abstention is exactly what the lower rung is robust to. This is the graceful-degradation principle that the functional-safety cases of Chapter 68 demand, and it is why "the model was uncertain so we did nothing" is a design error, not a safe state.
The safety margin is the last piece. When a downstream controller acts on a conformal interval \([\hat{y} - q, \hat{y} + q]\), it should act on the interval, not the point estimate: plan for the worst plausible value, not the expected one. A collision-avoidance system uses the near edge of a distance interval; a battery controller uses the pessimistic edge of a state-of-charge interval. Inflating the margin by choosing a smaller miscoverage \(\alpha\) trades availability for safety, and that tradeoff is a decision to state explicitly, because an interval so wide the system abstains constantly is as useless as a point estimate that is confidently wrong. Under drift, pair the margin with the adaptive conformal updates of Section 18.5 so the width tracks the current error rate rather than a stale calibration.
Research Frontier
The strongest current approach learns the gate jointly with the model rather than thresholding a post-hoc score. Deep Gamblers (Liu et al., 2019) and SelectiveNet (Geifman and El-Yaniv, 2019) add an abstain head trained under a coverage constraint, and Geifman and El-Yaniv's earlier SGR framework gives a selective classifier with a guaranteed risk bound at a chosen confidence level. The active frontier is making abstention robust under distribution shift: recent work couples conformal risk control (Angelopoulos et al., 2024) with selective prediction so the selective-risk guarantee survives the exchangeability breaks that Chapter 66 studies, and open questions remain about how to certify a full fallback hierarchy end to end rather than one gate at a time. For sensor systems the honest state of the art is that per-gate guarantees are solid, system-level guarantees are still assembled by hand.
The Right Tool
Hand-rolling a calibrated selective classifier (temperature scaling, a validated score, the risk-coverage sweep, and a threshold with a finite-sample risk bound) runs to roughly 80 to 120 lines before the guarantee is trustworthy. MAPIE exposes selective prediction and conformal risk control through a scikit-learn-style API: fit on a calibration split, call predict with a target risk, and get back predictions plus an abstain mask that respects your error budget in about 8 lines. The library owns the calibration, the risk-bound arithmetic, and the coverage guarantee; you own the cost model and the fallback hierarchy, which are the parts only you can specify. Roughly a 10x cut in code, and the statistical guarantee stops being yours to prove.
Exercise
Take any trained sensor classifier from an earlier lab and build its risk-coverage curve on a held-out split with risk_coverage_curve from Listing 18.6, using three gating scores: (1) softmax max-probability, (2) the same after temperature scaling, and (3) an epistemic score from a small deep ensemble (variance of the members' predictions). Overlay the three curves. Then inject an out-of-distribution shift into the test set (a new sensor placement, a scaled amplitude) and rebuild the curves. Report which score keeps its selective risk lowest at eighty percent coverage after the shift, and explain in two sentences why the softmax gate degrades most, tying your answer to the epistemic term of Section 18.1.
Self-Check
1. Selective risk is defined only over the answered subset. Give a gating policy that achieves zero selective risk, and explain why it is useless, and what second metric you must report alongside risk to rule it out.
2. A wrong classification costs 100 times what invoking the fallback costs. Using the Chow rule, at what calibrated confidence should the model start answering, and what happens to that threshold if the model is overconfident (uncalibrated)?
3. Your model abstains on a novel input. Why is "hold the last command and continue" sometimes a safe fallback and sometimes the most dangerous possible choice? Give one sensor example of each.
What's Next
In Section 18.7, we take the abstain flags, intervals, and margins built here and ask how to communicate them to the systems that consume them: the message schema for shipping an uncertainty alongside a prediction, the contract a downstream controller or fusion node (Chapter 49) can rely on, and the failure modes that appear when one component's "I do not know" is silently read by the next as a confident zero.