Incident response

"The dashboard is green, the pager is screaming, and forty thousand thermostats are convinced it is minus nine degrees indoors. One of us is wrong, and I have already checked my math."

An On-Call AI Agent

This section assumes you understand the monitoring signals that raise an incident (Sections 69.3 and 69.4), the versioning that lets you roll one back (Section 69.2), and the privacy-constrained logging (Section 69.6) that supplies the evidence you reconstruct a timeline from. It builds directly on the functional-safety framing of Chapter 68: incident response is where a documented safe state stops being a diagram and becomes a command you send to the fleet.

What counts as an incident, and how severe is it

Not every alert is an incident. A single device with a stuck accelerometer is a fault your fleet-management layer (Section 69.5) should quarantine automatically. An incident is a fleet-scale or safety-relevant deviation: many devices at once, or a small number producing decisions that can harm someone or damage property. The first job of an on-call responder is to place the event on a severity ladder, because severity sets the clock. A useful sensor-fleet ladder runs from SEV-1 (a perception failure that can cause physical harm, or a fleet-wide wrong decision) through SEV-3 (degraded accuracy, no safety impact, business hours acceptable) down to SEV-5 (cosmetic or single-device).

Severity should be a function you can compute, not a vibe. Two axes dominate: blast radius (the fraction of the fleet affected) and harm potential (what a wrong decision does in the physical world). A drift alert on 0.1% of devices running a step counter is low on both; the same drift on a clinical PPG model flagging atrial fibrillation is high on harm even at small radius, because a missed detection has clinical consequences (see the validation stakes in Chapter 34). Encoding this as an explicit rule removes the worst failure mode of on-call: a tired human under-classifying a SEV-1 at 3 a.m. because the graph looked familiar.

The response loop: detect, triage, contain, recover, learn

Borrow the incident-command structure that site-reliability practice made standard, and adapt it to sensors. Detect: an alert crosses from monitoring into paging with enough context (which model version, which device cohort, which contract field) to act. Triage: an incident commander is named, severity is set, and a single communication channel opens so the fleet does not get three conflicting rollbacks. Contain: stop the bleeding before you understand it, by halting the rollout, rolling back to the last known-good deployable unit, or tripping devices to the safe fallback. Recover: restore normal service on a verified fix. Learn: a blameless postmortem that produces action items with owners.

The sensor-specific twist is that containment actions have propagation delay and cost. A rollback is an over-the-air update to intermittently connected, battery-limited hardware; a device asleep in a field will not get the command for hours (the constraints of Chapter 63 apply). This is why the on-device safe-mode trip matters: a local circuit breaker that reacts to the device's own out-of-contract or high-uncertainty signal contains the incident in milliseconds, without waiting for the cloud to notice, decide, and reach back down.

import time
from dataclasses import dataclass, field

@dataclass
class FleetCircuitBreaker:
    """Trips a device cohort to a safe fallback when the incident
    signal (out-of-contract rate or model uncertainty) breaches
    a threshold, and refuses to auto-recover until it is stable."""
    error_threshold: float = 0.05      # 5% out-of-contract windows
    window: int = 200                  # inferences to average over
    cooldown_s: float = 900.0          # 15 min stable before recovery
    _recent: list = field(default_factory=list)
    tripped_at: float | None = None

    def observe(self, out_of_contract: bool, now: float) -> str:
        self._recent.append(1 if out_of_contract else 0)
        self._recent = self._recent[-self.window:]
        rate = sum(self._recent) / len(self._recent)

        if self.tripped_at is None and rate >= self.error_threshold:
            self.tripped_at = now                 # contain immediately
            return "TRIP_TO_SAFE_MODE"
        if self.tripped_at is not None:
            healthy = rate < self.error_threshold / 2   # hysteresis
            if healthy and now - self.tripped_at >= self.cooldown_s:
                self.tripped_at = None
                return "RECOVER_TO_PRIMARY"
            return "HOLD_SAFE_MODE"
        return "NORMAL"

cb = FleetCircuitBreaker()
t0 = time.time()
for i in range(300):
    bad = i > 210            # a firmware push starts breaking the contract
    print(i, cb.observe(bad, t0 + i))  # watch it TRIP, then HOLD

Runbooks, error budgets, and the blameless postmortem

Speed under stress comes from preparation, not heroics. A runbook is a per-failure-mode checklist: for "input contract violation after firmware push," it names the rollback command, the safe-mode trigger, the dashboards to open, and the stakeholders to notify. Runbooks turn a novel-feeling 3 a.m. crisis into a known procedure a junior responder can execute. Pair them with an error budget: if your fall-detection service targets 99.5% correct-decision availability, the 0.5% is a budget you spend, and burning it fast is itself the trigger to freeze all rollouts and shift the team from features to reliability. The budget makes the reliability-versus-velocity argument quantitative instead of political.

After recovery comes the part teams skip and regret: the blameless postmortem. It reconstructs a factual timeline (detection latency, containment latency, recovery latency), asks why the safeguards did not catch it earlier, and produces owned action items. Blameless means you interrogate the system, not the engineer who clicked deploy; people who fear punishment hide the context you need. For safety-relevant perception, root-causing is not optional, and the interpretability tooling of Chapter 67 is how you turn "the model got worse" into "the model relied on a spectral band the new anti-alias filter removed."

What's Next

In Chapter 70, we step back from operating a fleet to governing it: how surveillance, consent, biometric privacy, fairness across bodies and environments, and regulation (the EU AI Act, GDPR, and FDA pathways) reshape what you are even allowed to build and deploy, and how the incident-response and logging discipline you just learned becomes part of an accountability record a regulator can audit.