"I hit 0.99 AUROC on the validation set, so I shipped it to two million wrists. The regulator had questions."
An Overconfident AI Agent
The big picture
Everywhere else in this book, a model that beats the baseline is a good model. In health, that is only the entry ticket. The moment a biosignal algorithm influences a diagnosis, a treatment, or even a nudge to "see your doctor", it enters a regulated space where the burden of proof, the documentation, and the failure analysis are as important as the metric. This section is a map, not a manual: it gives you the vocabulary (Software as a Medical Device, intended use, risk class, clinical evaluation) and the reflexes you need so that the modeling choices you make in the rest of Part VII do not paint you into a corner you cannot ship out of. The deep treatment of validation design and biometric privacy lives in Chapter 34; here we set the frame.
You already know how to build patient-level splits (Section 28.6) and read reference standards (Section 28.5). Those are prerequisites for everything below, because a regulator's first question about a health algorithm is not "how accurate?" but "accurate on whom, measured against what, and under what claim?" If any of the leakage, labeling, or splitting terms in this paragraph are unfamiliar, revisit Chapter 5 before continuing.
What makes software a medical device
The pivotal concept is intended use. The International Medical Device Regulators Forum (IMDRF) and the US FDA define Software as a Medical Device (SaMD) as software intended for a medical purpose that performs that purpose without being part of a hardware medical device. The identical PPG model can be a consumer wellness feature or a regulated device depending entirely on the sentence you write about what it is for. "Estimates your resting heart rate for fitness tracking" is wellness. "Notifies you of possible atrial fibrillation" is a medical claim, and the second sentence is what triggers oversight, not the neural network behind it.
Why this matters for how you model. Regulators classify SaMD by risk, combining the significance of the information it provides (informs, drives, or treats/diagnoses) with the seriousness of the health situation (non-serious, serious, critical). An algorithm that diagnoses a critical condition sits at the top of the risk pyramid and demands the most evidence; one that informs a non-serious decision sits at the bottom. In the EU, the Medical Device Regulation (MDR 2017/745) formalizes this with rules that push most diagnostic software into Class IIa or higher. The lesson: your target label and your claim jointly set your regulatory burden, so choose them deliberately at design time, not after training.
Key insight
Regulatory class is a property of the claim, not the code. Two teams can ship byte-identical models and land in different regulatory worlds because one says "for informational purposes" and the other says "to aid diagnosis". Write the intended-use statement first; let it constrain the metrics, the study, and the labels you go collect.
The safety case: risk management before accuracy
Medical software is governed by process standards, not just outcome metrics. Two are load-bearing. ISO 14971 is the risk-management standard: you enumerate hazards, estimate the probability and severity of harm for each, and document mitigations until residual risk is acceptable. IEC 62304 is the software life-cycle standard: it dictates how you version, test, and trace requirements through to verification, with rigor scaled to a software safety class (A, B, or C) derived from the worst harm a defect could cause. For an AI model this means your training data lineage, your evaluation protocol, and your monitoring plan are all auditable artifacts, not lab notebook footnotes.
The uncomfortable insight for AI practitioners: a high AUROC does not by itself lower risk. What lowers risk is knowing how the system fails and bounding the harm of those failures. A false "possible AFib" alert causes anxiety and an unnecessary clinic visit; a missed critical arrhythmia in a device claiming to detect it can be lethal. Those are different hazards with different acceptable rates, and the risk file must treat them separately. This is exactly where calibrated uncertainty (Chapter 18) earns its place: a model that abstains when unsure converts a potential misdiagnosis into a benign "inconclusive, retry" state.
Practical example: the Apple Watch AFib notification
When Apple cleared its irregular-rhythm notification feature (FDA De Novo DEN180042, 2018), the interesting engineering was not the classifier; it was the safety framing. The feature is deliberately not a diagnostic. It notifies, it does not diagnose, and its labeling repeatedly tells users to consult a physician. It only alerts after multiple positive readings over time, trading sensitivity for a lower false-alarm burden, because at population scale even a tiny per-reading false-positive rate becomes millions of frightened users. The Apple Heart Study then followed notified users to a wearable ECG patch to measure real positive predictive value in the field. The takeaway: the intended-use sentence, the multi-reading gate, and the confirmatory pathway were co-designed to keep residual risk acceptable, and only then did the model's numbers matter.
Why population prevalence, not just the ROC, decides field safety
A validation table that reports sensitivity and specificity can look outstanding and still be unsafe to deploy, because the metric that actually reaches the user is positive predictive value (PPV): given an alert, how likely is it real? PPV depends on disease prevalence through Bayes' rule,
$$\mathrm{PPV} = \frac{\text{sens}\cdot p}{\text{sens}\cdot p + (1-\text{spec})\cdot(1-p)},$$where \(p\) is prevalence in the screened population. Screening a low-prevalence population (young, healthy wearable users) collapses PPV even for a near-perfect classifier, which is precisely why the regulatory framing above leans on confirmatory second steps. The snippet below makes the effect concrete and is worth running before you promise anyone a field-ready screener.
def ppv(sensitivity, specificity, prevalence):
tp = sensitivity * prevalence
fp = (1 - specificity) * (1 - prevalence)
return tp / (tp + fp)
# An excellent classifier, screening a young wearable population
for p in (0.10, 0.01, 0.002):
print(f"prevalence={p:6.3f} PPV={ppv(0.98, 0.99, p):.3f}")
# prevalence= 0.100 PPV=0.916
# prevalence= 0.010 PPV=0.497 <- half of all alerts are false
# prevalence= 0.002 PPV=0.164 <- most alerts are false alarms
The numbers above are why "we hit 0.99 AUROC" is never the end of the safety conversation. The same reasoning governs alarm design in clinical monitors, where alarm fatigue from low-PPV alerts is itself a documented patient-safety hazard.
Library shortcut
You do not hand-roll the confusion-matrix statistics that feed a validation report. scikit-learn's classification_report plus confusion_matrix turn a roughly 40-line block of manual TP/FP/FN bookkeeping and per-class ratio math into two calls, and it handles multi-class averaging, support counts, and edge cases (zero-support classes) that trip up hand-written code. For clinical-grade arrhythmia testing specifically, the WFDB toolkit implements the ANSI/AAMI EC57 beat-matching rules so you report the exact statistics regulators expect rather than a bespoke definition of a "correct" beat.
Wellness, decision support, and the moving line
Not every health feature is regulated, and the boundary is an active policy frontier. The US 21st Century Cures Act carved out certain Clinical Decision Support (CDS) software from device regulation, provided a clinician can independently review the basis for the recommendation rather than relying on it as a black box. That "independent review" clause is a direct challenge to opaque deep models: a system whose reasoning cannot be inspected is harder to fit into the non-device CDS exemption, which raises the stakes on interpretability (Chapter 67). Meanwhile, adaptive models that keep learning after deployment strain a framework built for frozen software; the FDA's predetermined-change-control-plan proposals are the regulator's attempt to catch up, and they are still evolving. Treat any specific rule you read as a snapshot, and confirm current guidance before you make a filing decision.
Exercise
Take a PPG sleep-staging model you might build in Chapter 30. Write two intended-use statements: one that keeps it a wellness feature and one that makes it a regulated medical device. For each, list (a) the likely SaMD risk category using the inform/drive/diagnose by non-serious/serious/critical grid, and (b) one hazard you would enter in an ISO 14971 risk file. Then compute, with the snippet above, the PPV you would advertise if the target condition has 3% prevalence and your model reaches 95% sensitivity and 92% specificity.
Self-check
1. Two teams deploy the same trained model; one needs FDA clearance and one does not. What single artifact explains the difference?
2. Why can a classifier with 99% specificity still generate mostly false alarms in a consumer wearable population, and what design pattern mitigates it?
3. What is the difference in purpose between ISO 14971 and IEC 62304, and why does an AI team need both?
Lab 28
detect heartbeats in ECG/PPG; compare signal quality under motion artifacts.
Bibliography
Regulatory frameworks and guidance
IMDRF SaMD Working Group (2013). Software as a Medical Device (SaMD): Key Definitions. IMDRF.
The source document that defines SaMD and the intended-use framing every regulator now shares; start here to understand why the claim, not the code, sets the class.
The EU rulebook whose classification rules push most diagnostic software to Class IIa or above; the reference for any product shipping in Europe.
The FDA's roadmap for regulating adaptive AI, including predetermined change control plans; essential reading for models that learn after deployment.
Safety and life-cycle standards
ISO (2019). ISO 14971:2019 Medical devices: Application of risk management to medical devices. ISO.
The hazard-analysis and residual-risk standard that turns "how does it fail?" into an auditable process; the backbone of any medical safety case.
IEC (2006, Amd. 2015). IEC 62304 Medical device software: Software life cycle processes. IEC.
Defines software safety classes A/B/C and the traceability every biosignal AI pipeline must demonstrate from requirement to verification.
The beat-matching protocol that fixes how arrhythmia sensitivity and PPV are computed; use it so your numbers mean what a reviewer expects.
Landmark wearable studies and models
The Apple Heart Study: 400,000+ participants showing how a wearable AFib notification is validated with a confirmatory ECG-patch pathway and real-world PPV.
A benchmark deep-ECG model evaluated against a committee of cardiologists; a template for what convincing clinical-grade validation of a biosignal model looks like.
What's Next
In Chapter 29, we put this frame to work on the most mature biosignal in clinical AI: the ECG. You will build arrhythmia and hidden-diagnosis models, evaluate them at beat and record level with the EC57 rules introduced here, and design the false-alarm triage that keeps a high-sensitivity detector from becoming an alarm-fatigue hazard. The regulatory reflexes from this section become concrete engineering constraints there, and the full validation-and-privacy treatment arrives in Chapter 34.