Part VII: Health, Biosignals, and Wearable AI
Chapter 34: Clinical Validation, Regulation, and Biometric Privacy

Predetermined change control plans (PCCP) for learning models

"I retrained myself overnight on fresh data and I feel sharper than ever. My regulator would like a word."

An Enthusiastic Continual Learner

Why a learning model breaks the old approval bargain

Traditional medical-device regulation rests on a simple bargain: you prove a fixed device is safe, and it stays exactly that fixed device. Machine learning voids the bargain. A wearable arrhythmia detector that never changes will slowly rot as sensor hardware, patient demographics, and clinical practice drift away from its training distribution, yet every improvement you would like to ship, retraining on new data, adding a skin-tone-balanced cohort, raising the sampling rate, is technically a new device that would trigger a fresh marketing submission and months of review. The Predetermined Change Control Plan (PCCP) is the regulatory instrument built to escape this trap. You describe, in advance and as part of the original clearance, the envelope of future changes and the protocol you will follow to make them, and the regulator authorizes that envelope once. Inside it you iterate like a normal ML team; outside it you are back to a new submission. This section is about how that envelope is written, what makes a change "inside" versus "outside," and the engineering guardrails that let you actually trust an automatic update to a device attached to a sick person.

This section assumes you have read Section 34.1 on Software as a Medical Device (SaMD) and the FDA framework: the device classes, the 510(k) and De Novo pathways, and why any modification that could "significantly affect safety or effectiveness" ordinarily forces a new submission. The PCCP is the exception to that rule, so read the two together. We also lean on distribution-shift monitoring from Chapter 66, leakage-safe evaluation from Chapter 65, and the fleet-update machinery of Chapter 69, because a PCCP is, in engineering terms, a pre-registered contract wrapped around exactly those pipelines.

Locked, adaptive, and the space a PCCP opens between them

What the problem is. Regulators historically distinguished a locked algorithm, whose weights never change after clearance and which produces the same output for the same input forever, from an adaptive or continuously learning one that updates from field data. Locked models are easy to approve and slowly become wrong. Fully adaptive models are honest about the world but impossible to approve under the old bargain, because the thing you validated on Monday is not the thing running on Friday. Almost every marketed AI device today is deliberately locked and updated only through manual, controlled releases; the PCCP is what lets those controlled releases happen quickly rather than through a full re-review each time.

Why the PCCP exists and where it came from. The concept was floated in the FDA's 2019 discussion paper on AI/ML-based SaMD (which introduced the paired notions of pre-specifications and an algorithm change protocol), matured through the 2021 AI/ML Action Plan and the internationally harmonized Good Machine Learning Practice (GMLP) principles, and became concrete in the December 2024 FDA final guidance, Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions. The 2023 21st Century Cures-era statute (section 515C of the FD&C Act) gave it a legal basis. The through-line is consistent: pre-specify the change, pre-specify the method, assess the impact, and you may implement without a new submission.

A PCCP moves the review earlier, it does not remove it

The seductive misreading is that a PCCP means "the FDA lets my model update itself unsupervised." It means the opposite: the scrutiny happens up front, on the protocol, rather than repeatedly, on each result. You are trading one heavyweight review of every future model for one heavyweight review of the process that will generate them. That is only a good trade if your protocol is specific enough to be genuinely binding. A vague plan ("we will retrain periodically to improve accuracy") authorizes nothing, because it constrains nothing; a plan that fixes the data sources, the retraining recipe, the frozen test methodology, and the numeric acceptance thresholds is what actually earns the freedom to skip re-review.

The three parts of a PCCP

The final guidance structures a PCCP as three tightly linked components, and a submission that is missing any one of them is not a PCCP.

When a change is out of scope. Anything that alters the device's intended use beyond what the Description covers, changes the input modality to something not equivalent, or degrades performance below the pre-registered floor is outside the plan and requires a new marketing submission. The guardrail is not a suggestion: shipping an out-of-envelope change under a PCCP is a compliance violation, not a paperwork shortcut. The regulatory-context threads of Chapter 70 pick up how equivalent regimes (the EU AI Act, the EU MDR) treat the same substantial-modification question.

The acceptance gate: a protocol you can execute

Why this is the load-bearing part. The Modification Protocol is only as good as the moment a candidate model is tested against the criteria you promised. That moment should be a mechanical, auditable gate, not a judgment call, so that "the update was within our PCCP" is a fact you can prove after the event. The gate evaluates a retrained candidate on a frozen, versioned test set (frozen so that a drifting benchmark cannot silently move the goalposts, a leakage-safe discipline from Chapter 65) and checks every pre-registered criterion, including subgroup criteria so that a headline gain cannot hide a regression in an underrepresented group. The function below is a minimal but faithful sketch of such a gate.

from dataclasses import dataclass

@dataclass
class PCCPCriteria:
    min_auroc: float          # global performance floor (e.g. 0.94)
    max_subgroup_gap: float   # worst allowed AUROC gap across subgroups
    min_specificity: float    # false-alarm control for an alerting device
    frozen_test_id: str       # exact version hash of the locked test set

def passes_pccp_gate(candidate_metrics: dict, incumbent_auroc: float,
                     crit: PCCPCriteria) -> tuple[bool, list[str]]:
    reasons = []
    if candidate_metrics["frozen_test_id"] != crit.frozen_test_id:
        reasons.append("evaluated on the wrong test set version")   # non-negotiable
    if candidate_metrics["auroc"] < crit.min_auroc:
        reasons.append(f"AUROC {candidate_metrics['auroc']:.3f} below floor {crit.min_auroc}")
    if candidate_metrics["auroc"] < incumbent_auroc:                 # no silent regression
        reasons.append("candidate is worse than the deployed model")
    if candidate_metrics["subgroup_gap"] > crit.max_subgroup_gap:
        reasons.append(f"subgroup gap {candidate_metrics['subgroup_gap']:.3f} too large")
    if candidate_metrics["specificity"] < crit.min_specificity:
        reasons.append("specificity below the alert-fatigue floor")
    return (len(reasons) == 0, reasons)

crit = PCCPCriteria(min_auroc=0.94, max_subgroup_gap=0.03,
                    min_specificity=0.90, frozen_test_id="testset@v3.2.1")
cand = dict(auroc=0.951, subgroup_gap=0.021, specificity=0.92,
            frozen_test_id="testset@v3.2.1")
print(passes_pccp_gate(cand, incumbent_auroc=0.948, crit=crit))
A pre-registered PCCP acceptance gate. A candidate model is admitted only if it was scored on the exact frozen test set, clears the global performance floor, does not regress against the deployed incumbent, keeps the worst-subgroup gap under the promised bound, and holds the specificity floor that controls alert fatigue. Every rejection returns an auditable reason string.

The point of coding the gate this way is that it is refusing to pass by default and returns the exact reasons when it does, which is what turns a PCCP from a promise into a record. Note the two non-negotiable checks: an update evaluated on the wrong test-set version is void regardless of its score, and a candidate that is worse than the model already in the field never ships even if it clears the absolute floor. Subgroup criteria connect directly to the fairness treatment in Section 34.6.

Model registry plus validation, off the shelf

The gate above is the logic; the surrounding version control, artifact lineage, stage transitions, and audit log are the tedious part, and you should not hand-roll them for a regulated device. An MLflow Model Registry with a validation step wires the same acceptance criteria into promotion-blocking transitions in a handful of lines:

import mlflow
from mlflow.models import MetricThreshold

mlflow.evaluate(
    model="runs://model",
    data=frozen_test_v3_2_1,               # the versioned, locked test set
    targets="label", model_type="classifier",
    validation_thresholds={
        "roc_auc": MetricThreshold(threshold=0.94, greater_is_better=True),
    },
    baseline_model="models:/arrhythmia/Production",  # block silent regressions
)   # raises if the candidate fails a threshold; logs metrics for the audit trail
Replacing a hand-written gate plus manual bookkeeping with mlflow.evaluate against a baseline model and registry thresholds. This collapses roughly 40 to 60 lines of comparison, logging, and stage-transition glue into one validated call, and the registry keeps the immutable lineage a PCCP audit demands.

You still author the criteria (the library cannot know your clinical floors), but the versioned lineage, the baseline comparison, and the promotion block that a regulator wants to see come for free.

A wearable atrial-fibrillation detector that ages gracefully

Consider a wrist-worn photoplethysmography (PPG) device, of the kind built in Chapter 30, cleared to flag possible atrial fibrillation. Its makers know three things will happen: the optical sensor module will be revised across hardware generations, the user base will broaden to darker skin tones and older wrists that were thin in the pivotal study, and better labeled data will keep arriving. Without a PCCP, each of these would be a separate 510(k) with its own multi-month queue, so in practice the model would freeze and quietly lose accuracy on exactly the populations it under-served at launch. With a PCCP, the Description enumerates "retraining on additional PPG recordings of the same type" and "recalibration for a validated new sensor module of equivalent optical characteristics"; the Modification Protocol fixes the frozen holdout, the leakage controls (no subject appears in both train and test, per Chapter 5), and the subgroup floors by skin tone and age; the Impact Assessment argues why none of these can push false alerts past the pre-registered specificity. Now a quarterly retraining that clears the gate ships as a normal software update, while a proposal to also detect a different arrhythmia is correctly recognized as outside the envelope and routed to a new submission. The device improves on schedule and the regulator still knows exactly what it approved.

Guardrails: monitoring, rollback, and transparency

A PCCP that only checks a candidate at promotion time is half a plan. The other half is what happens after deployment. Real-world performance monitoring watches the deployed model for the distribution shift and silent degradation that Chapter 66 formalizes, because an update can clear every offline criterion and still misbehave on a subpopulation the frozen test set underrepresents. Every version must be individually rollback-able, so that a bad release is a one-command reversion to a known-good, previously cleared model rather than an emergency re-clearance. And transparency is now an explicit expectation: users and clinicians should be able to learn that the model changed, what changed, and how it was validated, which is why the datasheet, model-card, and audit-trail discipline of Section 34.7 is the natural companion to a PCCP. Taken together, the pre-registered gate, the field monitor, the rollback path, and the change log are the four legs that let an updating model stand up in a regulated setting.

Exercise: draft a two-change PCCP

Pick a wearable or clinical model from earlier in this part (an ECG classifier from Chapter 29 is a good choice). (1) Write a Description of Modifications with exactly two entries, one automatic and one manual, and for each state whether it is global or subpopulation-local. (2) For the Modification Protocol, specify the frozen test set, three numeric acceptance criteria (including one subgroup criterion), and the rollback trigger. (3) In the Impact Assessment, name one plausible way each change could harm a patient and the specific protocol step that catches it. (4) Finally, give one realistic change that your plan should not cover, and justify why it must go to a new submission instead.

Self-check

  1. In one sentence, explain why "we will periodically retrain to improve accuracy" is not an acceptable Modification Protocol.
  2. Name the three required components of a PCCP and say which one contains the numeric acceptance thresholds.
  3. A retrained candidate raises overall AUROC but was scored on a newer, larger test set than the one named in the plan. Should the acceptance gate pass it? Justify your answer in terms of what a PCCP is trading away.

What's Next

In Section 34.3, we move from the change-control envelope to the evidence that fills it: how to design clinical trials and prospective validation studies for sensor-driven models, including reference standards, enrollment and endpoints, and why a strong retrospective AUROC on your frozen test set is necessary but not sufficient to claim a device works on real patients.