"A model with no datasheet is a witness with no memory: confident, plausible, and useless under cross-examination."
A Deposed AI Agent
The Big Picture
A wearable health model does not fail only in code; it fails in the gaps between what someone assumed about the training data and what was actually there. Documentation closes those gaps. A datasheet tells you where a physiological dataset came from and who is missing from it. A model card tells you what the model was measured to do and where it must not be used. An audit trail lets you reconstruct, months later and in front of a regulator, exactly which model version produced a given alert from which inputs. This section treats the three as one interlocking record: the connective tissue that makes the clinical validation of this chapter reproducible and defensible instead of merely asserted. Prerequisites are light: the leakage-safe dataset construction of Chapter 5 and the fleet-operations mindset of Chapter 69.
Datasheets: documenting the data before the model
A datasheet for a dataset, in the sense introduced by Gebru and colleagues, is a structured questionnaire filled in by whoever assembled the data. It answers five families of questions: motivation (why was it collected, funded by whom), composition (what each instance is, how many subjects, what is deliberately excluded), collection (the sensor, the sampling rate, the consent basis), preprocessing (filtering, resampling, artifact rejection), and distribution and maintenance (who may use it, when it expires). For sensor AI the composition and collection sections carry the weight, because a physiological dataset silently encodes the device firmware, the skin tones present, the clinical site, and the wear location.
Why it matters concretely: a photoplethysmography (PPG) dataset collected on one wrist-worn device at 25 Hz under resting conditions will not support a model claiming to detect atrial fibrillation during exercise, and the datasheet is where that boundary becomes visible before a clinician trusts an output. The datasheet is also where subgroup coverage is stated numerically, which is the raw material the equity analysis in Section 34.6 consumes. When you write one, record the generative process, not just summary statistics: which recordings were dropped and why, whether labels came from a 12-lead reference or a second wearable, and whether any subject appears in more than one split.
Key Insight
Documentation is not paperwork you attach after the model works; it is a measurement instrument for the invisible. The most dangerous properties of a health dataset (who is absent, which device dominates, where labels are noisy) leave no trace in accuracy on a random test split. A datasheet is often the only place those properties are ever written down, and an unwritten property cannot be reviewed, audited, or fixed.
Model cards: the label on the box
A model card, following Mitchell and colleagues, is a one-to-two page structured summary that travels with the trained model. Its non-negotiable fields are intended use and out-of-scope use, the evaluation data and metrics, and disaggregated performance across the subgroups the datasheet says are present. For a regulated wearable classifier the card should also name the operating point: the decision threshold, and the sensitivity and specificity at that threshold, not a single accuracy number that hides the tradeoff. A useful discipline is to report performance as a confusion-derived pair rather than a scalar, because a screening model tuned for high sensitivity, $$\text{sensitivity} = \frac{TP}{TP + FN}, \qquad \text{specificity} = \frac{TN}{TN + FP},$$ lives or dies on where you set the threshold, and the card must pin that down so a downstream integrator cannot quietly move it.
When and how to use the card: it is written once per released model version and re-checked at every retrain. Crucially, it must state calibration behavior, not only ranking metrics, so that a probability of 0.8 means what a clinician thinks it means. That connects the card directly to the calibration and conformal-prediction machinery of Chapter 18: a model card that reports an expected calibration error and the coverage of its prediction sets is far more actionable than one that reports AUROC alone.
Practical Example: the exercise-mode surprise
A sleep-staging startup shipped a wrist model whose card listed intended use as "overnight rest, adults 18 to 65" and out-of-scope use as "daytime and pediatric." A hospital integrator, unaware, wired the same model into a daytime delirium-monitoring pilot. Alerts were noisy and clinicians began ignoring them, eroding trust in the whole platform. The postmortem was fast precisely because the model card existed: the audit trail showed the model version, the card showed the out-of-scope flag, and the fix was a deployment guard, not a retraining project. The card did not prevent the misuse, but it converted a diffuse "the AI is unreliable" complaint into a specific, closeable configuration bug.
Audit trails: reconstructing what actually happened
An audit trail is the append-only record that lets you answer, for any past output, "which model, which weights, which inputs, which code, which time." In a regulated context this is provenance: each inference event is bound to a model version hash, the datasheet and card identifiers, the input window, and the software commit. The design constraint is immutability with integrity: you want to prove a record was not altered after the fact. A standard lightweight construction is a hash chain, where each record commits to the previous one: $$h_i = H(h_{i-1} \,\Vert\, r_i),$$ so that tampering with any earlier record \(r_i\) breaks every hash downstream and is detectable. This is the same tamper-evidence idea underlying the fleet governance in Chapter 70. A practical trail records enough to reproduce a decision but never stores raw biometric signal in the clear, because that same log would otherwise become a re-identification hazard of exactly the kind Section 34.4 warns about.
import hashlib, json, time
def log_inference(chain, *, model_hash, card_id, input_id, output, threshold):
prev = chain[-1]["record_hash"] if chain else "0" * 64
record = {
"ts": round(time.time(), 3),
"model_hash": model_hash, # weights + architecture fingerprint
"card_id": card_id, # links to the released model card
"input_id": input_id, # pointer, NOT raw physiological signal
"output": output,
"threshold": threshold, # the operating point actually used
}
payload = prev + json.dumps(record, sort_keys=True)
record["record_hash"] = hashlib.sha256(payload.encode()).hexdigest()
chain.append(record)
return chain
def verify(chain):
prev = "0" * 64
for r in chain:
body = {k: r[k] for k in r if k != "record_hash"}
payload = prev + json.dumps(body, sort_keys=True)
if hashlib.sha256(payload.encode()).hexdigest() != r["record_hash"]:
return False
prev = r["record_hash"]
return True
verify detects any retroactive edit. Note that input_id is a pointer to a governed store, never the raw waveform, keeping the audit trail from becoming a privacy leak.The code above is deliberately bare so the mechanism is visible: the load-bearing detail is that threshold and model_hash are logged per event, which is what lets you later prove the operating point matched the model card. In production you would not hand-roll this.
Library Shortcut
An experiment and model tracker such as MLflow gives you versioned model registries, per-run parameter and metric logging, artifact hashing, and a queryable lineage graph out of the box. Replacing the hand-rolled chain plus a bespoke registry with mlflow.log_model, mlflow.log_params, and the Model Registry stage transitions collapses roughly 300 to 400 lines of custom provenance and storage plumbing into about 15, and the library handles concurrent writers, artifact deduplication by content hash, and audit-friendly stage history (Staging to Production to Archived) that you would otherwise implement and test yourself.
Making the three records one system
Datasheet, card, and trail are only useful when they cross-reference each other by stable identifier. The datasheet ID appears in the card's "evaluation data" field; the card ID appears in every audit record; the model hash in the trail resolves to exactly one card. This closes the loop required by the predetermined change control plans of Section 34.2: when a model is retrained under an approved plan, a new card is minted, the datasheet is versioned if the data changed, and the trail's model_hash flips at a known timestamp, so the before-and-after populations are separable for post-market surveillance. Treat the trio as a single versioned bundle, released together, and reviewed together. The reward is that "explain this alert from six months ago" becomes a lookup rather than an archaeology project.
Exercise
Take a public wearable dataset (for example a PPG or accelerometer corpus) and draft a one-page datasheet covering composition and collection. Then write a matching model card for a simple resting-heart-rate estimator trained on it, including one honest out-of-scope statement and a disaggregated metric across at least one subgroup the datasheet reveals. Finally, extend the log_inference function to also record the datasheet ID, and show that verify still detects a tampered record.
Self-Check
- Why can a model card's out-of-scope section prevent harm even when the model itself is unchanged and accurate?
- What property does the hash-chain construction \(h_i = H(h_{i-1} \Vert r_i)\) give the audit trail, and why does storing raw waveforms in that trail undermine a different goal from Section 34.4?
- Which field must appear in both the model card and every audit record so that an old alert can be tied to the exact operating point that produced it?
Lab 34
write a validation-and-privacy plan for a wearable health model, including a re-identification risk check.
Bibliography
Foundational documentation frameworks
The originating proposal for dataset datasheets; its composition and collection questions map directly onto the hidden device, subgroup, and label properties of physiological corpora.
Defines the model card structure, including intended use, out-of-scope use, and disaggregated evaluation, the backbone of the card discipline in this section.
A complementary "data statement" framework emphasizing who is represented; useful when adapting datasheets to capture population coverage precisely.
Reporting standards for clinical AI
Consensus reporting items for AI trial protocols; its documentation requirements align the datasheet and card with regulatory expectations.
The companion reporting standard for completed AI trials, specifying what a model's evaluation record must disclose to be trustworthy.
Updated reporting guidance for AI prediction models, including calibration and fairness reporting that a rigorous model card should satisfy.
Provenance, governance, and lineage tooling
Google (2019). About ML and Model Card Toolkit documentation.
Open tooling that generates model cards from evaluation artifacts, showing how the card can be produced automatically as part of a pipeline.
The experiment tracker and model registry referenced in the library shortcut; supplies versioned lineage, artifact hashing, and auditable stage history.
What's Next
In Chapter 35, we leave the clinic for the factory floor. The datasheet, card, and audit-trail discipline you built here carries over almost unchanged: industrial telemetry has its own hidden device firmware, its own subgroup coverage (which machines, which duty cycles), and its own hard requirement to reconstruct why an alarm fired. We begin with how machines, processes, and telemetry become the sensor streams that predictive maintenance and condition monitoring will learn from.