"They never labeled a single heartbeat. They just told me the two minutes from the same wrist belong together, and let me infer the rest of cardiology."
A Self-Taught AI Agent
The Big Picture
Section 20.2 showed Google's route to a large sensor model: reconstruct masked patches across every channel at once. This section covers the other dominant recipe, exemplified by Apple's work on the Apple Heart and Movement Study. Instead of masked reconstruction over a fused multi-sensor tensor, Apple trains a separate self-supervised encoder per biosignal (one for photoplethysmography, one for electrocardiogram, and behavioral models over accelerometer-derived activity), and it exploits a resource that only wearables have in abundance: weeks of unlabeled data from the same identified person. That single fact reshapes the entire training objective. Understanding why lets you predict when this family beats a masked model, and why its central trick is also its central privacy hazard.
This section builds directly on the contrastive and self-supervised machinery of Chapter 17, the PPG signal physics of Chapter 30, and single-lead ECG from Chapter 29. It also assumes the leakage discipline of Chapter 5, because participant identity is the axis along which both the method and its failure modes turn.
The wearable advantage: identity as a free label
A consumer smartwatch produces a peculiar dataset. It is enormous (Apple's PPG corpus draws on roughly 20 million segments from more than 140,000 participants, and the ECG corpus from tens of thousands more), it is almost entirely unlabeled, and, crucially, it is partitioned by wearer. You do not know a participant's ejection fraction, but you know with certainty that two 60-second windows recorded from the same wrist on the same day come from the same cardiovascular system, the same vascular geometry, the same skin tone under the same green LEDs.
That structure is the whole game. In text or images, deciding which pairs of samples are "similar enough" to pull together is an art of augmentation. On a wearable, the data hands you a similarity relation for free: same participant means physiologically related. Apple's core objective, participant-level contrastive learning, promotes segments from one person to be neighbors in embedding space while pushing different people apart. The model is forced to discover the stable per-person factors (resting heart-rate morphology, pulse-wave shape, arrhythmia burden) because those are exactly what distinguishes one wrist from another across time.
Key Insight
Masked models (Section 20.2) learn within-window structure: predict the hidden part of one recording from the visible part. Participant-contrastive models learn across-window, within-person structure: what is invariant about this individual over hours and days. The two objectives capture different physiology. That is why the Apple recipe excels at person-level traits that persist (age, sex, BMI, chronic conditions) while a masked reconstruction model is often stronger at reconstructing a corrupted single beat. Pick the pretext task that matches the label you will eventually predict.
Participant-level contrastive learning, precisely
The training loop is a variant of InfoNCE, the loss you met in Chapter 17. For an anchor embedding \(z_i\) from participant \(p\), a positive \(z_i^{+}\) is another segment from the same participant \(p\) (optionally also augmented), and the negatives are segments from other participants in the batch or a momentum queue:
$$ \mathcal{L}_{\text{NCE}} = -\log \frac{\exp\!\big(\mathrm{sim}(z_i, z_i^{+})/\tau\big)}{\displaystyle\sum_{j}\exp\!\big(\mathrm{sim}(z_i, z_j)/\tau\big)}, \qquad \mathrm{sim}(a,b)=\frac{a^\top b}{\lVert a\rVert\,\lVert b\rVert} $$Here \(\tau\) is a temperature that controls how sharply the model separates people. Apple's refinement blends two positive-pair sources: augmentation-based positives (the standard SimCLR-style view of the same segment under jitter, cropping, and magnitude warping) and participant-based positives (a different segment from the same person). Weighting these two is a design knob: lean on augmentation and you learn signal-invariances; lean on participant positives and you learn person-invariances. The regularized combination is what gave their embeddings state-of-the-art transfer across dozens of downstream targets.
import torch, torch.nn.functional as F
def participant_infonce(z, participant_ids, tau=0.1):
# z: (N, d) L2-normalized embeddings; participant_ids: (N,) integer wearer id
z = F.normalize(z, dim=1)
sim = z @ z.t() / tau # (N, N) cosine similarities
N = z.size(0)
same = participant_ids[:, None] == participant_ids[None, :]
pos = same.clone(); pos.fill_diagonal_(False) # positives = same wearer, not self
sim.fill_diagonal_(float("-inf")) # never contrast a sample with itself
log_prob = sim - torch.logsumexp(sim, dim=1, keepdim=True)
# average log-prob over each anchor's positives (skip anchors with no same-wearer peer)
has_pos = pos.any(dim=1)
loss = -(log_prob * pos).sum(1)[has_pos] / pos.sum(1)[has_pos]
return loss.mean()
z = torch.randn(8, 32)
ids = torch.tensor([0, 0, 1, 1, 2, 2, 3, 3]) # 4 wearers, 2 segments each
print(float(participant_infonce(z, ids)))
participant_ids rather than by augmented views) is the entire conceptual leap of the Apple recipe.The snippet above is deliberately bare so the mechanism is visible. The production system adds a momentum encoder and a large negative queue (so a batch effectively contrasts against tens of thousands of other people), plus the augmentation-positive term.
Right Tool: don't rebuild the contrastive scaffold
The momentum encoder, negative queue, projection head, and distributed loss synchronization are a well-solved roughly 400-line stack. lightly (for MoCo, NNCLR, and friends) or pytorch-metric-learning's SupConLoss give you the queue, the exponential-moving-average target network, and multi-GPU gathering in about a dozen lines: instantiate the loss, pass embeddings and a group label, and let it treat your participant_id as the class. That replaces the batch-gather and queue bookkeeping that is easy to get subtly wrong (a desynced negative queue silently collapses the embedding).
One encoder per modality, and why
Apple trains modality-specific encoders rather than one fused tokenizer. The PPG encoder and the ECG encoder are separate 1D convolutional networks (EfficientNet-style backbones adapted to time series), each pretrained on its own signal at that signal's native rate. This is the opposite of the LSM philosophy in Section 20.2, and the reasons are physical, not arbitrary. PPG is an optical volume signal at tens of hertz with motion-dominated artifacts (see Chapter 30); single-lead ECG is an electrical potential at hundreds of hertz with entirely different noise (baseline wander, powerline). Their morphologies, sampling rates, and failure modes barely overlap, so a shared patch embedding wastes capacity forcing one projection to serve both. The accelerometer stream (from Chapter 23) enters more often through derived behavioral features (step cadence, activity bouts, sleep) than as raw triaxial waveforms, and Apple's later behavioral foundation model pretrains on years of these wearable-derived summaries rather than the raw accelerometer trace.
The tradeoff is explicit: separate encoders give clean, high-quality per-modality embeddings and let each network respect its signal's timescale, at the cost of losing the tight cross-modal fusion (PPG and ECG measuring the same heartbeat from two physics) that a joint model could exploit. When you need that fusion you combine the frozen embeddings downstream, which connects to the missing-modality robustness techniques in the next sections and in Chapter 50.
Practical Example: a startup screening for atrial fibrillation
A clinical wearables startup had 3,000 labeled AFib recordings, far too few to train a cardiac model from scratch. Instead they froze a PPG foundation encoder pretrained by participant-contrastive SSL, computed one 256-dimensional embedding per 60-second window, and trained a logistic-regression head on top. With only those 3,000 labels the head reached screening-grade sensitivity, because the encoder had already learned pulse-wave morphology from 140,000 unlabeled wearers. The instructive failure came next: the same frozen embeddings also predicted participant age and sex with uncomfortable accuracy, which the team had not intended and which forced a privacy review before deployment. The embedding that makes you data-efficient is the same embedding that encodes demographics you never asked for, a tension that runs straight into Chapter 34.
Evaluating and using the embeddings without fooling yourself
The standard evaluation is linear probing: freeze the encoder, fit a simple head per downstream target, and report transfer. The non-negotiable rule is that the split must be by participant. If windows from one wearer appear in both train and test, the participant-contrastive objective has explicitly optimized the model to recognize that person, so any leakage inflates every number catastrophically. This is the Chapter 5 discipline made sharp: for this model family, subject-level splitting is not best practice, it is the difference between a real result and a re-identification score in disguise. Because these embeddings feed clinical decisions, pair them with the calibration and conformal machinery of Chapter 18 so a downstream head reports honest uncertainty rather than an overconfident point estimate.
Research Frontier
As of 2026 the state of the art in this family is defined by Apple's Apple Heart and Movement Study biosignal encoders (PPG and ECG foundation models trained with regularized participant-plus-augmentation contrastive learning) and their subsequent wearable behavioral foundation model pretrained on billions of hours of activity data. The open questions are live: how to combine masked-reconstruction and participant-contrastive objectives so one encoder captures both within-beat and across-person structure; whether participant positives should be replaced by physiology-aware positives (same person only when in the same physiological state) to avoid entangling identity with health; and how to pretrain without turning the encoder into a covert biometric identifier. Expect the next generation to add explicit identity-disentanglement penalties so the useful physiology survives while the re-identification signal is suppressed.
Exercise
Using the participant_infonce function above: (a) construct a batch where every segment comes from a different wearer and explain what the loss does (and why training would collapse). (b) Sweep the temperature \(\tau\) over {0.05, 0.1, 0.5} on a fixed random batch and describe how the loss magnitude and the sharpness of the implied person-separation change. (c) Modify the positive mask so that a segment is a positive only when it shares both participant_id and a coarse activity label; argue in two sentences why that could reduce demographic leakage.
Self-Check
1. Why does a smartwatch corpus give you a "free" similarity label that an image corpus does not, and which loss term exploits it?
2. Give one physiological reason Apple uses separate PPG and ECG encoders instead of one fused tokenizer, and one capability that choice sacrifices.
3. A colleague reports 0.99 AUC for age prediction from PPG embeddings using a random 80/20 window split. What is almost certainly wrong, and how do you fix the evaluation?
What's Next
In Section 20.4, we turn from cardiovascular biosignals to motion: how accelerometer and IMU foundation models are pretrained for human activity recognition, why their augmentations and pretext tasks differ from the biosignal recipe here, and how the two families are beginning to merge into a single wearable encoder.