User personalization and adaptation

"I learned that humans walk at 1.4 metres per second. Then I met one who ambles, one who marches, and one on crutches, and discovered that the average human does not exist."

A Chastened AI Agent

Why one model does not fit all users

Formally, personalization is a domain shift problem in which each user \(u\) is a domain. The population model learns \(p_\text{train}(y \mid x)\) from a pooled dataset, but at deployment the wearer draws windows from their own \(p_u(x)\), and both the marginal (how your walking accelerometer signature looks) and the class-conditional (how your particular stair-climb differs from your walk) can differ from the pool. The gap is not subtle. Across standard human-activity benchmarks, evaluating with a leave-one-subject-out protocol, where the test user never appears in training, routinely costs a large fraction of the accuracy you would see under a naive random split that scatters a single person's windows across both train and test. That inflated random-split number is a leakage artefact of exactly the kind flagged in Chapter 65: adjacent windows from one person are near-duplicates, so any per-window split silently trains and tests on the same individual. The honest baseline for personalization work is always the leave-one-subject-out score, and personalization is judged by how much of that lost ground it recovers.

A ladder of adaptation methods

The cheapest and most underrated technique is per-user feature normalization. A large part of the cross-user gap is just an affine shift: different resting orientations, strap tightness, and body mass move the mean and scale of every feature. Standardizing each user's features by their own running mean and variance, rather than by the global training statistics, removes that shift before the classifier ever sees the data, costs no labels, and often recovers a meaningful slice of the gap for a few lines of code. It is the first thing to try and the last thing people remember.

One rung up, supervised fine-tuning takes the pretrained population network and continues training its last layers (or a small adapter) on a short labelled calibration set from the user. It is the strongest adapter when labels exist, but it risks catastrophic forgetting of the general model if you fine-tune too aggressively on a tiny set, so freeze the backbone and adapt only a light head. Few-shot prototype methods avoid gradient updates entirely: embed the user's few labelled examples with the frozen population encoder, average them into one prototype per class, and classify new windows by nearest prototype. This is cheap, stable, and pairs naturally with the contrastive encoders of Chapter 17, whose embeddings are built precisely so that same-activity windows cluster. At the top of the ladder, unsupervised test-time adaptation adjusts the model from unlabelled target data alone, for instance by recomputing batch-normalization statistics on the new user's stream or minimizing prediction entropy; this is the machinery of Chapter 66 applied with the user as the shifting domain.

The listing below implements the two label-free rungs that every practitioner should reach for first: per-user standardization and a nearest-prototype adapter over a frozen embedding. It reports the population baseline and the personalized score so the recovered gap is explicit.

import numpy as np

def personalize(z_train, y_train, z_user_few, y_user_few, z_user_test):
    # z_* are frozen-encoder embeddings; *_few is the tiny calibration set.
    # 1) Per-user standardization: remove this user's mean/scale shift.
    mu, sd = z_user_few.mean(0), z_user_few.std(0) + 1e-6
    zf = (z_user_few - mu) / sd
    zt = (z_user_test - mu) / sd

    # 2) One prototype per class from the few labelled user windows.
    classes = np.unique(y_user_few)
    protos = np.stack([zf[y_user_few == c].mean(0) for c in classes])

    # 3) Classify test windows by nearest prototype (cosine).
    protos /= np.linalg.norm(protos, axis=1, keepdims=True) + 1e-9
    ztn = zt / (np.linalg.norm(zt, axis=1, keepdims=True) + 1e-9)
    pred = classes[np.argmax(ztn @ protos.T, axis=1)]
    return pred

As Listing 26.4 shows, the entire personalized classifier is a normalization and a set of class means; there is no optimization loop to diverge and nothing to overfit beyond a mean vector. That stability is exactly why prototype adaptation is the default for on-device personalization where you cannot supervise a training run.

Cold start, drift, and evaluating honestly

Personalization has a cold-start problem: on the very first day, before any user data exists, you must fall back to the population model, so ship a strong user-independent baseline and let personalization improve it, never depend on it. Adaptation should also be reversible and monitored. An unsupervised adapter that quietly minimizes entropy can lock onto a confident wrong answer and drift away from truth, so cap how far the personalized model may move from the population prior and keep the original weights so you can reset. Because a person's behaviour itself changes over weeks (injury, ageing, a new gym routine), personalization shades into the continual-learning problem taken up next; the distinction is that personalization adapts to who the user is, while continual learning tracks how they change.

Evaluate with the same discipline the rest of the book insists on. The headline number must come from a leave-one-subject-out split, and the calibration data used to personalize a test user must come from a disjoint time block of that user, never from windows interleaved with the test windows, or you leak the answer through temporal correlation and report a fantasy. Report the population baseline and the personalized score side by side so the recovered gap is visible, and record how many labelled windows the personalization consumed, because a method that needs fifty labels per class is a different product from one that needs two.

What's Next

In Section 26.5, we hold the user fixed and let time move. Personalization adapts to who a person is on day one; but the same person's gait after an injury, their routine after a move, and their activity mix as they age all drift the target out from under a fixed model. We take up continual learning for changing behavior: how to keep updating a deployed model over months without forgetting what it already knew.