"They wanted me to upload the raw signal so a bigger model could learn from it. I told them the signal is not a fact about the world; it is a fact about a person. Facts about people have owners, and the owner was not in the room."
A Discreet AI Agent
Prerequisites
This section assumes you can read a sensor output as a stream of timed samples (Chapter 3) and that you understand why inference already runs at the edge for latency, bandwidth, and energy reasons (Chapter 59). It builds on the leakage-safe dataset discipline of Chapter 5 and the biometric-privacy and regulation material of Chapter 34. No new mathematics is introduced; the one code example uses only nearest-neighbour distances. This section is motivation; the machinery that acts on it (federated averaging, differential privacy, secure aggregation) is built across the rest of this chapter.
The Big Picture
Chapter 59 argued that inference belongs on the device. This chapter confronts a harder asymmetry: even when the model runs locally, we still want it to keep learning from the whole fleet, and learning has traditionally meant pooling everyone's raw data in one place. This section explains why that pooling is often forbidden, unsafe, or simply impossible. Sensor data is not a neutral commodity: it is frequently personal, biometric, and re-identifying, it is governed by law that treats it as a liability rather than an asset, and once centralized it becomes a honeypot that a single breach can spill. Understanding these five forces (identity leakage, failed anonymization, legal exposure, security surface, and lost trust) is the whole reason federated learning exists. When you finish, you will be able to look at a sensing product and state precisely why the raw stream must stay put, which reframes the entire architecture: bring the model to the data, not the data to the model.
The signal is the person, not just the label
The tempting intuition is that a sensor stream carries only the task you care about. You want fall detection, so surely the accelerometer trace is just "fall or not fall." This is wrong, and the gap is the heart of the privacy problem. A raw sensor stream is vastly higher in information content than the label you extract from it, and much of that surplus is about the identity, health, and behaviour of the person carrying the device.
The evidence is now overwhelming across modalities. Gait, captured by a wrist or waist accelerometer, is an established biometric: the way you walk identifies you almost as reliably as a fingerprint. A photoplethysmography (PPG) trace (Chapter 30) carries the shape of your vasculature and can reveal arrhythmia, stress, and pregnancy. An inertial measurement unit sampled fast enough near a keyboard has been shown to leak typed text from the micro-vibrations of the surface. Ambient microphones capture speech and identity even when the task is only "detect a smoke alarm." The label is a projection; the raw signal is the full space, and the full space is a description of a human being.
Key Insight
Formally, let \(X\) be the raw window and \(Y\) the task label. On-device inference computes and releases \(Y = f(X)\), where the mutual information \(I(X; Y)\) is tiny compared with the entropy \(H(X)\). Shipping \(X\) to a server exposes everything in \(H(X)\), including a large component \(I(X; \text{identity})\) that the task never needed. Privacy failures in sensor AI are almost always failures to keep this residual out of transit. Federated learning is the discipline of moving only functions of \(X\) (model updates), never \(X\) itself, off the device.
Anonymization does not survive contact with sensor streams
The classic escape hatch is to strip names and identifiers and call the data anonymous. For structured records this is already fragile; for high-rate sensor streams it collapses. A short raw window is itself a quasi-identifier: it is nearly unique to the individual who produced it, so removing a name field changes nothing when the waveform is the fingerprint. The small experiment below makes the point without any real data. It synthesizes a stable per-person motion signature, adds noise to simulate separate recording sessions, and then tries to re-identify people from a single fresh window using nearest-neighbour matching against a gallery.
import numpy as np
rng = np.random.default_rng(0)
N_USERS, DIM = 200, 64 # 200 people, 64-D window "embedding"
signatures = rng.normal(size=(N_USERS, DIM)) # each person's stable pattern
def record(sig, sessions): # one noisy window per session
return sig + 0.6 * rng.normal(size=(sessions, DIM))
gallery = np.stack([record(s, 1)[0] for s in signatures]) # 1 enrolled window each
probes = np.stack([record(s, 1)[0] for s in signatures]) # a later, unseen window
# 1-nearest-neighbour identification: no names used, only the raw pattern
d = ((probes[:, None, :] - gallery[None, :, :]) ** 2).sum(-1)
guess = d.argmin(axis=1)
acc = (guess == np.arange(N_USERS)).mean()
print(f"re-identification accuracy from one window: {acc:.1%}")
# re-identification accuracy from one window: 100.0%The code is a caricature, but the finding is robust in the literature on gait, keystroke, and heart-rate biometrics: once the discriminative structure that makes a model useful is present, that same structure re-identifies the source. You cannot have a signal rich enough to train an activity or health model and simultaneously so featureless that it protects identity. This is the death of the "just de-identify and upload" plan, and it forces a different question: can we learn from the data without ever collecting it?
The law and the liability treat raw data as a hazard
Even where a stream did not obviously identify anyone, the legal environment increasingly forbids default collection. Regulations such as the GDPR encode purpose limitation (data gathered for fall detection may not be repurposed to train an unrelated model) and data minimization (collect only what the stated purpose requires). Health signals fall under stricter regimes such as HIPAA in the United States, and biometric identifiers are separately protected under laws like Illinois BIPA, with statutory damages per violation. Several jurisdictions add data residency constraints that make it unlawful to move raw recordings across a border at all. These rules are developed in depth in Chapter 70; here the point is architectural: the compliant default is not to collect.
Beyond the statute book sits a plainer engineering truth. A central store of raw sensor data is a liability, not an asset. It is a single target whose breach exposes every contributor at once, it must be secured, audited, and honoured against deletion requests forever, and its mere existence invites subpoena and mission creep. Data you never gathered cannot be stolen, subpoenaed, or misused. The cheapest way to secure a firehose of biometric signal is to ensure it never leaves the device that produced it.
Real-World Application: a mobile keyboard that never reads your typing
Consider a smartphone keyboard that predicts the next word and adapts to how each person actually writes. Streaming everyone's keystrokes to a server would be a catastrophe: keystrokes contain passwords, medical questions, and private messages, and a single breach would spill all of it. Yet a keyboard that never learns from real usage stays generic and clumsy. The resolution shipped in production by major vendors is federated: each phone improves the shared model locally on the words its owner typed, then sends only a small, aggregated model update, never a single character. The raw text stays on the glass it was typed on. The product still improves nightly across hundreds of millions of devices, and the sentence you just typed is never a row in anyone's database. This is precisely the pattern this chapter formalizes.
The reframing: bring the model to the data
Put the four forces together and a design principle falls out. If the raw stream is identifying, if anonymization fails, if the law forbids collection, and if a central store is a hazard, then the only place the data can safely live is where it was born. So invert the usual data pipeline. Instead of shipping data to a central model, ship the model to each device, train it locally on data that never moves, and send back only what the model learned, expressed as a parameter update \(\Delta \theta\). The server averages those updates into a better shared model and the cycle repeats. That is federated learning, and it is the subject of Section 64.2.
One caution keeps this honest and motivates the rest of the chapter: the update \(\Delta \theta\) is not automatically private. It is a function of the local data, and a function can leak its inputs.
Research Frontier: gradients remember their data
The assumption that "only the update leaves, so the data is safe" was punctured by gradient-inversion attacks. Zhu, Liu, and Han's Deep Leakage from Gradients (2019) showed that an honest-but-curious server can reconstruct the original training samples, pixel by pixel or sample by sample, from the shared gradient alone, and follow-up work (Geiping et al., Inverting Gradients, 2020) scaled this to realistic batch sizes. Membership-inference attacks go further and ask only whether a specific person's record was in the training set, which can itself disclose, for example, that someone is a patient in a disease cohort. These results are why federated learning alone is a privacy architecture, not a privacy guarantee, and why Section 64.4 pairs it with differential privacy and secure aggregation so that the aggregated update provably reveals almost nothing about any single contributor.
Library Shortcut
Orchestrating "data never leaves the device" by hand means writing a client that trains locally, serializes only the weight delta, a server that collects and averages deltas, retry logic for dropped clients, and a versioned round protocol: comfortably a few hundred lines before a single model trains. The Flower framework (flwr) reduces the client to subclassing NumPyClient with a fit() method and the server to one start_server() call with a strategy, roughly 20 to 30 lines total. Flower handles serialization, the round loop, client sampling, and straggler timeouts, so your privacy-preserving loop is mostly the ordinary local training you already know from Chapter 26.
Exercise
Take the re-identification snippet above and turn it into a leakage audit. (a) Sweep the session-noise multiplier from \(0.6\) up to \(3.0\) and plot re-identification accuracy; find the noise level at which the gallery match falls to chance (\(1/N\)). (b) At that noise level, train a simple classifier to separate two synthetic "activity" classes you add to the signatures and report its accuracy. (c) Argue from your two curves whether there exists a noise setting that destroys identity while preserving the task, and connect your answer to the utility cost of differential privacy you will measure in Lab 64.
Self-Check
1. A teammate proposes uploading raw IMU windows after deleting the user ID column, calling the result anonymous. In one sentence, why is this claim false for sensor streams?
2. On-device inference already keeps raw data local, so why does a fleet that only ever runs inference still have a reason to reach for federated learning?
3. Federated learning sends only model updates, never raw data. Name the specific attack that shows this is not yet a privacy guarantee, and the two techniques introduced later in this chapter that close the gap.
What's Next
In Section 64.2, we turn the principle into an algorithm. You will build the FedAvg family that trains a shared model from updates alone, and confront the property that makes sensor federation genuinely hard: every device sees a different, non-IID slice of the world, so naively averaging updates from a runner's wearable and an office worker's can pull the model in contradictory directions.