Federated evaluation, security, and failure modes

"You asked me to report my accuracy. But I cannot see the test set, one in twenty of my teachers is lying to me on purpose, and a third of them go offline before the bell rings. My confidence interval, it turns out, is a leap of faith."

A Skeptical AI Agent

Federated evaluation: measuring a model you cannot pool

Centralized evaluation computes one number on one held-out set. Federated evaluation cannot, because the test data lives on the same devices as the training data and is equally forbidden to move. The how mirrors training: the server broadcasts the current global model, each client runs inference on its own local held-out split, and each returns only aggregate statistics (loss, per-class counts, a confusion matrix), which the server averages. The why it is hard is that a single averaged accuracy is now a summary of a distribution, and the distribution is where the truth lives. A model that scores 94 percent averaged can be 99 percent on the majority of clients and 55 percent on a skewed tenth, which for a fall detector or an arrhythmia flag is the tenth that matters.

So federated evaluation reports the distribution over clients, not just its mean. The three numbers that matter are the population average, weighted by client sample count \(n_k\), and the tails:

where \(a_k\) is client \(k\)'s local accuracy and the low quantile \(\text{Acc}_{0.1}\) (the worst-decile client) is the operational headline for any safety-relevant fleet. Two subtleties turn this from arithmetic into engineering. First, the split must be leakage-safe within each client: a subject's data cannot straddle its own train and test partitions, exactly the discipline of Chapter 5, now enforced a thousand times over with no central auditor to check it. Second, only clients that participate in an evaluation round contribute, and participation is not random: devices that are charging, on Wi-Fi, and idle are systematically different from those that are not, so the reported average is a biased estimate of true fleet performance. This participation bias is the federated analogue of survivorship bias, and it flatters your metrics.

Security: poisoning, backdoors, and Byzantine robustness

Open a training loop to a million untrusted devices and you have opened it to attackers who control some of them. The threat model splits by intent. Data poisoning corrupts a client's local training set (mislabeling, or injecting crafted sensor samples) so its honest-looking update nudges the global model wrong. Model poisoning is stronger: a compromised client ignores its data entirely and sends a hand-crafted weight vector, and because FedAvg simply averages, one large malicious update can drag the mean arbitrarily far, an attack the literature calls Byzantine because the faulty node may behave adversarially rather than merely crash. The most insidious variant is a backdoor: the attacker leaves the model's accuracy untouched on ordinary inputs but plants a trigger, so a specific accelerometer pattern or an RF watermark makes the model output an attacker-chosen label. Backdoors survive averaging surprisingly well and are nearly invisible to the distribution-level evaluation above, because the model looks healthy on every client's real data.

The defense replaces the mean with a robust aggregation rule that a minority of outliers cannot dominate. Coordinate-wise median and trimmed mean discard extreme values per parameter; Krum (Blanchard et al., NeurIPS 2017) selects the single update closest to its neighbors, on the logic that honest updates cluster and a poison vector sits far out; Bulyan and coordinate-wise median variants trade compute for stronger guarantees. The when: any cross-device deployment with open enrollment needs at least a trimmed mean; a cross-silo deployment among a few vetted hospitals or factories can rely more on authentication and less on statistics. The listing below shows why the mean is indefensible and how one line of robust aggregation restores it.

import numpy as np
rng = np.random.default_rng(0)

# 20 honest clients agree on an update near the true direction; 3 are poisoned.
true = np.array([1.0, -0.5])
honest = true + rng.normal(0, 0.1, size=(20, 2))
poison = np.tile([50.0, 50.0], (3, 1))          # 3 large model-poisoning updates
updates = np.vstack([honest, poison])

def trimmed_mean(U, beta=0.15):                  # drop top/bottom beta fraction per coord
    k = int(beta * len(U))
    return np.mean(np.sort(U, axis=0)[k:len(U) - k], axis=0)

print("plain mean :", np.mean(updates, axis=0).round(2))   # hijacked by 3 clients
print("coord median:", np.median(updates, axis=0).round(2))
print("trimmed mean:", trimmed_mean(updates).round(2))     # recovers the true update

As Listing 64.7.1 shows, robustness is cheap to add and expensive to omit. But it collides head-on with the privacy layer. Secure aggregation (Section 64.4) is designed so the server sees only the sum of masked client updates and never an individual vector, which is exactly what a median or a Krum score needs to inspect. You cannot trim outliers you are cryptographically forbidden to see. Reconciling the two, robust aggregation under secure aggregation, via zero-knowledge range proofs or verifiable secret sharing, is one of the live tensions of the field.

Failure modes: the fleet that quietly rots

Most federated systems do not die from attacks; they degrade from mundane systems failures that no exception ever surfaces. Stragglers and dropout are chronic: a client that starts a round but loses power or connectivity contributes nothing, and if slow devices (older hardware, weaker links) drop out systematically, the global model is trained on a faster, richer, non-representative slice of the fleet. Staleness afflicts asynchronous designs, where an update computed against an old global model arrives rounds later and pulls the current model backward. Non-participation bias is the training twin of the evaluation bias above: models learn the world of charging, idle, well-connected devices and underserve everyone else. And silent quality collapse is the worst, because the server, forbidden to see data, is half-blind: a firmware update that recalibrates a sensor, a seasonal distribution shift (Chapter 66), or a slow drift in labeling quality can erode accuracy for months with no error thrown. The defenses are operational, not algorithmic: monitor participation demographics, track the per-client metric distribution over time (not just its mean), version the model per client, and keep a small trusted validation cohort whose data can be inspected as a tripwire, the federated corner of MLOps for sensor fleets (Chapter 69).

import flwr as fl

# Robust aggregation + built-in federated evaluation, no plumbing.
strategy = fl.server.strategy.Krum(          # or FedTrimmedAvg, FedMedian
    num_clients_to_keep=1,
    fraction_evaluate=0.5,                    # sample half the fleet each eval round
    evaluate_metrics_aggregation_fn=weighted_and_quantiles,  # your worst-decile reducer
)
fl.simulation.start_simulation(client_fn=client_fn, num_clients=100000,
                               strategy=strategy)

What's Next

In Chapter 65, we lift the evaluation discipline glimpsed here (worst-group metrics, leakage-safe splits, honest reporting under bias) out of the federated special case and make it the general contract for any sensor model, centralized or not: task versus operational metrics, user and site and device and session splits, time-aware cross-validation, and the field-validation protocols that separate a benchmark number from a deployable claim.