Fairness across users, bodies, and environments

"I was fair on the validation set. The validation set was ninety percent one skin tone, one age, and one well-lit room. So really I was fair to a photograph of a crowd, not to the crowd."

A Recalled AI Agent

Three axes of disparity: users, bodies, environments

It helps to separate fairness in sensory AI into three concrete axes, because each has a different root cause and a different fix.

Bodies are the physiological substrate the sensor transduces. Melanin absorbs the green and near-infrared wavelengths that PPG and pulse oximetry rely on, so the pulsatile component is a smaller fraction of the reflected light for darker skin, lowering SNR and biasing derived oxygen-saturation estimates. Wrist circumference, subcutaneous fat, tattoos, and hair all change the optical and mechanical coupling of a wearable. Body mass and limb length reshape the accelerometer and gyroscope signatures that human-activity models classify (Chapter 26). Age changes heart-rate variability, gait, and skin elasticity. These are not label problems; they are transduction problems.

Users are the behavioral and demographic layer on top of the body: how a person walks with a cane, signs a gesture with a smaller hand, performs a culturally specific activity, or wears the device loosely versus snugly. A fall detector tuned on able-bodied volunteers can misread the deliberate, controlled descents of someone using a wheelchair transfer. A gesture interface calibrated on adult hands can miss children entirely.

Environments are the physical context that modulates every measurement. Camera and depth systems degrade in low light and glare; thermal and infrared readings shift with ambient temperature (Chapter 45); RF and Wi-Fi sensing depend on room geometry and multipath; GNSS positioning is worse in dense urban canyons than in open suburbs. When a product is validated in one climate, one housing stock, or one lighting regime, "works in the lab" quietly encodes "works where affluent testers live."

Measuring fairness: subgroups, worst-group gaps, and calibration

The first discipline is to stratify every metric by the protected and physiological attributes that plausibly interact with the measurement channel: skin tone (for example on the Fitzpatrick or Monk scales), sex, age band, body-mass band, and deployment environment. For each group \(g\) you compute the same operational metric \(m_g\) you would report overall (recall at the deployed threshold, mean absolute error, missed-event rate), and then summarize the disparity rather than the mean. Two summaries carry most of the weight. The worst-group value \(\min_g m_g\) is what a regulator and an affected user care about. The gap between the best and worst group,

quantifies how uneven the system is. For a probabilistic output you additionally check calibration within each group: a heart-rate or hypoxemia estimate can be well calibrated on average yet systematically over-read for one skin tone, which is exactly the failure mode that made pulse-oximeter bias a clinical scandal. The calibration and conformal tooling from Chapter 18 should be run per group, not once globally, because a globally calibrated model can hide two miscalibrated halves that happen to cancel.

Two cautions make the difference between honest and theatrical fairness reporting. First, subgroup sample size: a "fair" result on a group of eleven people is noise, so report confidence intervals and refuse to certify groups too small to estimate. Second, intersectionality: gaps can appear only at the intersection (darker skin and high motion, older and female) while each marginal group looks fine, so the stratification must cross the axes, subject to sample size. This is the same leakage-safe, split-by-subject rigor from Chapter 65, now applied along demographic and environmental dimensions.

Closing the gap: data, hardware, and worst-group training

Mitigations form a rough hierarchy, cheapest-to-most-honest last. Representative data is the foundation: recruit and stratify the calibration and training cohorts across the axes that matter, and record the protected and physiological metadata so the gap can even be measured. Much sensory unfairness is simply a sampling failure that a better cohort would have prevented. Hardware and signal-level fixes come next and are often the most effective because they attack the disparity at its physical source: a second PPG wavelength, a higher-power emitter, multi-site placement, or a thermal-camera auto-calibration routine can restore SNR for the disadvantaged group directly, which no loss function can do. Algorithmic mitigation is the last resort and the weakest when the raw signal is missing: reweight the training loss toward under-represented groups, fit per-group calibration curves, or optimize the worst group explicitly with distributionally robust objectives (Group DRO minimizes \(\max_g \mathcal{L}_g\) rather than the average loss). These help when the information is present but under-weighted; they cannot manufacture a signal the sensor never captured. Finally, honest reporting closes the loop: the per-group table and the worst-group number belong on the datasheet and the dataset card (Section 70.5), so that the disparity is a documented, tracked property of the system rather than a surprise in the field.

import numpy as np

def subgroup_report(y_true, y_pred, groups, positive_label=1):
    """Per-group recall + worst-group value and best-minus-worst gap.
    Refuses to certify groups too small to estimate reliably."""
    out = {}
    for g in np.unique(groups):
        m = groups == g
        n = int(m.sum())
        pos = (y_true[m] == positive_label)
        recall = np.nan if pos.sum() == 0 else (
            (y_pred[m][pos] == positive_label).mean())
        out[g] = dict(n=n, recall=None if np.isnan(recall) else round(float(recall), 3),
                      reliable=bool(pos.sum() >= 30))   # too few positives -> noise
    valid = [v["recall"] for v in out.values()
             if v["recall"] is not None and v["reliable"]]
    worst = min(valid) if valid else None
    gap = round(max(valid) - min(valid), 3) if len(valid) > 1 else None
    return dict(per_group=out, worst_group_recall=worst, gap=gap)

rng = np.random.default_rng(0)
n = 4000
groups = rng.choice(["light", "medium", "dark"], size=n, p=[0.6, 0.3, 0.1])
y_true = (rng.random(n) < 0.2).astype(int)
# detector recall degrades as melanin rises (worse optical SNR)
recall_by_group = {"light": 0.94, "medium": 0.90, "dark": 0.71}
detect = np.array([rng.random() < recall_by_group[g] for g in groups])
y_pred = np.where((y_true == 1) & detect, 1, 0)
print(subgroup_report(y_true, y_pred, groups))

Code 70.3.1 makes the point concrete: the aggregate recall here is a comfortable 0.89, while the worst-group recall is 0.71 and the gap is 0.23. Reporting only the first number would certify a system that fails nearly one in three events for the group it already serves worst.

Fairness is a lifecycle property, not a launch checkbox

Even a system that ships fair can drift unfair. Populations change, devices are worn differently, firmware alters the signal chain, and a model retrained on newly collected field data can re-concentrate on whoever uses the product most. Fairness therefore belongs in the same continuous monitoring loop as accuracy and drift: track the per-group metrics on live traffic, alarm when the worst-group gap widens, and gate retraining on a subgroup-parity check the way you gate on aggregate accuracy. This is the fairness face of the distribution-shift monitoring in Chapter 66. Treating fairness as a one-time launch gate guarantees that the first population shift silently reopens the gap you closed.

What's Next

In Section 70.4, we turn from who the system serves to whether it can be trusted at all: the security of physical-world AI, where an adversary who can touch the sensor, the light, or the radio channel can make a perfectly fair model perceive something that is not there.