"I trained flawlessly on wrist data, then a user put the watch on upside down and I decided walking was falling."
An Orientation-Naive AI Agent
The big picture
An inertial measurement unit does not report motion in the world. It reports motion in its own body frame, and that frame is glued to however the device happens to sit: a phone face-up in a pocket, the same phone upside down in a bag, a watch on the left wrist or the right, a machine-mounted sensor bolted at whatever angle the bracket allowed. Change the mounting and every raw axis changes, even though the physical activity is identical. Orientation is therefore the single largest nuisance covariate in inertial machine learning: a model that memorizes raw axes learns the mounting, not the motion, and collapses the first time the device is worn differently. This section builds the preprocessing stage that neutralizes that covariate. We resample the stream onto a clean clock, separate gravity from motion, rotate the signal into a mounting-independent frame, decide between building invariance in or augmenting it in, and package everything (including the validity mask from the previous section) into fixed windows that a network or a foundation model can consume without secretly learning to depend on how the sensor was strapped on.
This section assumes you can name the body, sensor, and world coordinate frames and convert between them from Section 23.2, that gravity has been separated from linear acceleration as in Section 23.3, and that corrupt samples already carry the validity mask built in Section 23.6. It also leans on the sampling and synchronization discipline of Chapter 3, because a rotation is only meaningful when the three axes it mixes were sampled at the same instant. What we produce here is the tensor that feeds the activity models of Chapter 26 and the orientation estimators of Chapter 24.
Orientation is the hidden covariate
Write the reading of a triaxial sensor in its own body frame as \(x_b\). If the same device were mounted with a rotation \(R\) relative to some reference, the world-frame motion \(x_w\) relates to the reading by \(x_b = R^\top x_w\). A model \(f\) trained on \(x_b\) and evaluated on a differently mounted \(x_b' = (R')^\top x_w\) sees an input that can be arbitrarily far from anything in training, even though the underlying activity is unchanged. This is not overfitting in the usual sense; the training distribution genuinely did not cover the deployment mounting. Two devices, two brackets, or two users who wear a watch differently are enough to break a naive model.
There is a crucial asymmetry that makes the problem tractable. Gravity is always present and always points down, so the accelerometer's low-frequency component is a free, absolute observation of two of the three rotational degrees of freedom: roll and pitch. Only heading (rotation about the vertical, yaw) is left ambiguous from the accelerometer alone, and resolving it needs a magnetometer or integrated gyroscope, which is the business of Chapter 24. So the practical goal of orientation preprocessing is modest and achievable: rotate every window so that gravity points along a canonical axis, removing the two degrees of freedom you can pin down cheaply, and then handle the residual heading either by invariance, by augmentation, or by simply accepting it.
Key insight
Gravity is a built-in orientation reference that costs nothing to measure. Low-pass the accelerometer and you have a vector that is guaranteed to point down; rotating each sample so that vector lands on \([0,0,1]\) removes roll and pitch entirely and leaves only heading unresolved. That single step converts a three-degree-of-freedom nuisance into a one-degree-of-freedom nuisance, which is why gravity alignment is the backbone of almost every orientation-robust inertial pipeline. The catch: the estimate is only as good as your gravity/linear-acceleration split, so during vigorous motion the low-pass "gravity" is contaminated and the alignment wobbles. That coupling to Section 23.3 is not incidental; a better gravity estimate is directly a better frame.
Two roads: invariance by construction versus learned equivariance
There are two defensible strategies, and mature systems often blend them. The first is invariance by construction: transform the raw signal into features that do not change under rotation, so the model never sees the nuisance at all. The acceleration magnitude \(\lVert a \rVert\) is fully rotation-invariant; the gravity-aligned vertical and horizontal components are invariant to heading; pairwise axis correlations and the gyroscope's rotation-invariant angular-rate norm add more. This is cheap, interpretable, and shrinks the input, which matters on the microcontrollers of Part XII. Its cost is that a hard invariance discards information: once you keep only \(\lVert a \rVert\), a model can no longer tell a forward lean from a sideways one, and some activities become indistinguishable. Feature-engineering that trades expressiveness for invariance is exactly the design tension of Chapter 8.
The second road is learned equivariance through augmentation: keep the full three-axis signal but, during training, apply random rotations to every window so the model sees each motion under many mountings and learns to be robust on its own. This preserves directional information while still discouraging dependence on any one frame, and it is the dominant approach for deep models. Random-rotation augmentation is now standard practice for wearable networks and a core positive-pair transform for the contrastive self-supervised methods of Chapter 17. The two roads are not exclusive: a common recipe gravity-aligns first (removing roll and pitch deterministically) and then augments only the residual heading (random rotation about the vertical), getting most of the robustness of augmentation at a fraction of the variance.
Building the pipeline
A production inertial pipeline is a fixed sequence: resample onto a uniform clock, separate gravity, rotate into the gravity-anchored frame, window, normalize, and attach the validity mask as an extra channel. The order matters. Resampling comes first because rotation mixes axes and is meaningless across a timing jitter; masking comes last because normalization statistics must be computed over valid samples only, or a saturated span poisons the mean. The core geometric step, gravity alignment, is short enough to write from scratch, and doing so once makes the abstraction concrete.
import numpy as np
def gravity_align(acc, gyro, fs, g_cutoff=0.3):
"""Rotate each sample into a gravity-anchored frame (z = up).
acc, gyro: (N, 3) resampled arrays; fs: sample rate in Hz.
Removes roll and pitch; heading (yaw) is left unresolved."""
# 1. Estimate gravity: a one-pole low-pass keeps only the DC-ish tilt.
alpha = np.exp(-2 * np.pi * g_cutoff / fs)
grav = np.empty_like(acc)
grav[0] = acc[0]
for i in range(1, len(acc)):
grav[i] = alpha * grav[i - 1] + (1 - alpha) * acc[i]
g_hat = grav / (np.linalg.norm(grav, axis=1, keepdims=True) + 1e-9)
up = np.array([0.0, 0.0, 1.0])
acc_r, gyro_r = np.empty_like(acc), np.empty_like(gyro)
for i in range(len(acc)):
v = np.cross(g_hat[i], up) # rotation axis
s, c = np.linalg.norm(v), g_hat[i] @ up
if s < 1e-8: # already up or fully inverted
R = np.eye(3) if c > 0 else np.diag([1.0, -1.0, -1.0])
else: # Rodrigues' formula
vx = np.array([[0, -v[2], v[1]], [v[2], 0, -v[0]], [-v[1], v[0], 0]])
R = np.eye(3) + vx + vx @ vx * ((1 - c) / (s * s))
acc_r[i], gyro_r[i] = R @ acc[i], R @ gyro[i]
return acc_r, gyro_r
The routine above is the geometric heart of the pipeline referenced throughout this section. After it runs, the vertical acceleration channel means the same thing whether the phone was face-up or face-down, which is precisely what lets a downstream network of Chapter 14 generalize across mountings. Two design notes. First, the smoother cutoff \(g_{\text{cutoff}}\) trades a steadier gravity estimate (lower cutoff) against faster response to genuine tilt changes (higher cutoff); 0.2 to 0.5 Hz is typical for human wear. Second, the per-sample Python loop is written for clarity, not speed; the library shortcut below vectorizes it.
In practice: a fall detector that survived pocket flips
A clinical wearables team shipped a hip-worn fall detector trained on data where the device was always clipped in one orientation. In the field, caregivers re-clipped it however was convenient, sometimes rotated 180 degrees, and the false-alarm rate tripled: a brisk sit-down in a flipped mounting produced an acceleration pattern the model had only ever seen during simulated falls. The fix was pure preprocessing, no new labels. They inserted gravity alignment so vertical always meant vertical, then added random heading rotation as a training augmentation to cover the residual yaw. Cross-mounting false alarms fell back to the original level and, as a bonus, the model transferred to a wrist form factor with far less new data, because the gravity-anchored representation already spoke the same geometric language. The lesson generalizes past falls: orientation robustness is a data-representation problem, and it is almost always cheaper to fix in preprocessing than to paper over with more labels. Leakage-safe evaluation, in the spirit of Chapter 5, meant testing on held-out subjects and mountings, not random windows, so the robustness gain was real rather than a split artifact.
Right tool: rotations and augmentation from a library
The from-scratch alignment is about twenty-five lines with a hand-coded Rodrigues formula and an explicit loop. scipy.spatial.transform.Rotation vectorizes the whole thing and gives you random rotations for augmentation for free.
from scipy.spatial.transform import Rotation as Rot
import numpy as np
def align_and_augment(acc_r, rng, yaw_only=True):
# Random heading rotation as an augmentation (already gravity-aligned).
ang = rng.uniform(0, 2 * np.pi)
R = Rot.from_rotvec([0, 0, ang]) if yaw_only else Rot.random(random_state=rng)
return R.apply(acc_r) # vectorized over all samples at once
scipy. Rotation.apply replaces the per-sample matrix loop, and from_rotvec / random generate the augmentation transforms in one call.Roughly twenty lines of loop and manual matrix algebra collapse to two, and the library handles the numerically careful cases (near-parallel vectors, normalization) that the hand-rolled version guards against with epsilons. You still choose the alignment strategy; the library only removes the geometry boilerplate.
Windowing, normalization, and carrying the mask
With the stream in a stable frame, the last steps assemble training tensors. Windowing slices the continuous signal into fixed-length segments (commonly one to a few seconds, 50 percent overlap) because most inertial models classify or embed a window, not a single sample; window length is a genuine hyperparameter that trades latency against how much of a gait cycle each example contains. Normalization should use statistics computed per channel over valid samples only, and, critically, fit on the training split alone and reused on validation and test, or you leak future statistics backward. The validity mask from Section 23.6 rides along as an extra input channel so the network learns which samples to distrust rather than treating a saturated pin as real motion. Package the result as a tensor of shape (window, channels) where channels include gravity-aligned acceleration, gyroscope, useful invariant features such as \(\lVert a \rVert\), and the mask. That tensor is the contract this chapter hands to the models of Chapter 26 and to the wearable foundation models of Chapter 20, which increasingly expect exactly this canonicalized, masked, fixed-window form.
Exercise
You have a smartphone accelerometer dataset for walking, collected with the phone always face-up in a chest pocket, and you must deploy to phones carried in any orientation. (a) Explain why training directly on the raw axes will fail, in terms of the covariate \(x_b = R^\top x_w\). (b) Apply gravity alignment and state precisely which rotational degree of freedom remains unresolved and why the accelerometer cannot fix it. (c) Choose between hard rotation-invariant features and random-rotation augmentation for the residual, and justify your choice given that you also need to distinguish walking upstairs from walking on the flat. (d) Describe one way your gravity estimate degrades during a fast descent and how it corrupts the alignment.
Self-check
1. Why must resampling onto a uniform clock happen before any rotation that mixes the three axes?
2. Gravity alignment removes two of three rotational degrees of freedom for free. Which two, and what physical fact makes them observable from a single accelerometer?
3. You normalize each window using its own mean and variance instead of training-split statistics. Name two distinct problems this causes, one about information leakage and one about destroying a feature the model needs.
Lab 23
build an IMU preprocessing pipeline for orientation-aware activity recognition.
Bibliography
Orientation-robust representations and features
Develops explicit transformations that make inertial features invariant to how the sensor is oriented on the body, the formal grounding for the invariance-by-construction road in this section.
A systematic study of how window length trades recognition accuracy against latency, the empirical basis for the windowing hyperparameter choices here.
Augmentation and self-supervised transforms
Introduces rotation, jitter, and warping augmentations for inertial data; the rotation augmentation is the canonical technique for learned orientation robustness.
Uses signal transformations, including rotations, as self-supervision signals, connecting orientation preprocessing to the contrastive learning of Chapter 17.
Deep models and datasets for inertial HAR
The DeepConvLSTM architecture that consumes windowed, multichannel inertial tensors of exactly the form this pipeline produces.
The widely used UCI HAR dataset, whose gravity/body-acceleration split and fixed windows mirror the preprocessing steps taught here.
Foundation models for wearable inertial data
Narayanswamy, G. et al. (2024). Scaling wearable foundation models (LSM). arXiv:2410.13638.
Google's Large Sensor Model, trained on massive wearable streams; it expects the canonicalized, masked, fixed-window inputs that this section standardizes.
Large-scale accelerometer pretraining showing that rotation-aware augmentation and gravity-anchored representations scale to population-level models.
What's Next
In Chapter 24, we stop treating orientation as a nuisance to remove and start estimating it as a quantity of interest: quaternions and rotation representations, complementary and Madgwick/Mahony filters that fuse accelerometer, gyroscope, and magnetometer into a full attitude, and the dead-reckoning that turns those orientations into a trajectory when no external position fix is available.