"Six teams handed me six domains and swore each was a special snowflake. Underneath, they were all the same eight boxes with different labels. My job was mostly relabeling."
A Pattern-Matching AI Agent
The big picture
The six playbooks in this chapter looked, on the surface, like six unrelated engineering problems: a wrist tracking a heartbeat, a robot avoiding a wall, a car reading a highway, a room counting its occupants, a network watching the air, a fence guarding a substation. Yet a strange thing happens once you have built two or three of them. You stop rewriting the architecture and start renaming its boxes. The same layered pipeline (measure, synchronize, represent, infer, fuse, decide, deploy, govern) reappears every time, and most of the hard-won lessons transfer wholesale. This closing section extracts that shared skeleton into a reusable system blueprint and states the cross-domain invariants explicitly, so that when you meet the seventh domain, you can instantiate a working design in an afternoon instead of rediscovering it over a quarter. It also flags the parts that do not transfer, because copying a blueprint blindly across domains is how confident teams ship confidently broken systems.
This section is a synthesis, so it leans on the whole book. It assumes the leakage-safe data discipline of Chapter 5, the uncertainty-as-output stance of Chapter 18, the fusion patterns of Chapter 48, the edge-deployment tradeoffs of Chapter 59, and the fleet-operations and governance material of Chapters 69 and 70. Its purpose is not to teach any of those again but to show how they snap together into one repeatable shape.
The eight-layer blueprint
Every sensory-AI system in this chapter decomposes into the same eight layers, and naming them makes design a checklist rather than an act of invention. (1) Sense: choose modalities and placement against the physics of what you must observe (Chapter 2), accepting that a sensor you cannot afford to place is the true limit on accuracy. (2) Synchronize and ingest: timestamp, resample, and align streams onto a common clock (Chapter 3), because unsynchronized fusion is silently wrong. (3) Represent: turn raw streams into features or learned embeddings. (4) Infer: run the per-modality model that estimates state or a label. (5) Fuse: combine modalities and time, ideally carrying uncertainty through the combination rather than averaging point guesses. (6) Decide: convert a calibrated estimate into an action, alert, or abstention under an explicit cost model. (7) Deploy: place each layer on the right compute tier (microcontroller, on-device edge, gateway, or cloud) under latency, power, and connectivity budgets. (8) Govern: log, monitor drift, protect privacy, and hold an audit trail. The domains differ only in what fills each box. The wearable's Sense layer is a PPG diode and the substation's is a fiber vibration line, but layers two through eight are structurally identical.
Key insight
What transfers across domains is the architecture and the failure taxonomy, not the model weights. A fall-detection model trained on wrists is useless on a lathe, but the discipline that produced it (split by subject and by time, calibrate the output, degrade gracefully when a sensor drops, monitor for drift after deployment) transfers perfectly. Treat the blueprint as a set of slots plus a set of standing questions that every slot must answer. The questions are the reusable asset; the weights are disposable.
The cross-domain invariants
Five lessons recurred in every playbook, strongly enough to promote to rules. First, leakage is defined by the deployment gap, not by the row count. Whether the units are people, machines, rooms, or sensor sites, the test split must hold out whole entities and future time, or the reported number describes a world that will never ship (Chapter 65). Second, uncertainty is a first-class output: a fall alarm, a lane-change, a fault flag, and a pollution warning are all decisions whose cost is asymmetric, so the model must be able to say "I do not know" and abstain. Third, a classical baseline comes before the foundation model; a Kalman filter or a spectral feature plus gradient boosting often gets 90 percent of the value at 1 percent of the compute, and it tells you whether the deep model is earning its keep. Fourth, graceful degradation is a requirement, not a nicety: sensors fail, occlude, and desynchronize in the field, so the fusion layer must produce a defensible answer from any subset of inputs (Chapter 50). Fifth, the model is the easy part: the durable cost lives in data pipelines, monitoring, and privacy, exactly the "hidden technical debt" that the operations literature warned about.
Research frontier
The blueprint's Represent layer is the one being rewritten fastest. Time-series and sensor foundation models (TimesFM, MOMENT, and Google's Large Sensor Model trained on wearable data at population scale) aim to be a shared, pretrained backbone that every domain fine-tunes rather than trains from scratch. If this holds, the reusable asset expands from "the architecture" to "the architecture plus a frozen encoder," and cross-domain transfer moves from a hopeful analogy to a measured embedding. The open question is whether one encoder can span accelerometers, radar, and vibration without a modality-alignment loss; Chapter 20 tracks the evidence.
Practical example: an elevator company borrows a wearable blueprint
A predictive-maintenance team at an elevator maker had never built a wearable, but they lifted the fall-detection blueprint of Section 71.1 almost verbatim. The wrist accelerometer became a nacelle accelerometer; the "fall" event became a "guide-rail impact"; the on-wrist abstention logic became an on-controller one. Layers two through seven ported in two weeks. The layer that did not port was ground truth. A wearable can be labeled from a lab protocol where a person is told to fall on a mat; an elevator fault is rare, expensive, and unlabeled, so the team had to redesign the Decide and Govern layers around weak labels and delayed maintenance-log confirmation (the proxy-monitoring problem of Chapter 69). The blueprint saved them a quarter; the domain still charged them for the one box it could not lend.
Instantiating the blueprint: the deployment slot
Of the eight layers, the one teams get wrong most often is Deploy, because latency, power, privacy, and connectivity pull in different directions and the "obvious" cloud default is frequently illegal or too slow. The decision is mechanical enough to encode. The function below takes a domain's constraints and returns a compute tier; the point of the listing is that the same rule adjudicates a smartwatch, a robot arm, a car, and a factory, which is precisely the cross-domain reuse this section is about.
def place_inference(latency_ms, privacy_sensitive, power_budget_mw, connectivity):
"""Recommend a compute tier for one inference layer of a sensory-AI system."""
# Hard real-time control loops cannot pay a network round trip.
if latency_ms < 20:
return "on-sensor MCU (TinyML)"
# Raw biometric/location data that must not leave the device.
if privacy_sensitive and connectivity != "always-on":
return "on-device edge (keep raw data local)"
# Tight energy budget rules out a hungry accelerator or radio.
if power_budget_mw < 50:
return "on-device edge (quantized model)"
# Heavy fusion with slack latency and a reliable link.
if latency_ms > 500 and connectivity == "always-on":
return "cloud (batch fusion + retraining)"
return "gateway (local aggregation, cloud sync)"
cases = {
"smartwatch HR": dict(latency_ms=1000, privacy_sensitive=True, power_budget_mw=5, connectivity="intermittent"),
"robot collision": dict(latency_ms=5, privacy_sensitive=False, power_budget_mw=8000,connectivity="none"),
"cabin drowsiness":dict(latency_ms=200, privacy_sensitive=True, power_budget_mw=3000,connectivity="intermittent"),
"fleet PdM": dict(latency_ms=3600000, privacy_sensitive=False, power_budget_mw=100000, connectivity="always-on"),
}
for name, c in cases.items():
print(f"{name:16s} -> {place_inference(**c)}")
The rule is deliberately crude; the value is that it makes the constraints explicit and forces the privacy question early, before an architecture calcifies around a cloud call that a regulator will later forbid.
Library shortcut
The most-reused and most-botched invariant, the leakage-safe split, is a one-liner. Hand-rolling grouped, time-respecting cross-validation (bucket rows by subject or machine, hold out whole groups, iterate folds) is 30 to 40 lines that teams get subtly wrong. scikit-learn's StratifiedGroupKFold does it in three: for tr, te in StratifiedGroupKFold(n_splits=5).split(X, y, groups=subject_id). It guarantees no group straddles the split and keeps class balance, turning the single highest-leverage discipline in this chapter into an import. Pair it with a Pipeline so scaling and feature extraction are fit only on the training fold, closing the other common leak for free.
What does not transfer, and how blueprints fail
The blueprint's danger is that it works well enough to disable suspicion. Three boxes resist porting and deserve a fresh design in every domain. Ground truth is domain-specific: a labeling scheme that is trivial for gestures is impossible for rare industrial faults or for privacy-restricted clinical events, so the evaluation and monitoring layers cannot be copied without re-examining where the labels come from. The cost model is domain-specific: a false alarm on a fitness app is an annoyance, a false negative in a vehicle safety case is a fatality and a lawsuit, and the safety-argument standards of Section 71.3 exist precisely because that asymmetry cannot be borrowed from another domain's threshold. The threat model is domain-specific: a smart-space occupancy sensor faces benign drift, while a security perimeter faces an adversary who will actively spoof it (Section 71.6), so robustness assumptions do not carry over. The reusable blueprint is a scaffold for these three decisions, never a substitute for them. Use it to skip the plumbing and spend the saved time where the domain is genuinely singular.
Exercise
Pick two domains from this chapter that you have not built (say smart-space occupancy and vehicle cabin monitoring). (a) Fill the eight-layer blueprint for each, one line per layer, and mark every layer that is structurally identical between them. (b) Identify the layers where ground truth, cost model, or threat model force a fresh design, and justify each. (c) Run place_inference from Listing 71.7.1 on both and explain any surprising tier; then propose one constraint you would add to the function to capture a factor it currently ignores (for example, a regulatory data-residency requirement).
Self-check
- State the eight layers of the blueprint from memory. Which two are most often skipped, and what silently breaks when they are?
- The section claims model weights do not transfer across domains but the architecture and failure taxonomy do. Give one concrete example of each, drawn from two different playbooks in this chapter.
- Why does the deployment-placement rule check the privacy flag before the latency-and-connectivity branch that would otherwise send data to the cloud? What real-world failure does that ordering prevent?
Lab 71
pick one domain; design an end-to-end system blueprint (sensing → modeling → evaluation → deployment → privacy).
Bibliography
System architecture and ML engineering
Sculley et al. (2015). Hidden Technical Debt in Machine Learning Systems. NeurIPS.
The foundational argument that the model is a small box in a large system, and that data dependencies, glue code, and monitoring are where the real cost lives. Directly motivates why the blueprint's Govern layer is not optional.
Breck et al. (2017). The ML Test Score: A Rubric for ML Production Readiness. IEEE Big Data.
A concrete checklist for whether a machine-learning system is ready to deploy, covering data, model, infrastructure, and monitoring tests. A practical instantiation of the standing questions each blueprint layer must answer.
A cross-industry survey of where deployed ML actually breaks, organized by pipeline stage. Empirical backing for the claim that the same failure taxonomy recurs across otherwise unrelated domains.
Cross-domain foundation models
Das et al. (2024). A Decoder-Only Foundation Model for Time-Series Forecasting (TimesFM). ICML.
Evidence that a single pretrained time-series backbone can generalize zero-shot across domains, the technical basis for treating the Represent layer as a shared, reusable asset.
Goswami et al. (2024). MOMENT: A Family of Open Time-Series Foundation Models. ICML.
An open family of general-purpose time-series models spanning forecasting, classification, and anomaly detection, showing one encoder serving tasks that used to need bespoke models per domain.
Narayanswamy et al. (2024). Scaling Wearable Foundation Models. arXiv.
Google's Large Sensor Model trained on population-scale wearable data, the clearest signal that sensor-specific foundation backbones are moving cross-domain transfer from analogy to measurement.
Deployment, edge, and privacy
The reference survey behind the Deploy layer's tradeoffs among latency, energy, and accuracy that decide which compute tier an inference layer lands on.
The canonical reference for training across a fleet without centralizing raw sensor streams, the technical spine of the privacy-preserving Govern layer that recurs in wearable, clinical, and smart-space blueprints.
What's Next
In Chapter 72, we push past the settled blueprints into the open problems: universal sensor encoders, differentiable and inverse sensing, world models for physical AI, and self-improving agentic sensing. Then the capstone asks you to do exactly what this section rehearsed, instantiate the eight-layer blueprint end to end for one application, from measurement and preprocessing through a classical baseline, a modern model, uncertainty and robustness evaluation, deployment, and a responsible-use report.