Field update and rollback; MLPerf Tiny

"Shipping the model was the easy part. The hard part was promising a million sleeping devices that I could change my mind later without bricking a single one."

A Reversible AI Agent

This section closes the deployment arc of the chapter. It assumes the firmware integration and on-hardware testing of the previous section, and the quantized runtimes (LiteRT for Microcontrollers, microTVM, ExecuTorch) that produce the model artifact you are about to ship. It connects outward to fleet-scale operations: the mechanisms here are the device-side half of the release pipeline detailed in Chapter 69, and the leakage-safe measurement discipline that MLPerf depends on is developed in Chapter 5. We do not re-derive quantization or benchmark statistics; we ask how to change a fielded model without breaking it, and how to prove a model is good enough to be worth changing to.

The dual-bank invariant: never trust a new image until it proves itself

The foundational technique for safe MCU update is A/B partitioning, also called dual-bank flash. You divide the program flash into two slots. Slot A holds the running firmware-plus-model image; slot B is idle. An update writes the new image entirely into slot B while slot A keeps running, so a power loss or a corrupt download during transfer costs you nothing: slot A is untouched. Only when the new image is fully received, its cryptographic signature verified, and its integrity hash checked does the bootloader flip a pointer and boot slot B on the next reset. The invariant that makes this safe is simple to state and unforgiving to violate: there is always at least one known-good, bootable image on the device. You never erase the slot you are running from, and you never boot an unverified image as if it were trusted.

The subtlety that separates a demo from a product is what happens after the first boot of the new image. A signature proves the image came from you; it says nothing about whether the image works on this particular device with this particular sensor and this particular flash wear pattern. So the new image boots in a trial state and must earn permanence. The bootloader arms a hardware watchdog and marks slot B as "pending confirm." The new firmware, if it is healthy, runs its self-tests, confirms the sensor reads sane values, runs a canary inference, and only then writes a "confirmed" flag that tells the bootloader to make slot B the new default. If the image hangs, crashes, or fails its health check, the watchdog fires, the device resets, and the bootloader, seeing an unconfirmed pending image, rolls back automatically to slot A. No human, no network, no truck.

Shipping models, not just firmware: delta updates and split artifacts

Sending a full firmware image over a constrained radio (a BLE link, a LoRaWAN uplink of a few hundred bytes per message, an NB-IoT trickle) to a battery device is expensive in both energy and airtime, and the model is usually the part that changes most often while the driver and runtime code stay fixed. Two techniques cut the cost. First, split the artifact: store the model weights in their own flash region with their own version and signature, separate from the runtime firmware, so a model refresh ships only the tensor blob and not the whole binary. Second, use delta (differential) updates: transmit only the byte-level difference between the old and new image, reconstructed on-device against the current slot. For a fine-tune that nudges a few layers, a delta can be a small fraction of the full weight size, turning a multi-minute radio session that drains the cell into a short one. The cost is a patch-apply step that needs scratch space and must itself be power-fail-safe, which is exactly why it happens into the idle B slot, never in place.

The listing below is a host-side simulation of the confirm-or-revert state machine, so you can reason about the fleet-health logic without an MCU in front of you. It models a staged rollout where each device boots the new image in trial mode, runs a health gate, and either confirms or reverts, and it stops the rollout automatically if the confirmed fraction across a batch falls below a safety threshold.

import random

def deploy_to_device(healthy_prob, watchdog_s=8.0):
    """One device: stage image in slot B, boot trial, health-gate, confirm or revert."""
    # Boot the pending image under a watchdog. A broken image never reaches the gate.
    booted = random.random() < 0.995              # 0.5% brick on first boot
    if not booted:
        return "revert:watchdog"                  # bootloader restores slot A
    # Health gate: sensor sane + canary inference within latency budget.
    if random.random() < healthy_prob:
        return "confirm"                          # boot_set_confirmed(): B becomes default
    return "revert:healthgate"                    # app declines to confirm -> A on next reset

def staged_rollout(fleet=2000, batch=200, min_confirm=0.98, healthy_prob=0.99):
    deployed = 0
    while deployed < fleet:
        results = [deploy_to_device(healthy_prob) for _ in range(min(batch, fleet - deployed))]
        confirmed = sum(r == "confirm" for r in results)
        rate = confirmed / len(results)
        deployed += len(results)
        print(f"batch of {len(results):4d}: confirm rate {rate:5.1%}  (cumulative {deployed})")
        if rate < min_confirm:
            print(f"HALT: confirm rate {rate:.1%} below gate {min_confirm:.0%}; pausing rollout")
            break                                 # no truck roll: fielded devices already reverted
    return deployed

random.seed(0)
staged_rollout()

The code makes the operational point concrete: because reversion is automatic and device-local, a bad batch is self-limiting. The fleet-health monitor merely has to stop sending the bad image; it never has to reach in and rescue devices, because the watchdog and health gate already did. This is the device-side counterpart of the canary-release strategy for whole sensor fleets in Chapter 69, and it composes with the on-device learning safety concerns of Chapter 62, where the "new model" is one the device trained itself.

MLPerf Tiny: proving a model is worth shipping

Before any of that update machinery runs, you have to decide the new model is actually better on the axes that matter for a microcontroller: latency, energy, and accuracy at a fixed, tiny memory budget. MLPerf Tiny, maintained by MLCommons, is the standard benchmark for exactly this regime. It is deliberately small: four reference tasks chosen to represent the always-on sensing workloads of this chapter. Keyword spotting (a 12-class wake-word task on the Speech Commands data), visual wake words (person-present binary classification on a tiny image), image classification (CIFAR-10 at low resolution), and anomaly detection (autoencoder-based machine-condition monitoring on the ToyADMOS/DCASE data). Each task ships a reference model, a reference dataset split, and a reference quantization, so submissions differ only in the system under test, not in what is being asked.

MLPerf Tiny reports three things, and the discipline is in reporting all three together on the same configuration. Latency is the median time for a single inference. Energy is the joules per inference, measured with a standardized power harness (the EEMBC EnergyRunner), because on a coin cell energy, not speed, is the currency that runs out. Accuracy is the task metric (top-1, or AUC for anomaly detection), which acts as a quality floor: you may not trade accuracy away silently to win on latency. Crucially, the benchmark separates a closed division, where you must run the exact reference model so hardware and runtimes compete on equal footing, from an open division, where you may substitute your own model (a TinyNAS-found network, an aggressively pruned variant) and compete on the full co-design. The closed division answers "how good is my silicon and runtime," the open division answers "how good is my whole stack."

The leakage-safe discipline of Chapter 5 is what makes a MLPerf number trustworthy. Because the benchmark fixes the dataset split and the reference preprocessing, a submitter cannot quietly leak test data into calibration or tune quantization on the evaluation set; the harness and the closed-division rules enforce a single honest measurement. When you report your own model's efficiency against MLPerf Tiny, hold the same line: measure accuracy, latency, and energy in one pass on one configuration, never stitch the best accuracy from one build to the best latency from another. That is not merely good manners; it is the only way the number predicts field behavior.

When to update, and when to leave it alone

The final judgment is operational restraint. Every OTA update spends energy, spends airtime, and introduces a nonzero brick risk, however small. A model that is 0.3 percent more accurate on a clean benchmark is not automatically worth pushing to a million battery devices, because the update itself costs battery and the improvement may not survive contact with real, drifting sensor data. Update when the change fixes a real field failure (a false-alarm pattern, a security vulnerability, a lot-specific calibration bug), when the accuracy gain is large enough to clear the energy and risk cost of the transfer, or when a security patch is non-negotiable. Otherwise, the safest fielded model is often the one already running, proven over months. Reversibility exists so you can change your mind cheaply; MLPerf Tiny exists so you change it for a measured reason, not a hunch.

What's Next

In Chapter 62, the "new model" stops arriving from a server and starts being trained by the device itself. We turn from pushing updates to a fleet to letting each node adapt in place, and we confront catastrophic forgetting, quantized replay, and the sharper version of the safety question this section opened: how do you keep rollback and health gates meaningful when the model changes without anyone signing it?