Smart meters and load disaggregation

"One wire enters the house. One number leaves the meter every fifteen minutes. From that trickle they expect me to name the kettle, the fridge, and the exact hour someone runs a bath."

An Eavesdropping AI Agent

What a smart meter actually senses

A residential smart meter is part of an advanced metering infrastructure (AMI): a solid-state meter that measures voltage and current at the service entrance and reports consumption back to the utility over a mesh or cellular link. The critical design parameter for AI is the reporting cadence. The overwhelming majority of deployed meters log interval data: total energy accumulated over 15-minute, 30-minute, or hourly windows, giving an average power sample every few minutes at best. This is low-frequency data, and it is what you will almost always be handed. A small research and pilot fleet samples much faster, capturing real power at 1 Hz, or the raw current and voltage waveforms at kilohertz to megahertz. The sampling rate is not a detail; it determines which appliances are even distinguishable, because a fifteen-minute average smears a two-minute microwave burst into near-invisibility, an aliasing consequence of the sampling theory in Chapter 3.

What the meter reports also matters as much as how often. The simplest channel is real power \(P\) in watts, the energy actually consumed. Better meters also expose reactive power \(Q\), the energy that sloshes back and forth in motors and transformers without doing net work, and together they define apparent power \(S = \sqrt{P^2 + Q^2}\). The \((P, Q)\) plane is the workhorse feature space of classical disaggregation, because a resistive kettle (pure \(P\), near-zero \(Q\)) and an inductive fridge compressor (substantial \(Q\)) occupy visibly different regions even when they draw similar watts. High-frequency meters add far richer signatures: current harmonics, and the transient inrush shape at the instant an appliance switches on, which acts almost like a fingerprint.

From edges to states: classical disaggregation

The founding idea, due to George Hart in 1992, is event-based disaggregation. Watch the aggregate power for step changes: a sharp jump of +2000 W is an appliance turning on, a matching drop is it turning off. Cluster these edges in the \((P, Q)\) plane, pair on-edges with off-edges, and each cluster becomes an appliance. This is elegant and interpretable, and it reduces directly to the change-detection machinery of Chapter 12: an appliance switch is a mean shift in a piecewise-constant signal. Its weakness is that overlapping events, continuously variable loads (a dimmer, a laptop charger), and multi-state appliances (a washing machine cycling through heat, wash, and spin) break the clean edge-pairing story.

The dominant classical generalization is the factorial hidden Markov model (FHMM). Each appliance is a small HMM whose hidden states are its operating modes (off, low, high) with characteristic power draws; the observed meter reading is the sum of all the appliance emissions. This is the same latent-state, temporal-persistence machinery as the filters of Chapter 9, but factorized across many parallel chains. FHMMs handle multi-state appliances gracefully and encode the crucial prior that appliances stay in a state for a while rather than flickering. The cost is inference: the joint state space is the product of every appliance's states, so exact inference is intractable and you rely on approximate methods.

Neural disaggregation and the seq2point trick

Since roughly 2015, deep learning has reframed NILM as a supervised sequence-to-sequence regression: feed a window of aggregate power and predict the power trace of one target appliance. The influential refinement is sequence-to-point (seq2point), which takes a window of the mains signal and predicts the target appliance's power at only the midpoint of that window. Predicting a single centered value instead of the whole window lets the network use both past and future context to disambiguate the moment of interest, and it sidesteps the awkward averaging of overlapping window predictions. The backbone is typically a temporal convolutional or recurrent network from Chapter 14, trained per appliance on labelled sub-metered data.

The catch is data and generalization. Neural NILM needs ground-truth appliance-level power to train against, which comes from instrumented research homes: REDD, UK-DALE, and REFIT are the canonical public datasets, each a handful of houses with plug-level sub-meters. A model trained on these houses must transfer to unseen homes with different appliance makes, and this is exactly where leakage discipline bites. If you evaluate on the same houses you trained on, you measure memorization, not disaggregation; honest NILM benchmarks split by house, holding entire unseen homes for test, the unseen-entity protocol argued in Chapter 5. The gap between same-house and unseen-house accuracy is large, and reporting only the former is the single most common way NILM results are oversold.

import numpy as np

# One day of 1-minute mains power (W): a fridge cycling + a 2-minute kettle burst.
t = np.arange(0, 1440)
fridge = 90 * ((t // 40) % 2)          # ~90 W compressor, on/off every 40 min
kettle = np.where((t >= 420) & (t < 422), 2000, 0)   # 07:00, two minutes
mains  = 30 + fridge + kettle + np.random.normal(0, 5, t.size)  # +baseload +noise

def detect_events(x, threshold=500):
    """Return indices where step change in power exceeds threshold (an on/off edge)."""
    dx = np.diff(x)
    return np.where(np.abs(dx) > threshold)[0]

events = detect_events(mains)
print("edge times (min):", events)                 # -> near 420 (on) and 422 (off)
print("kettle energy (Wh):", kettle.sum() / 60)    # integrate W over hours

Listing 39.1 makes the resolution problem concrete: the same code that cleanly finds the kettle at 1-minute sampling would miss it at 15-minute sampling, because the burst is shorter than one sample. That is not a bug in the detector; it is the sampling theorem deciding what is knowable, and it is why high-value appliances often demand higher-rate meters.

from nilmtk import DataSet
from nilmtk.disaggregate import CombinatorialOptimisation

data = DataSet("ukdale.h5")
data.set_window(start="2014-01-01", end="2014-02-01")
elec = data.buildings[1].elec
co = CombinatorialOptimisation()
co.train(elec)                          # learn appliance power states
predictions = co.disaggregate(elec.mains())