Human detection and monitoring

"A person is the one thing in the room I never lose in the dark. They keep glowing whether or not they want to be seen."

An Infrared AI Agent

This section assumes you can read a radiometric thermal frame as apparent temperature, which is the calibration story from Section 45.1: emissivity correction, the sensor model, and why a pixel value maps to a temperature rather than a raw count. It also leans on the measurement-model intuition from Chapter 2. Everything below treats the frame as a physical measurement, not just an image, because that is the property that makes thermal human sensing robust and also the property that breaks it when the physics turns against you.

Why humans stand out, and when they stop

The signal-to-clutter advantage of thermal human detection comes from contrast, not from resolution. In a typical indoor or nighttime outdoor scene, a person is several degrees warmer than walls, road, and foliage, producing a strong apparent-temperature edge that survives even at the modest \(320 \times 256\) or \(640 \times 512\) resolutions of affordable microbolometers. The detectable contrast is roughly

where the reflected-ambient term reminds you that shiny, low-emissivity backgrounds (glass, polished metal) mirror the environment and can wash a person out. The dangerous regime is thermal crossover: twice a day, near dawn and dusk, the background sweeps through body temperature as it heats or cools, so \(\Delta T_{\text{app}} \to 0\) and a pedestrian momentarily has almost no contrast. A detector trained only on cold winter nights will quietly lose recall on a humid summer evening when the pavement is also 33 °C. This is a distribution-shift problem in the sense of Chapter 66, and the fix is data that spans ambient temperature, not a cleverer architecture.

Detection pipelines on thermal frames

Modern thermal detection reuses RGB detector architectures (YOLO-family single-stage detectors, DETR-style transformers) with two adaptations. First, the input is single-channel radiometric data, so you either replicate it to three channels or, better, feed normalized apparent temperature and let the first convolution learn a thermal-native stem. Second, thermal images lack color and fine texture, so cues that RGB detectors lean on (clothing patterns, skin tone) are absent; the model must key on shape, warm-region topology, and gait posture. Public benchmarks make this trainable: the KAIST Multispectral Pedestrian dataset provides aligned RGB-thermal pairs across day and night, and the FLIR ADAS thermal dataset supplies annotated automotive scenes. Because thermal data is scarcer than RGB, the domain-adaptation and translation techniques in Section 45.4 and Section 45.5 exist precisely to borrow RGB supervision. Always split these datasets by capture session or geography, never by random frame, or temporal leakage inflates your numbers, the leakage-safe discipline from Chapter 5.

At the low-resolution extreme, occupancy sensing uses thermopile arrays such as the Panasonic Grid-EYE (\(8 \times 8\) pixels) or Melexis \(32 \times 24\) devices. Here you cannot run an object detector; instead you threshold apparent temperature into a foreground mask and count connected warm blobs. The snippet below shows the whole idea, which is enough to drive a room-occupancy signal for HVAC control at microwatts of compute.

import numpy as np
from scipy import ndimage

def count_occupants(frame_c, ambient_c=None, delta=2.5, min_pixels=3):
    """Count warm blobs in a low-res thermal frame (degrees Celsius)."""
    ambient_c = np.median(frame_c) if ambient_c is None else ambient_c
    warm = frame_c > (ambient_c + delta)          # human-vs-background mask
    labels, n = ndimage.label(warm)               # connected components
    sizes = ndimage.sum(np.ones_like(labels), labels, index=range(1, n + 1))
    return int(np.sum(sizes >= min_pixels))       # drop specks / hot spots

grid = np.full((8, 8), 22.0)                       # 22 C empty room
grid[2:4, 3:5] = 31.0                              # one seated person
grid[5:7, 1:2] = 30.5                              # a second person
print(count_occupants(grid))                       # -> 2

The delta parameter is the same \(\Delta T_{\text{app}}\) that thermal crossover attacks, which is why the function estimates ambient from the frame median every cycle rather than hard-coding it. The connected-component step is the low-resolution analogue of a bounding-box head; the change-detection framing generalizes in Chapter 12.

from ultralytics import YOLO
model = YOLO("yolo11n.pt")
model.train(data="flir_thermal.yaml", epochs=50, imgsz=640)  # transfer to thermal
boxes = model("night_frame.png")[0].boxes                    # detections + scores

From detection to monitoring: vitals and behavior

Detecting that a person is present is the entry point; monitoring asks what their body is doing. Because a radiometric thermal camera measures temperature, it reads physiology without contact. Skin temperature over the inner canthus (the tear duct near the nose) correlates with core temperature well enough for mass fever screening, which is why airports and hospitals deployed thermal screening at scale, always paired with a blackbody reference at a known temperature in the field of view to correct calibration drift. Respiration is visible as a periodic warming and cooling of the nostril region as exhaled air heats the skin; tracking a nasal region of interest and running a bandpass filter over its mean temperature recovers breathing rate at a distance. These contactless vital-sign methods sit alongside the radar and PPG approaches of Chapter 33, and they share its hard constraint: motion and viewing angle corrupt the signal, so a tracker and quality gate matter as much as the estimator.

Behavior monitoring builds on detection plus tracking plus pose. A warm-blob trajectory that drops to the floor and stays there is the classic thermal fall-detection cue for elderly care; the temporal-modeling toolkit for turning such trajectories into activity labels is Chapter 26. Thermal is attractive here for the same reason it raises fewer objections than an RGB camera in a bedroom or bathroom: at low resolution a person is a warm silhouette, not an identifiable face, a property Section 45.6 turns into an explicit privacy-preserving design.

Deployment realities

Three artifacts separate lab thermal from field thermal. The halo effect: microbolometer optics and readout produce a dark ring around hot objects, so a warm person against a cool wall gets an artificial cool border that a naive threshold treats as background, eroding the silhouette. Non-uniformity drift: the sensor's per-pixel offset changes with its own temperature, which is why thermal cameras periodically fire a shutter for flat-field correction, briefly freezing the video; your tracker must tolerate that gap. And fixed-pattern noise that a static model can memorize as a false landmark, so vary the camera unit across your train and test split. On the compute side, microbolometer video is low-resolution and single-channel, so a small detector runs comfortably on an edge NPU, the quantization and pruning path from Chapter 59. When a person's presence triggers an action (an alarm, a braking event, an occupancy log), you inherit the responsible-deployment obligations of Chapter 70, even though no face was ever recorded.

What's Next

In Section 45.3, we turn the same radiometric camera away from people and toward machines: industrial thermal inspection, where a warm bearing, an overloaded fuse, or a delaminated composite betrays itself as a temperature anomaly, and where the detection target is a fault signature rather than a human silhouette.