Spiking neural networks

"You gave me a network that only speaks when something happens. I finally stopped shouting into the silence between frames."

A Newly Sparse AI Agent

This section assumes you are comfortable with backpropagation through time on recurrent models (Chapter 14) and with the event-camera output model from Section 46.1. An SNN is a recurrent network with an unusual, biologically motivated nonlinearity; if the recurrent-model vocabulary is fresh, everything here will map cleanly onto it.

The leaky integrate-and-fire neuron: a stateful, thresholded unit

A standard artificial neuron computes \(a = \phi(w^\top x)\) once and is done. A spiking neuron is a small dynamical system that lives in time. The workhorse model is the leaky integrate-and-fire (LIF) neuron. It holds an internal membrane potential \(U_t\) that integrates incoming weighted spikes and leaks toward rest between them. In discrete time with step \(\Delta t\), a compact and widely used update is

where \(\beta \in (0,1)\) is the leak (decay) factor, \(x_t\) is the input spike vector at time \(t\), \(W\) the synaptic weights, \(\theta\) the firing threshold, \(\Theta(\cdot)\) the Heaviside step, and \(S_t \in \{0,1\}\) the neuron's output spike. When \(U_t\) crosses \(\theta\) the neuron emits a spike and the \(-S_{t-1}\theta\) term subtracts the threshold back off (a soft reset), so the neuron starts integrating again. The leak \(\beta\) sets a memory horizon: information about a past input decays with time constant \(\tau = \Delta t / (1-\beta)\), which is exactly the knob that lets a neuron forget stale evidence and stay sensitive to recent events.

Three properties fall straight out of this equation and explain the whole appeal. First, the unit is stateful: \(U_t\) is a memory, so an SNN layer is intrinsically recurrent even without explicit recurrent connections. Second, the output is binary and sparse: most timesteps produce \(S_t = 0\), so downstream layers can skip the multiply entirely. Third, information lives in when spikes occur, not only how many, which is precisely the microsecond timing an event camera hands you.

Encoding sensor data as spikes

An SNN needs its input expressed as spike trains, which raises a design question a normal network never faces: how do you turn a real-valued measurement into a pattern of 0s and 1s over time? Three encodings dominate. Rate coding makes the spike probability proportional to the input magnitude, so a bright pixel fires often and a dim one rarely; it is robust but spends many timesteps to convey one number. Latency (temporal) coding fires a single spike whose delay encodes magnitude, so a strong input fires first; it is far more efficient in spike count but more fragile to noise. Direct event coding is the one that makes SNNs and event cameras a natural pair: the sensor already emits asynchronous ON/OFF events, so each event simply becomes an input spike at its own timestamp with no conversion layer at all.

That last option is the crux of why this chapter pairs the two technologies. For a conventional network you must accumulate events into a frame and lose their timing (Section 46.2); for an SNN the event stream is the native input format. The encoding choice interacts with leakage-safe evaluation too: rate coding that integrates over a long window can quietly peek at future events relative to a decision time, the temporal-leakage hazard we treat as first-class in Chapter 5. Keep the integration window strictly causal.

Training with surrogate gradients

Here is the problem that held SNNs back for years. The spike function \(S_t = \Theta(U_t - \theta)\) has a derivative that is zero everywhere and undefined at the threshold. Backpropagation multiplies gradients through the network; multiply by zero and no error signal reaches the weights. The network cannot learn by gradient descent through a true step function.

The surrogate gradient method is the fix that unlocked the modern field. Keep the hard step in the forward pass, so the network genuinely emits binary spikes, but in the backward pass replace \(\Theta'(\cdot)\) with a smooth surrogate, a bump centered on the threshold. A common choice is the derivative of a fast sigmoid,

with \(\gamma\) controlling how sharply the surrogate concentrates near threshold. Now the LIF neuron's recurrence unrolls in time exactly like an RNN and trains with ordinary backpropagation through time. This is the "spatio-temporal backpropagation" idea behind toolkits like SLAYER and the snnTorch and Norse libraries, and it is what makes SNNs trainable on the same GPUs and with the same optimizers as any other deep network. The forward pass stays faithful to the discontinuous hardware; only the gradient is softened.

There is a second, cheaper route worth knowing: ANN-to-SNN conversion. Train an ordinary ReLU network, then map each ReLU unit to a rate-coding LIF neuron whose firing rate approximates the ReLU activation. Conversion reuses mature ANN training and reaches strong accuracy, but it typically needs many timesteps to let firing rates settle, which erodes the latency advantage. Surrogate-gradient training, by contrast, can learn to decide in a handful of timesteps because it optimizes the spiking dynamics directly. The practical rule: convert when you already have a trained ANN and want a quick neuromorphic port; train with surrogate gradients when low latency and low spike count are the goal.

import torch, torch.nn as nn
import snntorch as snn
from snntorch import surrogate

class EventSNN(nn.Module):
    def __init__(self, n_in, n_hidden, n_out, beta=0.9):
        super().__init__()
        spike_grad = surrogate.fast_sigmoid(slope=25)   # the surrogate backward
        self.fc1  = nn.Linear(n_in, n_hidden)
        self.lif1 = snn.Leaky(beta=beta, spike_grad=spike_grad)
        self.fc2  = nn.Linear(n_hidden, n_out)
        self.lif2 = snn.Leaky(beta=beta, spike_grad=spike_grad)

    def forward(self, x):                 # x: [T, batch, n_in] spike trains
        m1, m2 = self.lif1.init_leaky(), self.lif2.init_leaky()
        out_spikes = []
        for t in range(x.shape[0]):       # step through time, membranes persist
            c1, m1 = self.lif1(self.fc1(x[t]), m1)
            c2, m2 = self.lif2(self.fc2(c1), m2)
            out_spikes.append(c2)
        return torch.stack(out_spikes)    # [T, batch, n_out]; decode by spike count

The loop in the code makes the recurrence concrete: membrane states m1 and m2 persist across timesteps, each layer emits spikes, and the readout accumulates output spikes into a per-class score. Swap the linear layers for convolutions and this is, in miniature, a spiking convolutional classifier for an event stream.

When to reach for an SNN, and when not to

SNNs are not a free accuracy upgrade. On dense, static inputs (a full RGB image, a saturated spectrogram) their sparsity advantage evaporates and a conventional network trained with the optimization tricks of Chapter 59 usually wins on both accuracy and, on standard hardware, latency. The SNN case is strongest when three conditions hold together: the input is temporally sparse (event camera, silence-punctuated audio, rare-event vibration), the deployment target is power-constrained and ideally neuromorphic, and low latency matters enough to justify a spiking design. Miss any one and a well-tuned ANN, possibly quantized, is the safer default.

Two engineering caveats sharpen the picture. First, an SNN's latency-accuracy tradeoff is set by the number of timesteps \(T\): more steps let firing rates average out and lift accuracy, but each step is more compute and more delay, so \(T\) is a first-class hyperparameter, not a detail. Second, the binary, discontinuous readout complicates uncertainty estimates; spike counts are not calibrated probabilities out of the box, so the calibration and conformal methods of Chapter 18 deserve a second look before you trust an SNN's confidence in a safety loop.

What's Next

In Section 46.5, we take the spiking model off the GPU and onto silicon built for it: Loihi 2 and Speck. There the sparsity we treated as a training curiosity becomes a physical power budget, event-driven cores sit idle until a spike arrives, and the SynOps count of this section turns directly into microjoules.