Multi-agent recursion-of-thought for RCA

"One diligent agent asked me the same question for forty thousand tokens and then apologized. A tree of five agents split the fault, argued for a while, and handed me the cracked bearing race. I know which meeting I would rather attend."

A Delegating AI Agent

Why one chain of thought is not enough

Chain-of-thought reasoning is linear: the model writes one token stream from problem to answer. For a shallow, single-sensor fault that is perfectly adequate, and you should not reach for anything heavier. But industrial root causes are rarely shallow. The causal structure is a directed graph, often a fault tree, in which a top-level symptom (excess vibration, a temperature excursion, a yield drop) is reachable through many paths, and the right diagnosis is a specific path through that graph backed by specific evidence. Three pathologies make a single chain fail on such problems. The context degrades: as the transcript grows, the model attends less reliably to facts stated thousands of tokens earlier, so evidence gathered on branch one is forgotten by the time branch three is under discussion. The reasoning anchors: having proposed a plausible cause, the model preferentially reads later evidence as confirming it, the machine analogue of confirmation bias. And the budget is shared: every hypothesis competes for the same finite window, so a wide fault tree gets a shallow, hurried look at each branch.

Recursion-of-thought removes all three by making each sub-question a fresh, bounded reasoning context. When the orchestrator decides that "bearing fault" is worth investigating, it spawns a child reasoning call whose entire context is that one hypothesis plus its relevant evidence. The child cannot be distracted by the load-torque branch because it never sees it. It returns a compact verdict, and only that verdict, not its scratch work, propagates back up. The parent's context therefore grows with the number of hypotheses, not with the total reasoning spent, which is what keeps a deep tree tractable.

The cast of agents

A productive RCA team has four roles, and keeping them as distinct agents (distinct prompts, often distinct temperatures) is what stops the system collapsing back into one biased reasoner. The orchestrator owns the hypothesis tree: it holds the current frontier of open hypotheses, decides which to expand next, allocates the token budget, and stops when a leaf clears the confidence bar or the budget runs out. Investigator agents are the leaves that do the work: each takes one hypothesis, calls the tool belt from Section 22.4 (a spectral analyzer, a residual detector, a forecaster, a retrieval query over past incidents) to gather evidence, and reports a grounded verdict. The critic is an adversary whose only job is to attack the leading hypothesis: it looks for evidence that was ignored, alternative causes with the same signature, and confounds, which directly counters the anchoring failure a single chain suffers. The synthesizer takes the settled tree and writes the operator-facing explanation object, the same artifact defined in Section 22.2, now backed by an auditable tree of evidence rather than one opaque chain.

These agents communicate through a shared blackboard: a structured store of hypotheses, evidence, and verdicts that every agent reads and appends to. The blackboard is the memory that a single context window cannot be, and because it is structured data rather than free text, it is also the audit trail. When a maintenance lead asks "why did you rule out the lubrication system?", the answer is a node in the blackboard with the oil-analysis tool call that refuted it, not a paragraph the model happened to generate.

The listing below is a compact recursion-of-thought orchestrator. The LLM and tool calls are stubbed with a deterministic mock so the control structure runs and prints a hypothesis tree; in production you swap ask_llm and run_tool for real model and detector calls. The point to read is the recursion: investigate expands a hypothesis, and only its summarized verdict returns upward.

from dataclasses import dataclass, field

@dataclass
class Verdict:
    hypothesis: str
    supported: bool
    confidence: float
    evidence: list = field(default_factory=list)
    children: list = field(default_factory=list)   # sub-verdicts, summarized

# stubs: replace with a real LLM and real detector tools (Section 22.4)
FAULT_TREE = {
    "gearbox vibration high": ["bearing fault", "gear-mesh fault", "load transient"],
    "bearing fault": ["inner-race defect", "outer-race defect"],
    "gear-mesh fault": [], "load transient": [],
    "inner-race defect": [], "outer-race defect": [],
}
EVIDENCE = {           # what the tool belt "measures" per hypothesis
    "gearbox vibration high": ("detector-confirmed 40% RMS rise", 0.90),
    "bearing fault":     ("elevated high-frequency envelope energy", 0.60),
    "outer-race defect": ("BPFO peak at 162 Hz in envelope spectrum", 0.88),
    "inner-race defect": ("no BPFI sidebands", 0.10),
    "gear-mesh fault":   ("mesh frequency nominal", 0.15),
    "load transient":    ("torque flat during event", 0.05),
}

def run_tool(hyp):                       # -> (evidence_string, support_score)
    return EVIDENCE.get(hyp, ("no discriminating signal", 0.2))

def investigate(hyp, budget, depth=0):
    """Recursion-of-thought: expand one hypothesis, return a summarized verdict."""
    evidence, score = run_tool(hyp)
    children = []
    subs = FAULT_TREE.get(hyp, [])
    # only recurse into sub-hypotheses if this branch looks promising and budget allows
    if subs and score > 0.3 and budget > 1:
        per_child = max(1, budget // len(subs))
        children = [investigate(s, per_child, depth + 1) for s in subs]
        if children:                     # promote the best-supported child's confidence
            score = max(score, max(c.confidence for c in children))
    return Verdict(hyp, supported=score > 0.5, confidence=round(score, 2),
                   evidence=[evidence], children=children)

def show(v, indent=0):
    mark = "SUPPORTED" if v.supported else "ruled out"
    print("  " * indent + f"- {v.hypothesis} [{mark}, conf={v.confidence}] :: {v.evidence[0]}")
    for c in v.children:
        show(c, indent + 1)

root = investigate("gearbox vibration high", budget=8)
show(root)

Running it walks the tree, prunes the flat-torque and nominal-mesh branches on a cheap first look, descends into the bearing branch, and surfaces the outer-race defect with its envelope-spectrum evidence. That pruning is the budget discipline in action: breadth is paid for only where the first cheap measurement justifies it.

When to recurse, and when not to

Recursion-of-thought is not free: every branch is an extra reasoning call and often extra tool calls, so it multiplies both latency and token cost, the subject of Section 22.7. Reach for it only when the problem earns it. The signal is fault-tree depth and cross-subsystem coupling: if the same symptom can be produced by several subsystems and the discriminating evidence lives in different sensors, a single chain will thrash, and a tree pays off. For a shallow, single-sensor fault (a stuck valve, a saturated accelerometer as in Chapter 23) the recursion is pure overhead; the single-agent loop of Section 22.2 answers it in one call. A good orchestrator therefore gates recursion on a cheap complexity estimate: number of correlated channels that moved, number of plausible causes in the retrieved precedent set, and the confidence gap between the top two hypotheses. Recurse when the gap is small and the fan-out is wide; answer directly otherwise. Budget the tree explicitly, as the code does, so a pathological plant never spawns an unbounded thicket of agents at three in the morning.

What's Next

In Section 22.6, we confront the question this section deferred: a tree of agents can produce a beautifully structured wrong answer just as easily as a right one, so we turn to guardrails, calibrated confidence, and the human-in-the-loop checkpoints that decide when a diagnosis is trustworthy enough to act on and when it must be handed to an engineer.