Architecture
Context, Reasoning, and Hierarchy in Compound LLM Agents
Deterministic context engineering provides the most reliable return per token, while distributing deliberation across an agent hierarchy can amplify hesitation and degrade decisions.
- Venue
- ACM proceedings paperACM CAIS 2026
- Role
- First author · study design · full implementation · experiments · analysis
- Themes
- Agent architecture
- Evaluation
Three columns labelled Context, Reasoning, and Hierarchy, each listing three levels of increasing cost. Context: raw observations, observation history, structured network state (up to 76% better). Reasoning: direct action, question–critique–improve, chain of thought. Hierarchy: monolithic agent, bounded specialists (can improve return), distributed deliberation (up to 3.4 times worse).
- 3,475
- episodes
- six models from five families, twelve configurations
- 283.9M
- tokens consumed
- fully accounted per configuration
- up to 76%
- lower cumulative penalty
- from programmatic, structured context
- up to 3.4×
- worse return
- from distributed deliberation across the hierarchy
- 1.8–2.7×
- token increase
- under the deliberation cascade
01
The problem
Compound agents are assembled from choices: what the model is shown, how much it is asked to reason, and whether the task is split across specialist agents. In practice these choices are made by intuition and evaluated by a single success number, so it is rarely clear which choice earned the result or what it cost.
This study isolates three axes (context, reasoning, and hierarchy) and measures each against return and token spend. The environment is again CAGE Challenge 2, a stochastic adversarial network-defence POMDP: a setting where noisy observations and a live adversary make the value of good context and the cost of hesitation visible.
The attacker, horizon, reward, and action interface were held constant. Only the agent-side design varied, so differences in return and cost can be attributed to architecture and model choice.
02
System and method
The three design axes. Context ranges from raw observations to a programmatically maintained structured network state. Reasoning ranges from direct action to question → critique → improve loops and chain of thought. Hierarchy ranges from a single monolithic agent to bounded specialists to specialists that deliberate with each other.
03
Experimental design
- Environment
- CAGE Challenge 2 adversarial network-defence POMDP
- Models
- Six models from five model families
- Configurations
- Twelve agent configurations across the three axes
- Scale
- 3,475 episodes · 283.9 million tokens
- Primary metrics
- Cumulative penalty (return) · tokens per episode · return per token
- Baselines
- Monolithic agent on raw observations with direct action
- Cost accounting
- Every model call logged with input and output tokens
- Validation
- Repeated seeds per configuration · cross-model replication · ACM reproducibility artifact badges
04
Main findings
- 01
Structured context dominates the low-cost efficient frontier
A programmatically maintained, structured view of network state produced the best return per token of any intervention: up to 76% lower cumulative penalty than raw observations, with no additional reasoning.
up to 76% improvement
Why it mattersDeterministic context engineering is cheap, reproducible, and model-agnostic. It should be exhausted before adding deliberation.
- 02
Raw observations are noisy and destabilizing
Agents fed unprocessed observations made more erratic decisions. The gap to structured context widened over the episode as errors compounded.
Why it mattersThe observation interface is part of the agent design, not a fixed input.
- 03
Bounded hierarchy can improve absolute performance
Splitting the task across specialists with narrow, well-defined interfaces improved return for several models relative to a monolithic agent.
Why it mattersDecomposition helps when each agent's responsibility and information are bounded.
- 04
Distributed deliberation degrades every tested family
Allowing specialists to deliberate with each other produced up to 3.4× worse return and a 1.8–2.7× token increase. The degradation is an interaction failure (hesitation propagating through the hierarchy), not simply the overhead of extra tokens.
up to 3.4× worse · 1.8–2.7× more tokens
Why it mattersMore agent-to-agent discussion is not a safe default. The deliberation cascade is a named failure mode to design against.
- 05
Model choice changes magnitude, and occasionally sign, of architectural effects
The same architectural change helped some models and hurt others, so conclusions drawn from a single model do not transfer automatically.
Why it mattersArchitecture studies must replicate across model families before recommending a design.
05
What I built
The engineering behind the result. Every number above was produced by systems I designed and implemented.
Harness
Hierarchical, ReAct-style agent harness over CAGE-2 with pluggable context, reasoning, and hierarchy modules.
Context engines
Programmatic state-tracking layer that converts raw observations into a structured network state.
Agent definitions
Configuration-driven definitions for monolithic, bounded-specialist, and deliberating-specialist topologies.
Model adapters
Unified multi-provider inference with retries and token accounting for six models.
Experiment runner
Parallel orchestration of 3,475 episodes with deterministic seeds and resumable runs.
Logging and evaluation
Per-step trajectory logs and an analysis pipeline producing cost–performance frontiers.
Container setup
Dockerized environment for the simulator, agents, and analysis.
Reproducibility artifact
Archived on Zenodo. ACM reproducibility artifact badges.
06
Limitations
07
Artifacts
- paperPaper (ACM DL)Context, Reasoning, and Hierarchy: A Cost–Performance Study of Compound LLM Agent Design in an Adversarial POMDP. ACM CAIS 2026. (opens in a new tab)
- codeDevelopment repositoryagent-design-study on GitHub: harness, configurations, and analysis. (opens in a new tab)
- artifactArchived artifactZenodo snapshot used for the ACM reproducibility badges. (opens in a new tab)
08
Related research
- AdaptationFORGE: Self-Evolving Agent Memory With No Weight UpdatesPrompt-only reflection becomes more reliable when useful discoveries are selected and broadcast across a population.Open project
- EvaluationEvaluation of Multi-Turn Consistency in LLM AgentsReliability is temporal: models fail at different stages and produce systematically different narratives before doing so.Open project
Contact
Building reliable AI systems requires both research and engineering.
I’m interested in research engineering, applied research, agent infrastructure, evaluation, reliability, interpretability, and research-to-production work.