Adaptation
FORGE: Self-Evolving Agent Memory With No Weight Updates
Reflection is noisy in stochastic environments. Selecting and broadcasting strong memories across a population makes prompt-only adaptation substantially more reliable.
- Venue
- ACM proceedings paperACM CAIS 2026
- Role
- First author · protocol design · full implementation · experiments
- Themes
- Adaptation
- Agent architecture
Left-to-right pipeline: a failed trajectory is processed by a Reflector or Exemplifier into rules or examples. A parallel agent population runs with that memory. Champions are selected by measured return. Their memory is broadcast to the population. Graduation retires converged agents before a final evaluation. A dashed arrow shows broadcast feeding the next generation.
- 1.7–7.7×
- improvement over zero-shot
- episode return, across four model families
- 29–72%
- improvement over Reflexion
- versus isolated per-agent reflection
- ≈1%
- major-failure rate
- in the strongest settings
- ≈40%
- fewer tokens for rules
- rule memories versus example memories
01
The problem
Language-model agents can improve by reflecting on their own failures and carrying the lesson forward in the prompt. In a stochastic, partially observable environment this process is noisy: a single reflection may describe luck rather than a cause, and an agent that learns alone can lock in the wrong lesson.
Weight updates are not always an option. Many deployments use API-served models, cannot afford fine-tuning loops, or need behaviour to change within hours. FORGE asks whether the selection pressure that makes evolutionary methods robust can be applied to prompt memory instead of parameters.
The environment is CAGE Challenge 2, a stochastic adversarial network-defence POMDP. It is useful because a defender must act over a horizon under uncertainty against an attacker with its own policy, so short-horizon success does not guarantee a good episode. The attacker policy, horizon, and observation interface were held constant across all conditions so that the only variable is how memory is produced and shared.
02
System and method
The FORGE loop. Failed trajectories are distilled by a Reflector (rules) or an Exemplifier (worked examples). A parallel population of agents runs with candidate memories. Champions are selected on measured return. Their memory is broadcast to the whole population. Graduation removes converged agents to save compute before the final evaluation.
03
Experimental design
- Environment
- CAGE Challenge 2, B_line attacker, 30-step horizon
- Models
- Four model families, including a deliberately weak starting model (Gemini 2.5 Flash-Lite)
- Memory forms
- Rules (Reflector), worked examples (Exemplifier), and mixed
- Baselines
- Zero-shot agent · isolated Reflexion without population sharing
- Ablations
- Broadcast on/off · graduation on/off · rules vs. examples vs. mixed
- Primary metrics
- Episode return · major-failure rate · tokens per episode
- Cost accounting
- Per-run token counts logged for every model call
- Validation
- Repeated seeds · cross-family replication · ACM badges: Artifacts Available, Evaluated (Functional), Results Reproduced
04
Main findings
- 01
Population broadcast is the critical mechanism
Ablating champion broadcast while keeping reflection removes most of the gain. Sharing a memory that has been validated on measured return filters out reflections that only explain a lucky or unlucky episode.
29–72% over isolated Reflexion
Why it mattersReflection alone is not enough in stochastic settings. A selection step is what converts reflection into reliable learning.
- 02
Graduation mainly reduces computation
Champion broadcast is the essential mechanism. Graduation reduces compute, while its effect on final return is model-dependent.
Why it mattersCompute can be saved, but check the reliability impact per model before enabling graduation.
- 03
Examples give the strongest return, rules give the best cost–reliability balance
Worked examples produce the highest absolute return for most tested models. Rule memories reach comparable reliability while using roughly 40% fewer tokens.
≈40% fewer tokens with rules
Why it mattersMemory form is a tunable knob: choose examples for peak performance and rules when inference budget is the binding constraint.
- 04
Weaker starting models benefit disproportionately
The largest relative gain, 7.7× over zero-shot, was observed on the weakest model. In the strongest settings the major-failure rate fell to approximately 1%.
7.7× on the weakest model
Why it mattersIn this benchmark, the weakest zero-shot model benefited most, suggesting prompt-only adaptation may partly mitigate capability gaps without changing weights.
05
What I built
The engineering behind the result. Every number above was produced by systems I designed and implemented.
Agent harness
ReAct-style hierarchical reasoning harness over the CAGE-2 action and observation interface.
Population coordinator
Runs agent populations in parallel, tracks per-agent return, performs champion selection, broadcast, and graduation.
Reflector and Exemplifier
Prompted components that convert failed trajectories into rule memories or worked-example memories.
Model adapters
Multi-provider connectors with retries, pooling, and per-call token accounting.
Configuration system
Declarative experiment definitions so every condition is reproducible from a single file.
Logging and evaluation
Structured trajectory logs, failure classification, and the aggregation pipeline behind every reported number.
Container setup
Containerized environment and runner so the study can be replayed on a fresh machine.
Reproducibility artifact
Archived reproducibility artifact on Zenodo. ACM badges: Artifacts Available, Evaluated (Functional), Results Reproduced.
06
Limitations
07
Artifacts
- paperPaper (ACM DL)FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast. ACM CAIS 2026. (opens in a new tab)
- codeDevelopment repositoryforge-protocol on GitHub: harness, coordinator, configs, and analysis notebooks. (opens in a new tab)
- artifactArchived artifactZenodo snapshot used for the ACM reproducibility badges. (opens in a new tab)
08
Related research
- ArchitectureContext, Reasoning, and Hierarchy in Compound LLM AgentsWhat an agent sees can matter more than how long it deliberates.Open project
- EvaluationEvaluation of Multi-Turn Consistency in LLM AgentsReliability is temporal: models fail at different stages and produce systematically different narratives before doing so.Open project
Contact
Building reliable AI systems requires both research and engineering.
I’m interested in research engineering, applied research, agent infrastructure, evaluation, reliability, interpretability, and research-to-production work.