Researchers Introduce BREW Framework for Language Agents to Learn from Experience

Researchers have introduced BREW (Bootstrapping expeRientially-learned Environmental knoWledge), a framework that enables large language model-based agents to learn from past interaction trajectories rather than starting each session from scratch. The system distills agent experiences into structured, retrievable natural-language recipes that capture what to do, when it applies, and what to watch for. BREW uses an Expand-and-Gather Monte Carlo Tree Search algorithm to jointly optimize recipe accuracy and retrievability across parallel concept-level search trees, and adapts hindsight relabeling to convert near-miss trajectories into positive demonstrations. On three benchmarks, OSWorld, tau²-Bench, and SpreadSheetBench, BREW achieved 10-20% gains in task success and 10-15% fewer execution steps over base agents, while outperforming existing memory-augmented baselines that can degrade below memoryless performance. The resulting knowledge base is inspectable, modular, and extensible.