EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-making

Abstract

Complex decision-making often requires agents to progress through intermediate tasks rather than solve the final target directly. Existing LLM self-refinement methods typically iterate on a fixed target, while curriculum-learning methods often rely on hand-designed schedules, training-time optimization, or domain-specific difficulty metrics. To smooth out the learning curve with adaptive curriculum desgin, we introduce EvoCurr, a general inference-time framework that co-evolves curricula and executable policies. A Designer proposes verifiable intermediate tasks from the latest accepted progress and recent failures, while a Solver generates or trains policies for these tasks. A task-policy pair is accepted only when it passes hard feasibility checks and reaches a specified performance threshold meanwhile an accepted-floor constraint preserves the latest verified checkpoint and prevents failed harder attempts from overwriting mastered behavior. We evaluate EvoCurr across code-as-policy and closed-loop MARL settings on StarCraft II super-late-game micromanagement, hardest stress test, where EvoCurr achieves 96.7--99.2% macro-average win rates across three LLM backbones without LLM parameter training. The performance improvments on other experiments, Flatland, POGEMA, Overcooked, and LiveCodeBench Hard, reflect the generality of our EvoCurr.