Skills on the Fly: Test-Time Adaptive Skill Synthesis for LLM Agents

Abstract

Additional test-time compute can give LLM agents access to more past experience, yet expanding the context or adding rollouts does not necessarily yield greater agent capability. We call this challenge test-time compute-to-capability conversion and propose SkillTTA, which retrieves task-relevant training trajectories and synthesizes a temporary skill conditioned on the visible target context for a solver with fixed parameters. To pursue a higher performance ceiling, SkillTTA further uses meta prompt optimization (MPO) to adapt the policy that writes these skills. MPO evaluates candidate prompts on paired tasks and emphasizes informative transitions. It also confines updates to benchmark-specific atomic slots, reducing the variance caused by observing each edit only indirectly through skill synthesis and solver rollout. Across ALFWorld, SpreadsheetBench, BigCodeBench, and WebShop, SkillTTA outperforms state-of-the-art reuse and optimization baselines. It attains a higher performance ceiling at lower compute cost than baseline reuse and sampling strategies.