介绍一下我们这个 Benchmark,它专门用来测试 Agent 的自进化能力,非常关键。
今年我们一直在挖掘 self-improving / self-evolving 这个方向,因为我们发现 Agent self-improving 在无数场景下都有实际价值。
在现在爆火的 Loop 架构中,memory-first 的自进化更是最核心的一环,可能没有之一。
正是为了把这块做扎实,我们才专门打造了这个 Benchmark。
这个 Benchmark 也是接下来我们发布 Raven 🐦⬛ 的重要一环。
Everyone is talking about self-improving agents. The harder question is how to measure whether an agent is actually getting better.
That is why we built EvoAgentBench: a benchmark for agent self-evolution.
It tests whether agents can learn from past trajectories, extract reusable skills/memory, and improve on held-out tasks across 5 domains:
- information retrieval
- reasoning and problem decomposition
- software engineering
- code implementation
- knowledge work
917 train tasks. 288 test tasks.
In the included Omni-MATH run, skill injection moved 27B from 21% to 65%, and 397B from 25% to 66%.
EvoAgentBench has 2K+ all-time downloads on Hugging Face. The benchmark card, task splits, evaluation files, and citation are open there.
Self-improving agents need more than vibes. They need measurement.
显示更多