注册并分享邀请链接,可获得视频播放与邀请奖励。

艾略特 (@elliotchen100) “介绍一下我们这个 Benchmark,它专门用来测试 Agent 的自进化能力,非常关键。 今年我” — TopicDigg

艾略特 的个人资料封面
艾略特 的头像
艾略特
@elliotchen100
加入 December 2023
0 正在关注    0 粉丝
介绍一下我们这个 Benchmark,它专门用来测试 Agent 的自进化能力,非常关键。 今年我们一直在挖掘 self-improving / self-evolving 这个方向,因为我们发现 Agent self-improving 在无数场景下都有实际价值。 在现在爆火的 Loop 架构中,memory-first 的自进化更是最核心的一环,可能没有之一。 正是为了把这块做扎实,我们才专门打造了这个 Benchmark。 这个 Benchmark 也是接下来我们发布 Raven 🐦‍⬛ 的重要一环。
显示更多
Everyone is talking about self-improving agents. The harder question is how to measure whether an agent is actually getting better. That is why we built EvoAgentBench: a benchmark for agent self-evolution. It tests whether agents can learn from past trajectories, extract reusable skills/memory, and improve on held-out tasks across 5 domains: - information retrieval - reasoning and problem decomposition - software engineering - code implementation - knowledge work 917 train tasks. 288 test tasks. In the included Omni-MATH run, skill injection moved 27B from 21% to 65%, and 397B from 25% to 66%. EvoAgentBench has 2K+ all-time downloads on Hugging Face. The benchmark card, task splits, evaluation files, and citation are open there. Self-improving agents need more than vibes. They need measurement.
显示更多
0
28
105
15
转发到社区