Guanlan Dai (@guanlan) “We’re spending up to 100B tokens across 26,040 agent runs to answer one question” — TopicDigg
Guanlan Dai
@guanlan
加入 January 2010
0 正在关注 0 粉丝
We’re spending up to 100B tokens across 26,040 agent runs to answer one question: when you switch models, should you switch harnesses too?
Our v1.0 FrontierHarness leaderboard ranked harnesses on a single model.
V1.1 expands it to 21 harnesses × 5 models × 62 tasks × 4 attempts, 72x the size of v1.0.
Results are coming in live. Link below🧵
显示更多
