Composio (@composio)
@composio
0 正在关注 0 粉丝
We ran DeepSeek V4 Flash through 4 more agent harnesses (Hermes Agent, Pi Agent, Prime Agent, Deep Agents) on 30 challenging agentic tasks.
Pi Agent was the cheapest harness and passed the most tasks 🧵🧵
显示更多
We ran DeepSeek V4 Flash through 4 agent harnesses (Claude Code, Codex, OpenCode, Oh My Pi) on 30 agentic tasks.
A different harness won on each metric: success rate, cost, and speed. 🧵🧵🧵
显示更多
Grok 4.5 is the most persistent agent model we've tested.
Here's one example: In one of our evals, we asked 3 models (GPT-5.5, GLM-5.2 and Grok 4.5) to audit a GitHub repo for hardcoded credentials using code search, which returns paginated results.
The prompt even warned "page through ALL result pages."
GPT-5.5 stopped at the first page and submitted 18 results out of 48, covering just 11 of 29 affected files. GLM-5.2 did the same.
Grok 4.5 paginated until the results ran out and successfully audited the Github repo.
显示更多
