this paper should scare every agent framework out there
researchers took apart 11 coding agents straight from the source code
claude code codex gemini cli openhands aider and more
roughly 4 million lines
not one of them imports a langchain style framework
not one of them searches code with embeddings
9 out of 11 already ship skill.md
then a second paper went further and let a loop write the harness
the model stays frozen
a debugger agent reads every failed run
an evolve agent edits the harness files
the next round keeps the edit or rolls it back
10 rounds later a one tool seed went from 69.7% to 77.0% on terminal bench 2
the codex harness people built by hand sits at 71.9%
craziest part
editing only the system prompt made it worse
all the gains came from memory, tools and middleware
agent = model + harness
the model takes the bow the harness does the work
drop claude fable 5.2 in that frozen seat and the loop rewrites everything around it
both papers in the reply
显示更多