Paweł Huryn (@PawelHuryn)
@PawelHuryn
0 正在关注 0 粉丝
What a day! I tested Gemini 4 Carbon on real work. 2 repos, 105 replanted bugs. Find and fix what you can.
It scores just behind Fable 5.2. The results:
- Fable 5.2 (max): 58 for $139.65
- Gemini 4 Carbon (high): 57.3 for $31.32
- Sonnet 5.5 (max): 51.3 for $154.03
- GPT-6 Astra (max): 45 for $33.03
- GPT-6.1 Sol (max): 44.3 for $5.88
- Gemini 4 Argon (high): 40.7 for $7.16
Google's own benchmarks looked super strong, but nobody could access the model. Turns out they didn't lie: +16.6 points over Argon, at less than a quarter of Fable 5.2's cost.
Nobody should be laughing at Google anymore.
(the next effort level drops soon in the thread 🧵)
显示更多
Tried Grok 4.6 on my bug bench an hour after release. 105 hidden bugs in two real repos, judged blind.
Grok 4.5: 17 (+10 non-planted)
Grok 4.6: 27 (+15 non-planted)
Fable 5: 29 (+2 non-planted)
Looks like it may be my new default model. The best combination of time, value, and cost.
4.7 is dropping soon. That may be an even bigger jump.
显示更多
Try Grok 4.6 on tough real-world tasks!
