What's actually happening during the plateau?
The model isn't stuck. It's reorganizing its internal representations.
During 10,000 "useless" epochs:
→ Circuits form and dissolve
→ Weight patterns crystallize
→ Spurious correlations get pruned
→ True structure emerges
It's like insight in humans. Slow, then sudden.
显示更多