alphaXiv (@askalphaxiv)
@askalphaxiv
High fidelity research
56 正在关注 48.4K 粉丝
"Improved Large Language Diffusion Models"
ByteDance just made bidirectional masked diffusion on-par with autoregessive LM!
This paper iLLaDA trains an 8B Transformer from scratch on 12T tokens, then keeps the same denoising objective for SFT on a 25B-token instruction corpus.
It improves LLaDA with GQA, tied embeddings, variable-length generation, confidence-based MCQ scoring, and packed-sequence diffusion SFT.
iLLaDA-Base raises the average score from 51.1 to 63.9 and slightly exceeds Qwen2.5 7B Base at 63.3, while iLLaDA-Instruct still trails Qwen2.5 Instruct without RL alignment.
显示更多
Introducing GLM 5.2 for autoresearch
GLM 5.2 is the first open weights model we've tried on our autoresearch pipeline that's proven capable for real research tasks.
With Fable 5's restrictions on research, having an open weights alternative is a huge win for open source
Watch it carry out fully async vs colocated sync RL training on Harbor code contests across two 8xH100 nodes on top of SkyRL. Resolves setup issues, tracks runs to completion, and produces a full comparison of throughput and reward stability
显示更多
Introducing autoresearch for arXiv papers
Change 'arxiv' to 'autoarxiv' in any paper URL
An agent deploys to resolve setup issues on the codebase, run a minimal reproduction, and estimate full replication cost. Read more below
显示更多
