与「ScalingLaw」相关的搜索结果

腾讯混元AI负责人姚顺雨点破了做AI的人都觉得反直觉的真相： Scaling是AI里最重要的概念。 通常算力越多、数据越多、参数越多，下一个Token就预测得越准，而且改善极平滑，近似对数线性。 但模型能力平滑上涨，并不等于任务有用度平滑上涨。 数学、写代码、多模态理解……这些领域一遍遍重演同一幕。 能力在涨，真正有用的那一下却往往突然到来。 直到有了能像人一样行动的LLM，也就是Agent，它才能照着人解题的方式去解题。 大力出奇迹是真的，但“有用”从来不是线性的。 #AI# #ScalingLaw# #Agent# #腾讯混元# #姚顺雨# #Hy4# #中国开源# #马化腾#

如果科技史是一部悲剧，百度拿的一定是“神谕者”的剧本： ​它每一次都提前看穿了未来，但每一次，都成了别人的垫脚石。 ​2003 贴吧：比 Reddit 早 2 年 ​2005 知道 / 地图：早于 Quora 4 年，对标 Google Maps ​2013 轻应用：概念比微信小程序早了整整 4 年 ​2014 硅谷实验室：吴恩达带队，二作 Dario Amodei 后来创办了 Anthropic (Claude)，连 Scaling Law 的早期论文底稿都在百度手里 ​2019 ERNIE 与飞桨：在 GPT-3 前夜死磕大模型与底层框架 ​2023 文心一言：抢跑全国大厂，如今声量却依旧微妙 ​算力、人才、眼光全在手里。如果当年转型做硬科技 VC，百度的回报率恐怕能打残半个硅谷。 ​为什么握着一手王炸，最后却成了“科技慈善家”？

Thoughts About Scaling Law Scaling, but not only of parameters. Every model release now ends with the same question: how many parameters? It isn't a question that can be answered on its own. Parameter count is only meaningful alongside three others — how much data you have, where you intend to spend your compute, and who will run the model, under what conditions. The field learned this the hard way. Kaplan et al. (2020) fit an exponent that told everyone to grow parameters faster than data — roughly 2.7:1 — and the industry complied: GPT-3, Gopher, MT-NLG. Hoffmann et al. (2022) redid the experiment across four hundred models and found the compute-optimal split is closer to 20 tokens per parameter, and that with sufficient compute the two should grow at the same rate rather than drifting apart. The error in the earlier fit compounded with every order of magnitude of compute, which is why the largest models of that generation were the most misallocated. The trillion-parameter round was, in retrospect, a detour the whole field took together and then reversed. Chinchilla wasn't the end either. It optimized training compute for models that would be trained once and evaluated. Today a model is called billions of times a day and inference dominates lifetime cost. Put inference into the objective and the optimum moves toward smaller models trained far longer — deliberate over-training, which is what Llama-2-7B and Gemma-2-9B were doing at roughly 290 and 889 tokens per parameter. Sparsity moved the target again. In a MoE model two quantities have to be kept apart: total parameters govern roughly how much the model can hold — knowledge, facts, the long tail — while activated parameters and effective depth govern roughly how far it can think, how many steps of a causal chain it can carry before it comes apart. A dense 20:1 ratio does not transfer. And the ratio isn't a single number at all: Roberts et al. (2025) find the optimal tokens-per-parameter is task-dependent, with memorization favoring more parameters and reasoning favoring more data. Follow-up work on MoE observes that at fixed TPP, pushing total parameters higher actually degrades reasoning, while activating more experts reliably helps it. This matters for what we are building toward. Finding a vulnerability is not a retrieval problem. It doesn't come from having memorized more CVEs; it comes from carrying a twenty-step chain of inference to the end without losing the thread. That capability does not live in total parameter count. Which brings us to this release. Total parameters appear to matter up to a threshold — enough to hold the world — after which additional capability comes from scaling elsewhere: effective depth per forward pass, and above all post-training. GLM-5.3 is our controlled experiment on that claim. Same base, same architecture, same total and activated parameters as GLM-5.2. One month of scaling long-horizon environments and RL. The gains are not marginal. Well, scaling has more than one dial. We turned the post-training one this time because it had the most slack left in it — not because the others are finished. Base model size, pretraining data, compute spent per forward pass: all of them are still on the table, and we will come back to each. What this experiment taught us is that the dials do not have to be turned together, and that the one worth turning next is rarely the one that was worth turning last. We are not done scaling. Next time, maybe mid-training, pre-training, and even more.

2/ 📊 More human video → sharper manipulation From 1K to 1M hours—five orders of magnitude—precision improves monotonically. All four metrics, no exceptions. First time anyone's pushed a real-world manipulation scaling law to a million hours, and the curve still hasn't bent.

1/ 🧭 Robots just found their scaling variable Dyna Robotics dropped Dyna-2: a world-action model pretrained on 1M+ hours of human video—about 170 years of nonstop waking experience. The headline isn't the robot. It's the scaling law behind it—still climbing, no ceiling yet 👇

1/ 🧭 Robots just found their scaling variable Dyna Robotics dropped Dyna-2: a world-action model pretrained on 1M+ hours of human video—about 170 years of nonstop waking experience. The headline isn't the robot. It's the scaling law behind it—still climbing, no ceiling yet 👇

3/ 🤖 Human data transfers straight to robots Key result: the model never trained on a single robot trajectory, yet the more human data it saw, the better it got at robot tasks it had never seen. A scaling law crossing the "embodiment gap"—proven to exist for the first time.

Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws: • world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours, • this human data scaling law implied a scaling law on never seen robot data, • both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge 🧵

底层看稀缺，顶层看客户！看到 @chamath 分享的这张 AI Stack，感觉这个架构划分和 Rewire Index 5 Layer 相当类似，分享一下我对每一层的理解： 1. 能源与基础设施层（最底层） 与 Rewired Index 的逻辑一致。电力会迎来爆发式增长，尤其是无需接入电网的独立供电商（IPP）；而土地受政策影响太大，弹性有限。在大众的舆论压力之下，太空基建应该会是未来几年的新机会，无需土地，无限电力⚡️ 2. 芯片层 做独立芯片，初创公司基本没有机会：性能要求极高、工艺极其复杂，最关键的是供应链已被完全锁死——这是头部玩家的战场。 真正的机会在融合与生态。从 Google TPU 的发展路径，到 Cerebras 等高速推理芯片的崛起，可以看出芯片会与云厂商、模型公司深度绑定。围绕芯片构建数据中心的整体供应与创新，机会很多；单做独立芯片，机会渺茫。 3. 云服务层 云可以分为 Hyperscaler 和 NeoCloud两类。模型商品化之后，几乎所有的负载都要靠云来承载，这会是非常赚钱的生意。但构建极其复杂，堪称 AI 时代的重资产业务——或者说，智能时代的房地产。 4. 模型层 模型公司面临的核心问题是正在被商品化： - 如果 Scaling Law 已到极限，模型百分之百会被商品化； - 即使 Scaling Law 还有很大空间，大家对「最好智能」的需求也在被分解——大量日常应用不需要最顶级的智能，中等水平模型和开源模型会逐渐接管这些需求，反而加速了商品化； - 最尖端的头部模型公司，更像是「先进制程」的芯片：能从中获取很高的价值，但并非所有任务都需要它。 5. 应用层：Harness vs Application 这张图最有趣的地方，是把应用层拆成了 Harness 和 Application 两层。 我的判断是：按目前模型的进化速度，Application 还没有任何机会，但 Harness 的机会已经大量出现，尤其在企业端。企业内化的知识只能通过 Context 和约束来落地，所以 Harness 就是新的企业应用层；它们会替代旧的 SaaS，或倒逼旧 SaaS 升级。 如果把 AI 扩展到大语言模型之外的更广义范畴，应用层更可能以垂直集成的形态出现，例如：自动驾驶 / RoboTaxi / 任何可端到端自动化工业流程和武器系统；生物研究 / Wet Labs；把执行能力直接部署进企业内部的模式（类似 Palantir 的服务方式）