与「LatentMAS」相关的搜索结果

多智能体不用说话就能协作了 🧠 LatentMAS 入选 ICML 2026 Spotlight——Agent 直接在隐空间传递推理状态，跳过文字编解码。 准确率 +13.3%，速度 4.3 倍，token 用量少 83.7%。 不额外训练，插进现有 LLM 直接用。 这个方向你们觉得会成主流吗？

Excited to share that #LatentMAS# has been accepted to ICML 2026 as a spotlight! 💻Code: 📄Paper: We push multi-agent collaboration into the latent space — beyond human language. Most multi-agent systems rely on text: agents reason in words, exchange messages, and repeatedly decode/re-encode information. But language can be slow, lossy, and unnecessarily constrained. 💡LatentMAS takes a different path: LLM agents reason and communicate directly through hidden embeddings. No text decoding. No extra training. No token-level message passing. Instead, agents collaborate through: 🧠 Autoregressive Latent Thoughts — hidden-state-level reasoning steps 🔁 Latent Communication — information sharing via KV-cache transfer 📌 Input-output Alignment — keeping latent representations in-distribution 🚀 Training-free Collaboration — plug-and-play with existing LLMs Why it matters: ✅ Up to +14.6% better accuracy on complex reasoning tasks ⚡ 4-4.6x faster end-to-end inference ✂️ 70.8%–83.7% reduction in output token usage A step toward multi-agent systems that collaborate not by speaking more, but by thinking together in latent space. #MultiAgentSystems# #ModelCollaboration# #LatentReasoning# #LLM# #AgenticAI# #ICML#

转一下苏剑林老师对 K3 架构的复盘。 一句话概括，K3 = KDA + MLA + Stable LatentMoE + AttnRes。整套设计没什么炫技，核心就是在模型效果、计算效率和训练稳定性之间做取舍。 这里稍微解释一下： KDA，一种线性 Attention MLA，一种 KV Cache 较小的 Full Attention 变体 Stable LatentMoE，对 LatentMoE 做过稳定性改造的 MoE 方案 AttnRes，用可学习的跨层注意力，替代固定等权的残差累加 几个比较有意思的点： ① K3 同时使用了 KDA 和 MLA。另外，训练仍然用 Moonlight 版本的 Muon 优化器，Attention 权重改成 Per-Head Muon，每个 Head 独立优化。苏神说这不会直接提高效果，主要是数学和结构上更合理：各个 Head 本来就相对独立，不应该在优化时耦合在一起。 ② 解决MoE“容易炸”的问题。 LatentMoE 会先降维，再使用更多专家，最后升维，在训推成本大致相同的情况下效果略好。不过连续的矩阵投影也让训练更容易出现数值不稳定。 K3 把 SwiGLU 换成了 SiTU-GLU，用 softcap 压住异常激活；又在 LatentMoE 升维前加了一层 RMS Norm。这个 Norm 不只是让训练更稳定，在 Valid Loss 差不多的情况下，不加它，某些 Benchmark 会稳定变差。 （这里有超级多的技术细节，感兴趣的可以看苏神原文） ③ 关于MLA DSV4 看上去都换设计了，K3 怎么还在用 MLA？苏神的答案是，目前 MLA 仍然很难被全面击败。它在训练阶段是 MHA 形态，推理时 KV Cache 较小；在固定训练成本和 KV Cache 大小时，MLA 依然近乎最优。 它的问题是对 MTP（推测解码）不够友好。MTP 的思路是“用计算换速度”，而 MLA 在 Decoding 阶段本身就比较吃计算，再叠一个 MTP，两边就容易抢算力。 换别的方案也有代价。比如换成 128+128 的 GQA8，效果很难打赢 MLA，KV Cache 还是 MLA 的三倍多；换成 256+256 的 MFA（本质上是 MQA），训练和 Prefill 成本又会上去。 目前还没有一个简单的 Attention 设计，能同时占住效果、训练成本、Prefill、KV Cache 和 Decoding 计算量。在 KDA+MLA 的混合架构下，MLA 的部分问题得到缓解，所以 K3 最后还是选了 MLA。 ④ 苏神觉得，DSV4 也不算真正“抛弃 MLA”。 DSV4 看上去重新设计了 Attention，但底层仍然有 MLA 的影子。它采用的是 head_dims=512、K=V 的 MQA，这正是 MLA 在 Decoding 阶段的形态；再加上 Sparse + Compress，前者减少计算，后者进一步压缩 KV Cache，同时减少计算。 方向很激进，不过 Infra 也更复杂。 ⑤ K3 的 MLA 可以去掉 RoPE，是因为 KDA 已经隐含提供了一种广义的位置信息。 这只适用于 KDA+MLA 的混合结构，像 K2 那种全 MLA 模型，直接去掉 RoPE 还是会明显掉效果。 感觉苏神真正的观点是：大模型架构里很少有免费的升级。K3 的设计哲学不是找一个碾压所有方案的新架构，而是在效果、训练成本、Prefill、Decoding、KV Cache 和稳定性之间，找一个当前更合适的组合。 苏神原文里还有很多技术内容，非常值得一看。 原文地址：

kimi k3 vs gpt 5.6 sol vs fable 5 vs grok 4.5 @Kimi_Moonshot just dropped kimi k3 – a 2.8t param native multimodal model, the first open 3t-class release. key facts: • 1m token context. stable latentmoe activating 16 of 896 experts, built on kimi delta attention (kda) and attention residuals • quantization-aware training from the sft stage onward – mxfp4 weights, mxfp8 activations. moonshot claims ~2.5x scaling efficiency over k2 • max thinking effort by default. low- and high-effort modes are "coming in updates" – there is no way to turn the thinking down today, and you feel it in every run • pricing: $0.30/mtok cache-hit input, $3.00/mtok cache-miss, $15.00/mtok output. claims >90% cache hit rate on coding workloads • benchmarks: swe marathon 42.0 (1st – fable 5: 35.0, sol: 39.0, opus 4.8: 40.0), terminal bench 2.1 88.3, browsecomp 91.2 (1st), program bench 77.8 (1st), gpqa-diamond 93.5. loses frontierswe 81.2 vs fable's 86.6, and deepswe 67.5 vs sol's 73.0 our test – 3 prompts, single-file html, @threejs, fully procedural, no assets: 1. photorealistic european roulette wheel – 37 pockets in the real sequence, mahogany clearcoat bowl, chrome turret, diamond deflectors, flick-to-spin, ball that spirals inward and settles on a mathematically real number 2. las vegas slot machine – 3 reels behind transmissive glass, drag the chrome lever to play, mechanical odometer counters modelled in 3d, coin physics on win 3. full pinball table – 6.5° tilted playfield, flipper impulse physics, spline ramps, drop targets, 6 bumpers, mechanical score reels in the backbox we ran the test on @aimlapi platform results: - cost #1# grok 4.5 – $0.30 #2# kimi k3 – $0.71 #3# gpt 5.6 sol – $2.05 #4# fable 5 – $7.69 - tokens #1# grok 4.5 – 34,241 #2# gpt 5.6 sol – 51,748 #3# fable 5 – 144,126 #4# kimi k3 – 157,999 - lines of code #1# gpt 5.6 sol – 3,054 #2# grok 4.5 – 3,047 #3# kimi k3 – 2,255 #4# fable 5 – 1,950 - generation time #1# grok 4.5 – 5.1 min #2# gpt 5.6 sol – 22.0 min #3# fable 5 – 31.5 min #4# kimi k3 – 75.6 min observations: • kimi k3 is cheap and it is slow. 75.6 minutes across three prompts against grok's 5.1. it is 2.4x grok's price and 15x grok's wall clock. the roulette took 15 min, the slot 18, the pinball 42 • it failed 2 of 3. only the roulette works. the slot machine has reel cutouts on both faces of the cabinet and the symbols face backwards – you can only read your spin by walking around to the rear of the machine. the pinball table stands vertically on its edge with the legs floating detached beside it. • 81% of kimi's output tokens are reasoning, not code. grok: 22%. you are not paying for a bigger answer, you are paying for a longer argument with itself • price per 100 shipped lines – grok $0.010, kimi $0.031, sol $0.067, fable $0.394. a 39x spread for the same three files kimi k3's code quality: upsides: • the roulette is genuinely good – procedural wood grain with real specular breakup, correct european sequence (0-32-15-19-4...), chrome turret, diamond deflectors, clean console • the pinball artwork is the best in the test – a synthwave "nova strike / deep space" field with six individually coloured neon bumper rings, a retro sun on a grid horizon, a nova burst, and a scoring legend printed on the apron. no other model printed the rules on the machine. it is a beautiful texture on a broken object • physics reasoning is real – it derived a 480hz substep for the collider, worked out ball settle conditions and termination guarantees, and checked every ramp exit vector by hand before writing any of it • it is the only model that saw the importmap trap coming. sol shipped a blank white page twice because three.js addons import the bare specifier 'three' and die without an import map downsides: • it dodged that trap on the slot by loading three.js r128 through classic script tags – a 2021 build with no working transmission. its slot glass rendered fully opaque and buried all three reels behind a white pane. the code asks for transmission: 0.93, ior: 1.5 – correct, and silently ignored by a renderer that predates the feature • after 42 minutes and 212k characters of reasoning, the pinball cabinet is not assembled. the table stands vertically on its edge like a wardrobe – the prompt asked for 6.5° from horizontal, it delivered 90°. the legs float detached in the void beside it. head-on it photographs beautifully; orbit ten degrees and it is a painted slab with four chrome rods hovering nearby • the playfield z-fights with the glass – hard black banding across the whole field as soon as you pull the camera back a note on the pinball, in fairness to kimi: nobody passed it. every model shipped broken ball physics and controls you cannot trust. it is the hardest prompt we have run and the whole field failed it, each in its own way kimi k3 reasons better than anything else here and it shows exactly where reasoning pays – physics constants, sequences, edge cases, traps the others walked into follow @thehypedotnews for 24/7 ai news, analysis and breakdowns

晚点LateTask这篇访谈的信息量非常大，关于百度美研、Scaling Law、OpenAI、Anthropic、Cerebras的往事。 > "Dario 能进百度，其实是他职业生涯里很重要的一步。他是 Greg Diamos 招进来的。而且加入百度前，Dario 并不是计算机或 AI 科班出身，而是数学、物理和生物背景，Greg Diamos 发现他很有 AI 直觉和训练模型的能力。" > 十年前（早于transformer），百度已经在训练接近 3 亿参数的语言模型——也就是发现Scaling Law雏形的时候。用 GPU 训练一次要三个多月，这个模型基于一个自研的框架——Paddle（飞桨）。 > Sam Altman 本人是 Cerebras 的投资人。百度是 2017 年投的，Sam Altman 2016 年就投了。 > 百度早期投资 Cerebras，投资决策只用了2天，由李彦宏、陆奇、CFO做出——这个投资决策也证明了百度当时的投资眼光有多超前。 > 百度曾有机会成为 OpenAI、Anthropic 的早期天使。当时OpenAI、Databricks、Scale AI 这些公司都在百度的待投名单上。可惜的是中美关系恶化导致没投成。 > 陆奇早年曾是 Sam Altman 的 mentor。2018 年 5 月陆奇从百度离职后，同年 8 月便接受 Sam Altman 的邀请，出任 YC 中国的创始人兼首席执行官。 > 2020 年夏天，一些在 OpenAI 的百度前员工说 GPT-3 快训练出来了，当年在百度想做的事情，快在 OpenAI 做成了——维基百科水平的语言模型。那时 GPT-3 还在后训练阶段，距离 ChatGPT 出来还有两年多。 > 百度美研顶峰时期至少 250 多人，人才密度很高，甚至在 Google DeepMind 都没有过。很多人是冲着吴恩达来的。 > 后来这里的很多人加入了核心 AI 创业公司，或自己创业，除了前面提到的 OpenAI、Anthropic，也有人参与创办 Adept、xAI，还有一些人成了 Meta FAIR 等实验室的重要成员。

🗺️ El fin de los paneles de telemetría aburridos: El mapa de observabilidad Open Source definitivo Monitorear arquitecturas distribuidas y microservicios usando logs planos o interfaces densas hace que rastrear un error en producción sea una pesadilla. Maple es un monorrepo de código abierto que transforma la telemetría de tu infraestructura en un mapa de servicios interactivo y animado en tiempo real. Pasa de leer métricas frías a ver exactamente cómo fluyen tus datos. El ataque a la yugular: 🔄 Service Map en Tiempo Real: Visualiza las dependencias e interacciones vivas entre tus APIs, bases de datos (MySQL, ClickHouse) y colas de mensajería como Kafka. ⚡ Métricas al Vuelo: Haz clic sobre cualquier nodo para auditar instantáneamente latencias p99, tasas de error y rendimiento sin salir del flujo visual. 🔍 Rastreo de Queries: Identifica cuellos de botella de inmediato visualizando cuáles son las consultas más pesadas ejecutándose directamente en tus almacenes de datos. 🛠️ Arquitectura de Vanguardia: Diseñado como un monorrepo que integra ingesta de OpenTelemetry (OTLP), backend nativo basado en Effect y un servidor de código MCP para que tus agentes de IA puedan auditar el sistema. Deja de adivinar qué microservicio está fallando o rompiendo tu base de datos. Dale visibilidad real al flujo de datos de tu infraestructura de backend. Enlace al repositorio en los comentarios. Guarda este post en marcadores antes de que tu próximo sistema distribuido entre en cuello de botella 🔖