注册并分享邀请链接,可获得视频播放与邀请奖励。

Göktuğ (@GuvercinGoktug) “Thinking Machines - Inkling Thinking Machines Lab, the company founded by Mira M” — TopicDigg

Göktuğ 的个人资料封面
Göktuğ 的头像
Göktuğ
@GuvercinGoktug
ML Scientist | MSc @TU_Muenchen & @imperialcollege 👨🏼‍💻| Computer Vision, Language Modeling
加入 November 2021
495 正在关注    1.4K 粉丝
Thinking Machines - Inkling Thinking Machines Lab, the company founded by Mira Murati, has released Inkling, its first open-weights model under the Apache 2.0 license. Inkling is a mixture-of-experts transformer with 975 billion parameters in total, yet only 41 billion of them are active for any given token. Every layer contains 256 routed experts and 2 shared experts, and a router selects just the 6 most relevant experts per token. This means that only about 4 percent of the model performs computation during inference. The backbone is a 66-layer decoder-only transformer that combines local and global attention layers and supports a context window of one million tokens, which is roughly enough to fit eight novels or an entire codebase into a single prompt. The model was pretrained on 45 trillion tokens spanning text, images, audio, and video. Instead of relying on separate vision or audio encoders, it converts images into patches and audio into discrete tokens, then projects everything into one shared hidden space. All modalities are therefore fused from the very first layer. Inkling accepts text, images, and audio as input, while its output remains text only. On public benchmarks, it performs at the level of GPT 5.6 Sol and Claude Fable 5 in reasoning and agentic coding. The weights are available on Hugging Face, and the model can be fine-tuned through the Tinker API. #MiraMurati# #thinkingmachines# #inkling# #OpenSource#
显示更多