Karl Mehta (@karlmehta) “Mark Zuckerberg explains the 405B teacher-model flywheel that could make one gia” — TopicDigg

Mark Zuckerberg explains the 405B teacher-model flywheel that could make one giant AI the wrong end state "People are gonna wanna do inference directly on the 405 because it's, you know, by our estimates, it's gonna be about 50% cheaper, I think, than GPT-4o to do that directly." "Because it's open weights, the ability to take the model and distill it down to whatever size that you want, to use it for synthetic data generation, to use it as a teacher model." "Our vision is that there should be lots of different models. I think every startup out there, every enterprise, governments, they all kind of wanna have their own custom models." "Right now, as open source basically closes the gap, I think you're just gonna see this wide proliferation of models where people now have the incentive to basically customize and build and train exactly the right size model for what they're doing, train their data into it." "They're gonna have the tools to do it because of a lot of the partner integrations that the companies like Amazon are doing with AWS or Databricks or different folks like that who are building these whole suites of services for distilling and fine-tuning open models." The counterintuitive edge is that the 405B model may be most valuable as raw material, not an endpoint. The open model compresses into the right size, absorbs proprietary data, and turns one frontier release into thousands of company-specific systems. Distribution of intelligence beats centralization. - Mark Zuckerberg (@finkd), CEO of Meta, with @rowancheung