MTS (@MTSlive) “DeepMind's Andrew Trask on why the scaling laws are pushing AI from one big mode” — TopicDigg

DeepMind's Andrew Trask on why the scaling laws are pushing AI from one big model toward a protocol: "The zoomed-out picture is that in the end, AI is gonna be a protocol instead of a program. We're seeing that evolution start to really gather steam as the scaling laws constrain how much data, compute, and talent one company can bring together." "When you combine models from multiple different providers, you're implicitly combining the data, compute, and talent that they trained on. So you can get better, faster models for a lower price, which is pretty crazy when you think about it." "If you want the absolute most accurate model, ensembling the top models is always going to win, you'll get higher scores than any single model. And if you want the best accuracy relative to any unit of price, ensembling some open and closed models is likely gonna own that Pareto frontier quietly for a while." "It won't be until there's a leaderboard that widely recognizes ensembles as comparable to individual models that we start to really see it. Then in 12 to 18 months, that saturates a bunch of benchmarks across the space, and the harnesses pick it up, routing you to the best combinations of models on the fly. That's when the market really starts to change in terms of how people buy intelligence." @iamtrask @openminedorg