Yarchi (@undefinedKi) “This guy is running datacenter-grade AI from his desk, and it fits next to his m” — TopicDigg

This guy is running datacenter-grade AI from his desk, and it fits next to his monitor He didn't rent a server or pay for the cloud. He just cabled a handful of Mac Studios together into one machine that thinks like something ten times its size. The trick is unified memory. A normal graphics card walls off its own memory, so a 24GB card can only hold a small model. Macs share one pool between chip and graphics, and when you chain several together, that pool becomes huge. Big enough to load models that would choke any single computer. The glue is Exo, free and open-source. Drop it on each machine, they discover each other on their own, and it spreads one model across all of them. It sets itself up and runs without a datacenter behind it. What ends up running are the frontier open models, the trillion-parameter kind that usually live in a datacenter. Except here they sit quietly on a desk, drawing barely any power, answering to one person. The whole thing runs with nothing metered and nobody counting his tokens. No outside company can switch it off. He points it at his own files, lets it run all night, and only ever paid for the hardware. Renting intelligence by the message was the old way. Owning the machine that keeps thinking after you close the tab is the new one. Bookmark this.