Kiyoro (@0xKiyoro) “$18,796 of hardware replaces a cloud bill that runs $1,900 a month, and pays its” — TopicDigg

$18,796 of hardware replaces a cloud bill that runs $1,900 a month, and pays itself off before the third invoice the box is a DGX Spark, $2,999 each. 128GB unified memory, a petaflop of compute, 70B models at full precision, where a 4090 caps out at 24GB and sends you back to renting i've been reading into local setups and the scaling is the part people miss > two boxes over the 200Gbps ConnectX-7 link, 256GB, up to 405B parameters > four boxes, near-linear, around 700B after payback it's about $22,000 a year staying in the business instead of routing into someone else's data center. fine-tunes that ran $400 a piece cost nothing, and client data that can't touch a public model never leaves the room cloud pricing teaches you to ration ai. ownership deletes the hesitation the edge wasn't a faster model, it was noticing the rent