Antid (@antisadh) “ONE WEEKEND AND $249 IN HARDWARE TURNED A 3 GIGABYTE AI MODEL INTO AN OFFLINE CH” — TopicDigg

ONE WEEKEND AND $249 IN HARDWARE TURNED A 3 GIGABYTE AI MODEL INTO AN OFFLINE CHATGPT THAT COSTS $5 A MONTH WHILE OPENAI CHARGES $200 the video shows a small lilliput monitor sitting on a sparkfun box with the jetson orin nano dev board wired up underneath, the whole setup costs less than a single month of chatgpt pro the guy spent saturday installing the OS and using docker images to run microsoft's phi-2 model locally, then he built a simple script that lets him chat with it directly on the device he asks the box for a joke and the response comes back instantly with full chat history saved, so he can ask for another one and it remembers the context just like chatgpt does on the cloud the model is only 3 gigabytes and runs at conversational speed on a chip that fits in your palm and pulls 7 to 25 watts, while a regular PC running AI work pulls 300 to 500 openai charges $200 a month for chatgpt pro and rate limits you at the worst moments, this box does 80% of the same job for $5 a month in electricity with no logging, no API key, no cloud server most people are still renting AI from someone else's data center while a few are running it on a $249 board in their bedroom the window is open, follow and bookmark before it closes.