The apocalypse will be a headache for several possible reasons. Giant robot dinosaurs! The constant 50’s music blaring from eerily functional radios! The possibility of ending up like cattle in a terrifying cannibal basement and wondering why you didn’t read The path closer. But clearly, the absolute worst part of this will be not having ChatGPT to tell you what to do. How can you live without a strange Internet parrot whispering sweet nothings in your ear all day?
Well, fear not, because one brave inventor has already foreseen the disaster that a post-apocalypse without LLM would present, and has planned for exactly that eventuality. Behold: CrankGPT!
From the name, you might think this is just ChatGPT with a built-in set of anti-vaxx messages, but no! This is an LLM for an off-grid future: a self-contained, battery-less box that uses an old-style hand crank for power. The box contains a Raspberry Pi 5 with 8GB of RAM, an audio input/output card, and a 20W handheld generator.
CrankGPT is the work of a two-person company. Squeez Laboratorieswhose website describes its mission as “making AI smaller, cheaper and faster so it can run anywhere.” Oh happy day! Depending on the device websiteThe generator connects to a capacitor board the duo designed; the board ensures that the voltage supplied to the Raspberry Pi remains stable. An interesting aspect is that “you can feel (the energy load) through the crank: when LLM inference and speech synthesis work together, the crank becomes much more difficult to turn.”
The LLM itself can be one of several alternatives: SqueezLabs recommends “small LFM2 Liquid AI variants (e.g. 350M or 1.2B), along with gem 3 in its 1B form.” The machine receives voice input through its audio card, converts the voice to text through a custom voice agent written by the designers, and then generates the LLM response in text-to-speech software. Flutist.
In all seriousness, this is an interesting proof of concept for running demanding software on a relatively small power budget, and also a demonstration that LLMs don’t have to suck up large amounts of energy to function. As the designers explain, “it offended our European sensibilities toward small, practical automobiles to see people around us throwing kilowatts and thousands of chips at tasks that small models could perform as well as large ones, for a fraction of the cost and energy.”
Unfortunately, with Silicon Valley at the wheel, it doesn’t look like the supersized philosophy that characterizes LLM development right now is going away anytime soon. But when the bombs drop, it will be the small, practical car guys who have the last laugh. Well, at least until the team with necks wider than their heads comes along with their AR-15s. ChatGPT, what’s for dinner? …Me, you say? Oh.
