TechCrunch: Microsoft researchers say they’ve developed a hyper-efficient AI model that can run on CPUs. “Microsoft researchers claim they’ve developed the largest-scale 1-bit AI model, also known as a ‘bitnet,’ to date. Called BitNet b1.58 2B4T, it’s openly available under an MIT license and can run on CPUs, including Apple’s M2.”
#SmallAI
Okay. Now I've seen literally everything. An LLM and an inference engine, embedded in a font. https://fuglede.github.io/llama.ttf/ #SmallAI https://www.youtube.com/watch?v=Q4bOyYctgFI
By me for @hackster_io, "Bringing GPT-4o in From the Cloud to the Edge." #TinyML #SmallAI https://www.hackster.io/news/bringing-gpt-4o-in-from-the-cloud-to-the-edge-289fbf6e0978
By me for @hackster_io, "Creating cross-platform Small AI with PicoLLM." https://www.hackster.io/news/creating-cross-platform-small-ai-with-picollm-3cf6076db9ec #tinyML #smallAI
It’s sort of interesting to see the financial markets paying attention to #SmallAI. That means that anything that affects “big” AI is important enough to move the markets. Makes sense, if it’s going to affect the $NVDA stock price, it’ll affect the market. https://finimize.com/content/small-talk
By me for @hackster_io, “The beginnings of Small AI, https://www.hackster.io/news/the-beginnings-of-small-ai-64ef3768b7b5.” #TinyML #SmallAI
#Firefox 118 has local, in-browser machine translations. That’s pretty freaking cool!
> Automated translation of web content is now available to Firefox users! Unlike cloud-based alternatives, translation is done locally in Firefox, so that the text being translated does not leave your machine.
On My Disk is a personal cloud storage solution that keeps your files secure and private. We are integrating PeARS search into the service to make your files incredibly easy to find, regardless of their physical location.
One ideal shared by PeARS and On My Disk is energy efficiency. So we are dedicated to create 'small AI' solutions that combine old and new machine learning techniques, to create the most climate-friendly NLP tools 🍃 🍐 #smallai #NLP
One of the arguments in favour of surveillance capitalism is the great usefulness of cloud-based ML predictions.
After all, who can deny the usefulness of photo apps that automatically recognize faces, detect your speech or help you making sense of the deluge of information in a social feed?
The argument usually goes like this: these features require large neural networks, which in turn require a lot of computational power to train the models, and a lot of memory and disk storage in order to load and save those models.
You can't do such things on small devices that run on batteries. Therefore your phone *HAS* to send your data to #BigTech servers if you want to get those features. Otherwise, you just won't get those features.
Except that... What if this whole argument is bollocks?
#POET (Private Optimal Energy Training) proves that you can run both the training and the predictions locally, without compromising neither on precision, nor on performance.
After all, the really expensive part of training is back-propagation. POET breaks down the back-propagation performance issue by quantizing the layers (so real-number large tensor multiplications get reduced to smaller multiplications of integer tensors, without sacrificing precision too much), and a clever way of caching the layers that are most likely to be needed, so we don't have to recalculate them, without caching everything though (which would be prohibitive in terms of storage).
The arguments in the paper sound very convincing to me. The code is publicly available on Github. I haven't yet had time to test it myself, but I will quite soon - and try to finally build an alternative voice assistant that can completely run on my phone.
Client Info
Server: https://mastodon.social
Version: 2025.04
Repository: https://github.com/cyevgeniy/lmst