I'm making LLMs work on small and old devices
Trending on Hacker News: I'm making LLMs work on small and old devices (229 points / 80 comments, via github.com)
In one line
maxxing the ability to run different LLMs on the smallest devices possible - hyperionaidev/smolmaxxing
Opening excerpt
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} main Branches Tags Go to file Code Open more actions menu Latest commit History 38 Commits 38 Commits Folders and files Name Name Last commit message Last commit date assets assets bench bench data data docs docs ipad-air ipad-air patches patches screenshots screenshots tiny-llm-web tiny-llm-web tools tools tv-llm-ui tv-llm-ui zigcc zigcc .gitignore .gitignore EXPERIMENT.md EXPERIMENT.md LICENSE LICENSE README.md README.md View all files Repository files navigation 🤏 smolmaxxing
TV boxes, old tablets, single-board computers and routers all have a CPU, some RAM and flash storage that sit idle most of the day. smolmaxxing finds the biggest model each of them can really run, then pushes it further with custom kernels, careful memory use and weights streamed from flash. Every number in this repo was measured on the device itself.
(Excerpted from the original; full article via the source link below.)
This story hit the Hacker News front page today (229 points / 80 comments, via github.com). Our Tech Radar aggregates daily signals on AI engineering, backend architecture and DevOps — browse the related services and further reading below, or get in touch with our team.
Source: Hacker News