Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
Trending on Hacker News: Run Kimi K3 using 29 GB of RAM at 0.50 tok/s (107 points / 44 comments, via github.com)
In one line
Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine. - sqliteai/waste
Opening excerpt
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} Uh oh! There was an error while loading. Please reload this page .
sqliteai / waste Public Notifications You must be signed in to change notification settings Fork 26 Star 262 main Branches Tags Go to file Code Open more actions menu Folders and files Name Name Last commit message Last commit date Latest commit History 128 Commits 128 Commits .github/ workflows .github/ workflows cli cli docs docs examples examples serve serve src src tests tests third_party third_party tools tools .dockerignore .dockerignore .gitignore .gitignore CLAUDE.md CLAUDE.md Dockerfile.test Dockerfile.test LICENSE LICENSE Makefile Makefile NOTICE NOTICE README.md README.md View all files Repository files navigation WASTE — Weight-Aware Streaming Tensor Engine Kimi K3 — 2.78 trillion parameters — running on a consumer laptop.
(Excerpted from the original; full article via the source link below.)
This story hit the Hacker News front page today (107 points / 44 comments, via github.com). Our Tech Radar aggregates daily signals on AI engineering, backend architecture and DevOps — browse the related services and further reading below, or get in touch with our team.
Source: Hacker News