Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
Hacker News 热议:Run Kimi K3 using 29 GB of RAM at 0.50 tok/s(107 赞 / 44 评论,来源 github.com)
一句话概要
Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine. - sqliteai/waste
原文开头节选
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} Uh oh! There was an error while loading. Please reload this page .
sqliteai / waste Public Notifications You must be signed in to change notification settings Fork 26 Star 262 main Branches Tags Go to file Code Open more actions menu Folders and files Name Name Last commit message Last commit date Latest commit History 128 Commits 128 Commits .github/ workflows .github/ workflows cli cli docs docs examples examples serve serve src src tests tests third_party third_party tools tools .dockerignore .dockerignore .gitignore .gitignore CLAUDE.md CLAUDE.md Dockerfile.test Dockerfile.test LICENSE LICENSE Makefile Makefile NOTICE NOTICE README.md README.md View all files Repository files navigation WASTE — Weight-Aware Streaming Tensor Engine Kimi K3 — 2.78 trillion parameters — running on a consumer laptop.
(以上为原文节选,完整内容见下方”原文来源”)
这条动态今日登上 Hacker News 首页(107 赞 / 44 评论,来源 github.com)。技术雷达每日自动聚合 AI 工程、后端架构、DevOps 方向的前沿动态;相关工程落地可浏览下方的相关服务与延伸阅读,或直接与我们团队交流。
原文来源: Hacker News