From the creator of Redis; run LLM locally with ds4
Trending on Hacker News: From the creator of Redis; run LLM locally with ds4 (88 points / 16 comments, via dwarfstar.sh)
In one line
Docs, benchmarks and setup notes for ds4, antirez’s C inference engine for DeepSeek V4/V4.1, Qwen3.8 Flash Next and GLM 5.x on Metal, CUDA and ROCm.
Opening excerpt
DwarfStar 4 is a narrow C inference engine for high-memory Mac, CUDA and ROCm machines. It supports DeepSeek V4 and V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next, with text and vision models, local APIs, a CLI and a native agent in one stack.
SUPPORTED: DEEPSEEK V4 / V4.1 + GLM 5.x + QWEN3.8 · MIT LICENSE · C / METAL / CUDA / ROCM · QWEN ON 64GB
DeepSeek V4 Flash is a large mixture-of-experts model. The usual path is remote serving; ds4 starts from the opposite constraint.
Asymmetric quantization targets the routed experts while preserving critical paths. The model becomes practical on high-memory machines.
(Excerpted from the original; full article via the source link below.)
This story hit the Hacker News front page today (88 points / 16 comments, via dwarfstar.sh). Our Tech Radar aggregates daily signals on AI engineering, backend architecture and DevOps — browse the related services and further reading below, or get in touch with our team.
Source: Hacker News