Show HN: DeepSeek V3 in ~200 lines of pure Jax
Trending on Hacker News: Show HN: DeepSeek V3 in ~200 lines of pure Jax (114 points / 7 comments, via github.com)
In one line
DeepSeek V3 (MLA, DeepSeekMoE, aux-loss-free balancing, MTP) in 190 lines of pure JAX, trained on a MacBook Air. Write-up: vaibhaav.in/writing/deepseek-v3-jax - Vaibhaav-Tiwari/deepseek-v3-jax
Opening excerpt
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} main Branches Tags Go to file Code Open more actions menu Latest commit History 4 Commits 4 Commits Folders and files Name Name Last commit message Last commit date post post .gitignore .gitignore LICENSE LICENSE README.md README.md charts.py charts.py deepseek_v3.py deepseek_v3.py plots.py plots.py prepare_data.py prepare_data.py requirements.txt requirements.txt sample.py sample.py smoke_metal.py smoke_metal.py sources.txt sources.txt tests.py tests.py train.py train.py View all files Repository files navigation DeepSeek V3 in pure JAX The architecture from the DeepSeek-V3 technical report in 190 lines of pure JAX , pretrained from scratch on a MacBook Air on public-domain detective fiction.
No Flax, no Equinox, no PyTorch. Params are plain dicts of arrays and every part of the model is a function you can read top to bottom in …
(Excerpted from the original; full article via the source link below.)
This story hit the Hacker News front page today (114 points / 7 comments, via github.com). Our Tech Radar aggregates daily signals on AI engineering, backend architecture and DevOps — browse the related services and further reading below, or get in touch with our team.
Source: Hacker News