Compute-efficient pretraining and scaling to trillion-parameter models

一句话概要

Research update on compute-efficient pretraining and scaling to trillion-parameter models.

原文开头节选

10x More Efficient Pretraining Research update on compute-efficient pretraining and scaling to trillion-parameter models.

Magic Team , on September 8, 2026 Frontier pretraining is said to be a big-lab-only game. We don’t have 100k chips yet, so there’s only one way: algorithmic efficiency. After compounding for … a while …, our pretraining recipe is now >10x more compute-efficient than that of leading open-weight base models.

We match DeepSeek V4 Pro Base using ~50x fewer FLOPs – that’s around half of GPT3’s pretraining compute, or $0.5M on GB200. We continued scaling 10x ($4M) and meaningfully outperformed all publicly available open base models on perplexity evals. By the scaling laws in Figure 1, training a model this capable would cost >$100M under DeepSeek V4 Pro’s recipe (and this is ignoring how much data exists). Of course, we won’t stop scaling there.

We believe pretraining, agentic RL, and long-context are sufficient to build superhuman coding agents and automate AI R&D. We started with long-context . Today’s blog post is about pretraining.

(以上为原文节选,完整内容见下方”原文来源”)

这条动态今日登上 Hacker News 首页(104 赞 / 57 评论,来源 magic.dev)。技术雷达每日自动聚合 AI 工程、后端架构、DevOps 方向的前沿动态;相关工程落地可浏览下方的相关服务与延伸阅读,或直接与我们团队交流。