Compute-efficient pretraining and scaling to trillion-parameter models

In one line

Research update on compute-efficient pretraining and scaling to trillion-parameter models.

Opening excerpt

10x More Efficient Pretraining Research update on compute-efficient pretraining and scaling to trillion-parameter models.

Magic Team , on September 8, 2026 Frontier pretraining is said to be a big-lab-only game. We don’t have 100k chips yet, so there’s only one way: algorithmic efficiency. After compounding for … a while …, our pretraining recipe is now >10x more compute-efficient than that of leading open-weight base models.

We match DeepSeek V4 Pro Base using ~50x fewer FLOPs – that’s around half of GPT3’s pretraining compute, or $0.5M on GB200. We continued scaling 10x ($4M) and meaningfully outperformed all publicly available open base models on perplexity evals. By the scaling laws in Figure 1, training a model this capable would cost >$100M under DeepSeek V4 Pro’s recipe (and this is ignoring how much data exists). Of course, we won’t stop scaling there.

We believe pretraining, agentic RL, and long-context are sufficient to build superhuman coding agents and automate AI R&D. We started with long-context . Today’s blog post is about pretraining.

(Excerpted from the original; full article via the source link below.)

This story hit the Hacker News front page today (104 points / 57 comments, via magic.dev). Our Tech Radar aggregates daily signals on AI engineering, backend architecture and DevOps — browse the related services and further reading below, or get in touch with our team.