Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA

In one line

Benchmarking over 1,000 agentic tasks demonstrates that routing between the open-source Kimi K3 model and the closed Fable 5 model creates a new state-of-the-art (SoTA) approach that surpasses the performance of either model alone.

Opening excerpt

K3 is a frontier quality open model at a fraction of the cost. Even bigger is that it complements Fable predictably, which makes it possible to get the highest quality intelligence by routing tasks.

🧭 tl;dr: We ran Kimi K3 (open) against Fable 5 (closed) on ~1,000 agentic tasks finding:

We averaged benchmarks, each aimed at a different kind of work , and ran K3 and Fable 5 through the same harness. About 1,030 tasks in all, in real agent loops.

One quick definition before we get into the results. Oracle routing is a method for measuring the best theoretical performance by running the task through each model and then picking the cheapest correct option (the cost/performance ceiling). In a practical router, you don’t get to run your task against multiple models. The router makes a prediction of which model has the best cost and quality trade off, but ultimately it’s a guess.

(Excerpted from the original; full article via the source link below.)