Show HN: JevBench, a reproducible benchmark for typed decision models

In one line

52 Jev-class systems tested on 534 decisions. Jev 1.13.0 leads JevBench v1.3.0 with 74.4; compare open-source, self-hostable and hosted options.

Opening excerpt

Hosting = where inference runs; company = where the provider or lab is registered.

EU hosting means the route’s inference runs inside the EU: an EU region, AWS Bedrock’s EU cross-region (geo) profiles, Azure’s Europe Data Zone, or a provider whose entire public fleet is documented as EU-hosted, each checked per model against the provider’s documentation. Global deployments do not count, and neither does an EU billing region, an EU company or an EU control plane on its own. One disclosed company-policy exception stays in, marked “EU equivalent”.

Evidence requirements (benchmark evidence, priced provider, measured task tokens) sit above the table they apply to.

All tasks is the published default. Choose a scope to see how the ranking changes by difficulty. Hard only uses all 220 hard-tier decisions and their measured Intelligence, Calibration, Speed and Cost.

(Excerpted from the original; full article via the source link below.)

This story hit the Hacker News front page today (36 points / 4 comments, via benchmarkheaven.com). Our Tech Radar aggregates daily signals on AI engineering, backend architecture and DevOps — browse the related services and further reading below, or get in touch with our team.