Unverified: LeahW Says Karpo Decision 31B Ties for First on a Decision Benchmark
Picking the next tool shouldn’t take a paragraph. We built Karpo Decision. A family of six open-weight models, from 0.8B to 31B.Give it the context, a question, and the options. It returns a probability for each option in one forward pass. Karpo Decision 31B 🏆 73.9 on Decision Index, tied for #1 on the Nov 12 board ⚡ 94 ms median per request on a single RTX PRO 6000 📊 87.2 on JevBench Capability vs. 78.5 for Google’s Gemini Decision 1 🤗 Hugging Face t.co/HpEmbxOSzD 📝 Technical write-up t.co/Rt3WeXfuBD Here’s where I’d try it first: Search. Recommendations. Tool choice. Model routing. Browser actions. Point it at the slowest decision in your stack and tell us what breaks.
Context
A family of open weight models that return option probabilities in one pass could be useful for routing and tool choice, but the board scores in this post are unverified claims from the poster.
LeahW posted that her team built Karpo Decision, a family of six open weight models from 0.8B to 31B, which take context, a question, and options, and return a probability for each option in one forward pass. The post said Karpo Decision 31B scored 73.9 on Decision Index, tied for first on the Nov 12 board, ran at a 94 millisecond median per request on a single RTX PRO 6000, and scored 87.2 on JevBench Capability versus 78.5 for Google's Gemini Decision 1. Those benchmark and latency figures are unverified claims from the poster. The post time is Nov 16 in Eastern time, and no matching story was already on that day's deck.
Deep dive
LeahW posted that her team built Karpo Decision, a family of six open weight models, from 0.8B to 31B, that take context, a question, and a set of options, and return a probability for each option in one forward pass. For Karpo Decision 31B, the post claimed a 73.9 score on Decision Index, tied for first on the Nov 12 board, a 94 millisecond median per request on a single RTX PRO 6000, and an 87.2 score on JevBench Capability, versus 78.5 for Google's Gemini Decision 1. It linked a Hugging Face page and a technical writeup, and suggested uses in search, recommendations, tool choice, model routing, and browser actions. A video is attached.
Those board scores, the latency figure, and the comparison with Gemini Decision 1 are unverified claims in this file. The post does not show a third party leaderboard snapshot, does not define Decision Index or JevBench in the text, and does not establish that 'Gemini Decision 1' is an official Google product name. This story therefore labels the result unverified, as requested, and does not treat the tie for first as a confirmed ranking.
A small open weight model that emits option probabilities in one pass would be a useful tool for routing if the scores hold. That remains a hypothesis until the boards can be checked independently.
This file confirms only that LeahW announced the Karpo Decision family and stated those figures. It does not confirm the benchmark results, the hardware timing, or the Google comparison. A full check of the Nov 16 deck found no earlier story on this model.
What to expect next
The Hugging Face page and the writeup should be checked for weights, a license, and a methods section that defines Decision Index and JevBench. Watch whether an independent board reproduces the 73.9 and 87.2 figures, and whether Google recognizes a model called Gemini Decision 1. If a first party Google post or a known leaderboard later confirms or contradicts the comparison, this story should be updated. Until then, the benchmark claims stay unverified.
