ScaleDown launches Constrained Decoding Models, task specific AI models for classifying, extracting and summarizing that skip token by token generation
You already gave the LLM the answer. So why is it writing it out from scratch? Introducing a new AI model architecture: Constrained Decoding Models (CDMs) LLMs were built to write. Most production workloads just need an answer: fast, consistent, and affordable. CDMs are purpose-built for 1 task each. They read your input once and skip the token-by-token generative loop that makes LLMs slow and expensive. Not just a decision model, but a family of 3 CDMs live today: • Classify: define your labels and get back a calibrated probability + reason for each. Used for routing, tagging, and intent detection. • Extract: describe the entities in natural language and get back well-formed JSON with confidence scores + citations. Used for web scraping, document processing, and PII redaction. • Summarize: pass in long text, images, or PDFs and get back a consistent summary under 1,000 tokens. Used for context compaction, metadata generation, and transcript digests. Built for production: • 1M tokens of multimodal (text + image) input, 31x larger than Jev • 77.2% on DecisionBench vs Jev's 72% • 200x faster, 467x cheaper than frontier LLM APIs • $0.05/M input tokens. Output is free (yes, even for summarize) Live on your favorite platforms today: @OpenRouter @vercel_dev @LiteLLM @LangChain @TrustedRouter @itsArthurAI @useRouterPlus Get 50M input tokens for free (for the first 50 hours) at link below 👀
Context
Many production AI calls only need a label, a set of fields or a short summary, so models built for just those jobs could cut costs and latency compared with general LLMs.
Neal Patel is cofounder and CEO of ScaleDown, according to his X profile. OpenRouter lists ScaleDown's Extract and Summarize models as released October 13, and LangChain announced an integration. All speed, cost and accuracy figures are ScaleDown's own.
Deep dive
On October 13, Neal Patel posted that ScaleDown is introducing a new AI model architecture called Constrained Decoding Models, or CDMs, arguing that LLMs were built to write while most production workloads just need an answer that is fast, consistent and affordable.
According to the post, each CDM is built for one task, reads the input once and skips the token by token generative loop. Three are live: Classify returns a calibrated probability and reason for each label, Extract returns JSON with confidence scores and citations for entities described in natural language, and Summarize returns a summary under 1,000 tokens from long text, images or PDFs.
The post claims 1M tokens of text and image input, 77.2% on DecisionBench against 72% for a model it calls Jev, and speeds 200 times faster and costs 467 times cheaper than frontier LLM APIs. It lists pricing of $0.05 per million input tokens with free output, and availability on OpenRouter, Vercel, LiteLLM and LangChain among others.
LangChain described CDMs as trained for one task and only returning answers that task allows, and said its integration uses them to summarize history, compress tool results and extract data for agents. OpenRouter listed Summarize at $0.04 per million input tokens and $0 output, slightly below the post's price.
What to expect next
Watch for independent tests of the 200 times speed and 467 times cost claims, and for details on how CDMs are built, since the post calls them a new architecture but gives no paper.
Also watch adoption on OpenRouter and LangChain, whether pricing settles between the $0.05 in the post and the $0.04 OpenRouter lists, and whether larger labs launch similar task specific models for routing and extraction.