vLLM says it now matches DeepInfer, DeepSeek's own inference engine, when serving DeepSeek V4.1 Flash, and posts a record AgentX result on a GB300 NVL72 rack
vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64. DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now matches DeepInfer's performance curves.
Context
vLLM is the open source engine many companies use to serve models, so matching the model maker's private engine means everyone can get near first party speed. It narrows a cost advantage that labs have when serving their own models. The numbers are vLLM's own measurements.
DeepInfer is the engine DeepSeek uses to serve its own models, and vLLM says it was originally built on vLLM. AgentX is a SemiAnalysis benchmark that measures serving performance on realistic agent workloads. The lead post was published at 1:59 PM Eastern on October 24, and the engineering work was led by the startup Inferact.
Deep dive
The vLLM project posted at 1:59 PM Eastern on October 24 that it now matches DeepInfer, the engine DeepSeek built to serve DeepSeek V4.1 Flash, on DeepSeek's own decode configuration across batch sizes from 8 to 256. vLLM says it did this using 32 GPUs per serving group where DeepInfer used 64, and that the gains came from integrating DeepSeek's open source kernels and heavily optimizing the prefill and decode stages.
The chart attached to the post carries an important caveat. DeepInfer's values were read off DeepSeek's published charts, the hardware and parallel layouts differ, and vLLM says it is not an engine to engine speedup comparison. In other words, vLLM is showing that its curves now line up with what DeepSeek published, not that it ran both engines side by side.
In the same thread, vLLM said it scaled out on a full GB300 NVL72 rack by splitting prefill and decode onto separate GPUs and spreading experts widely across the rack. On SemiAnalysis AgentX it reports 954K tokens per second per GPU of total throughput at 196 tokens per second per user at the 90th percentile, which it calls 7.9 times its single node result and the best DeepSeek V4.1 Flash result on AgentX so far. A later post put the revenue at $38.96 per GPU hour at DeepSeek's off peak prices.
The project credited DeepSeek for building in the open, NVIDIA for collaboration and the startup Inferact for leading the engineering.
What to expect next
Independent reproduction is the key next step, since the blog post includes instructions and SemiAnalysis can publish its own AgentX runs. If the results hold, providers that serve DeepSeek models on vLLM could close the price gap with DeepSeek's own API. Watch whether rival open engines such as SGLang publish matching numbers, and whether the same techniques reach other large mixture of experts models like Kimi K3.1 Pro once its weights ship.