Eagle releases EAGLE 4, its latest speculative decoding method, reporting 961.6 tokens per second on Qwen3.8 27B

Context

EAGLE is a family of speculative decoding methods, where a small draft model proposes several tokens that the main model checks in one pass, speeding up generation without changing the output. The speed figure is Eagle's own measurement on a single setup.

Deep dive

On October 20, Eagle announced EAGLE 4, saying that from EAGLE 1 to EAGLE 3 it kept making inference faster and that EAGLE 4 is its fastest yet.

It reported 961.6 tokens per second on Alibaba's Qwen3.8 27B using TokenSpeed, in BF16 precision on four NVIDIA B300 GPUs with tensor parallelism across four GPUs, one request at a time and temperature zero. It linked a GitHub repository.

Those conditions matter. A single request at temperature zero is the setting where speculative decoding shines most, since the draft model's guesses are easiest to verify. Speeds under heavy load with many users, or with sampling at higher temperatures, are usually lower. Eagle did not give a comparison to plain decoding on the same hardware in the post.

The video shows the names Meta, Nvidia, AMD and Perplexity, without explaining their connection to the project.

Earlier EAGLE versions became widely used in serving systems, which makes a new version notable for anyone running open models. Qwen3.8 27B has also appeared elsewhere in this deck as a base for other projects.

What to expect next

Watch for serving frameworks such as vLLM and SGLang to add EAGLE 4, and for independent benchmarks under realistic multi user loads. A comparison against plain decoding and against EAGLE 3 on the same hardware would show how much of the gain is new. Also watch whether Eagle publishes a paper describing the method. Support for more models beyond Qwen would broaden its reach.