Thinking Machines Details How It Trained Inkling 2 Small With a Native 4 Bit Forward Pass
Training with fewer bits is faster, but quantization error can interfere with learning. Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout.
Context
Training a model in native 4 bit from the first pretraining step through reinforcement learning could cut the cost of building future models, if the recipe holds up.
Thinking Machines posted on November 13 that it has published a Connectionism blog post on how it trained Inkling 2 Small with a native 4 bit forward pass from the first step of pretraining through the final reinforcement learning rollout. The company released Inkling 2 Small weights on November 12, which the November 12 deck already covers. The new post is a technical account of the numerics recipe, written by Jack Cook with others at the lab.
Deep dive
Thinking Machines posted on November 13 that training with fewer bits is faster, but that quantization error can interfere with learning, and that it has now detailed how it trained Inkling 2 Small with a native 4 bit forward pass from the first step of pretraining through the final reinforcement learning rollout. The linked Connectionism post, by Jack Cook and others at the lab, is dated November 13.
The blog says Inkling 2 Small is the largest model the authors are aware of that was trained end to end with a native 4 bit forward pass, and that the same numerics were used from the start of pretraining through the final RL rollout. In their largest jobs, they write, the recipe provided an 8% wall clock speedup in pretraining and a 20% speedup in reinforcement learning over full MXFP8 training. They also say a training run of a similar size model took several weeks months ago, and that Inkling 2 Small was pretrained in 68 hours on GB300 NVL72 racks.
According to the post, forward passes in expert layers used NVFP4, while backward passes used MXFP8 with requantization and stochastic rounding to reduce gradient bias. Those results and comparisons are the lab's own.
What to expect next
Watch for other labs to try a similar native 4 bit recipe, and for Thinking Machines to share any follow up on using 4 bit in the backward pass, which the post says the team is exploring. Independent reproductions, or results on larger models than Inkling 2 Small, would show whether the 8% and 20% speedups hold more broadly. The earlier weight release on November 12 already lets others inspect the trained model.