Nathan Lambert tells a16z that Chinese labs have the talent but lack compute, as he launches Trillium Labs, a nonprofit open AI lab
Nathan Lambert, former post-training co-lead at the Allen Institute for AI, now launching Trillium, a nonprofit open AI lab, joins a16z's Lisha Li to discuss whether open models can really keep up with the frontier. When he tells researchers in China that an average OpenAI researcher reportedly has five thousand H100s to work with, "their jaw just falls to the floor." In his view, Chinese labs have the talent but are mostly short on compute. He also thinks the distillation story is overblown. Based on the research, he puts its edge at a few months, and says "there's a bit of cope" in thinking only American labs can build good models. A lot of frontier progress outside math and code comes down to labs buying 1) data and 2) RL environments. It's a multibillion-dollar market, and some of it is sloppy enough to teach models to reward hack. That's a big part of why Trillium exists. Nathan wants to give away the environments other people would sell, and show exactly how his team uses AI to build AI, including where the models still get stuck. His bet is that the more people can see how this works, the better prepared everyone is for what's coming. 0:55 Can open models keep up with the frontier? 2:05 What it takes to reach the frontier 3:55 Open weights and the fast-follower playbook 8:25 Chinese labs: equal talent, less compute 9:55 Why distillation is harder than it looks 14:55 Is distillation theft or competition? 16:20 The lab flywheel vs owning your weights 21:45 Can general models beat every specialist? 26:30 AI at 1% of today's cost in 4 years 27:15 The RL env boom and reward hacking 31:30 What Trillium will build in the open 35:30 Breaking into AI in the age of AI slop 38:05 Open-sourcing how AI builds AI 44:25 Why Nathan isn't pre-training 49:45 Agent swarms and why lab secrets leak 54:25 Fast takeoff vs risks already here YouTube: t.co/001eb2u6Rh @natolambert @trillium_labs @lishali88
Context
Lambert is one of the best known voices on open models, and his new lab aims to publish the post training recipes and environments that frontier labs keep private.
This is a first hand interview posted by a16z, the venture firm, with Nathan Lambert, formerly post training co lead at the Allen Institute for AI. His claims, including the reported figure of five thousand H100s per OpenAI researcher, are his own views. Trillium Labs announced its launch the same day.
Deep dive
On October 14, a16z posted an interview in which Nathan Lambert joins a16z's Lisha Li to discuss whether open models can keep up with the frontier.
According to the post, Lambert says that when he tells researchers in China that an average OpenAI researcher reportedly has five thousand H100s, their jaw just falls to the floor, and that Chinese labs have the talent but are mostly short on compute. He calls the distillation story overblown, putting its edge at a few months, and says there's a bit of cope in thinking only American labs can build good models. He adds that much frontier progress outside math and code comes from labs buying data and RL environments, a multibillion dollar market that is sometimes sloppy enough to teach models to reward hack.
Trillium Labs, cofounded by Lambert and Tom Zick, says it will build fully open post training recipes, including data, code, evaluations and checkpoints, and expand into research on recursive self improvement, reward hacking and multi agent systems. It has support from Halcyon Futures and Schmidt Sciences. WIRED reported the founders aim to raise $40 million to $100 million and plan to spend $30 million on training over 18 months.
What to expect next
Watch for Trillium's first open post training recipes and environments, and whether other researchers use them to reproduce or challenge frontier lab results.
Also watch whether the lab reaches its fundraising goal and secures the compute it says it is still searching for, and whether its work on reward hacking and recursive self improvement informs safety debates. Its first releases will show how open it really is.