Dream Labs comes out of stealth with Dream 1, a robot foundation model it says controls two armed robots it has never seen, through a public API
Today, Dream Labs is coming out of stealth with Dream-1, a robot foundation model that controls bimanual robots it has never seen, on tasks it was never shown, in places it has never been, from $3/hour. Dream API is live for everyone, today. 馃У
Context
Robot models usually need demonstrations from each new robot, so a model that works on unseen robots out of the box would make general purpose robotics much cheaper to try.
The generalization and price claims are self reported by Dream Labs, which calls Dream 1 a research beta not validated for safety critical use. Dream Labs says it raised $90 million in seed funding, and Forbes reported the team are former Nvidia researchers.
Deep dive
On October 19, Dream Labs posted that it is coming out of stealth with Dream 1, a robot foundation model that controls bimanual robots it has never seen, on tasks it was never shown, in places it has never been, from $3 an hour. It said its Dream API is live for everyone today.
The company's research post calls Dream 1 a World Action Model, which learns to predict video and actions together. Its argument is that through the head camera every robot is just shapes moving through a scene, so if the model can predict what a robot will look like while completing a task, it can produce the actions to get there. Besides joint angles, it can output where each gripper should be and whether it is open, a format shared by any two armed robot with parallel grippers, and each robot's own inverse kinematics solver turns those poses into joint commands.
Dream Labs says no data from the specific robots was used for its zero shot tests, that it even controlled the original PR2 research robot released 18 years ago, and that performance varies by robot and by the quality of its solver. It says developers can send camera images, the robot's state and a task to the API and get actions back, and can fine tune the model on their own robot with demonstrations.
What to expect next
Watch for independent users reporting how well Dream 1 works on their own robots, since the zero shot results come from Dream Labs' own tests and the company says performance varies. The limit to two armed robots with parallel grippers is a meaningful constraint, so watch whether it extends to hands or other bodies. Also watch how its price compares with robot models from Google, Nvidia and Physical Intelligence.