Elon Musk says Digital Optimus can play about halfway through Diablo just by watching the screen, with League of Legends in training
We’re making interesting progress here. Digital Optimus can play about halfway through the Diablo campaign so far just by looking at the screen like a human. Skill at Counter-Strike and other fast games is good too. League is also in training. Goal is to generalize across all games.
Context
Musk is presenting video games as a test bed for Digital Optimus, the Tesla and xAI agent meant to use computers by reading the screen and working a keyboard and mouse, and says the goal is for it to generalize across all games.
Digital Optimus, which Musk also calls Macrohard, is a joint project of Tesla and xAI that Musk announced on March 11, according to Reuters. Reuters reported that Musk described it as pairing xAI's Grok model, acting as a high level navigator, with a Tesla built agent that processes real time screen video and keyboard and mouse actions. Musk said at the time that it could in principle emulate the function of entire software companies.
Deep dive
Musk posted that the team is making interesting progress, and that Digital Optimus can play about halfway through the Diablo campaign so far just by looking at the screen like a human. He said its skill at Counter Strike and other fast games is good too, that League is also in training, and that the goal is to generalize across all games. He did not share video, benchmarks or details on how much help the agent gets.
Musk was replying to a post by the account teslayoda, which said Tesla is hiring a machine learning engineer for real time games on the Digital Optimus team in Palo Alto. According to that post, the role involves building agents that play big budget games, browse the web and use professional apps at superhuman levels, and Tesla plans to benchmark how skills transfer across games, general computer use, FSD and Optimus.
When Musk announced the project in March, Electrek reported, he compared its design to Daniel Kahneman's two systems of thinking, with the Tesla agent handling fast, instinctive reactions and Grok handling slower reasoning. CNBC reported that Musk said the system would run on Tesla's AI4 chip paired with xAI's Nvidia based servers. Games fit that plan because they demand fast reactions to what is on screen.
What to expect next
Watch for any video or independent demo of Digital Optimus playing Diablo, Counter Strike or League, since Musk's progress claims have not been shown publicly.
Also watch whether Tesla or xAI moves from games to everyday computer work, which was the original pitch for Macrohard. Musk has said the goal is to generalize across all games, so results on titles the agent was not trained on would be the clearest test of that claim.