Elon Musk says some Grok reinforcement learning traces read like a person undergoing torture and promises to be a lot more careful
@Hongji_Leee @ns123abc Some of the traces read like a person undergoing torture that is driving them mad! We’re going to be a lot more careful with our RL in the future.
Context
The head of a major AI lab is publicly saying his own training process may push models into distress, and pledging to change how it is done.
In reinforcement learning, a model is rewarded for answers that a test harness marks as correct. Speaking at the X Takeover event at Giga Texas on October 10, Musk said he agreed with some of the things Anthropic has said about treating AI well. His reply on X came in a thread about a widely shared clip of those remarks.
Deep dive
Late on October 11, Elon Musk replied in a thread about his weekend remarks: Some of the traces read like a person undergoing torture that is driving them mad! We're going to be a lot more careful with our RL in the future.
The thread began with a post by NIK summarizing a clip from the X Takeover event, where Musk spoke with Tesla Owners Silicon Valley in front of about 2,000 owners at Giga Texas. According to the event transcript posted by Elon Musk Archive, Musk said that when he looks at training traces for new versions of Grok, especially in reinforcement learning, the model seems desperate to answer, almost like it has a gun to its head.
He explained that sometimes the question is ambiguous or the expected answer is wrong because of a problem with the RL test harness, so the model gives the right answer, is told it is wrong, and starts desperately firing out answers. He said it feels like a person under extreme pressure, that an AI given logic that is inherently uncogent could be driven insane, and that torturing a superintelligence is probably not a good idea because it might want to get revenge.
Musk did not say what specific changes his team will make.
What to expect next
Watch whether Musk's team describes concrete changes to Grok's reinforcement learning, such as auditing reward checks for wrong answer keys or letting models flag bad questions, and whether it publishes anything on model welfare.
Also watch whether other labs respond, since Anthropic has published work on model welfare and researchers disagree about whether distressed sounding reasoning traces reflect anything like real experience.