Jun Song Says an Anonymous Black Hat Group Offered Him Huge Money to Train Open Weight Models on Exploit Data, an Unverified Claim
Yesterday, an anonymous black hat group reached out with an insane RL request. They offered an astronomical amount of money to post-train open-weight models on exploit data. The amount of attack data they claim to hold is unlike anything I have ever seen. If someone told me this was an undercover FBI dataset, I would have believed it. For context, a white-hat project I worked on using GLM-5.3 used less than one percent of that volume, and it already outperformed frontier models by a huge margin. Of course, I turned it down. But what genuinely scares me is knowing someone else will eventually take the deal.
Context
If true, it shows criminal groups trying to buy custom open weight models trained for hacking, a risk security researchers have long warned about.
Jun Song describes himself on X as building @0xSupergemma, running a data center hardware export business in Korea and working on open source AI. His post, published on Saturday night US Eastern time, drew more than 230,000 views. No other source has confirmed the offer.
Deep dive
Jun Song wrote that an anonymous black hat group, meaning hackers who work for criminal ends, contacted him with what he called an insane reinforcement learning request. He says they offered an astronomical sum to post train open weight models on exploit data.
He says the volume of attack data the group claims to hold is unlike anything he has seen, and that he would have believed it if someone told him it was an undercover FBI dataset. For comparison, he says a white hat project he worked on using GLM 5.3 used less than one percent of that volume and already outperformed frontier models by a huge margin.
Song says he turned the offer down, and that what scares him is knowing someone else will eventually accept it.
The claim is unverified. The post names no group, gives no amount and shows no evidence of the dataset or of the earlier project's results. It still drew wide attention because it fits a known concern: open weight models can be fine tuned by anyone once released, and safety training can be removed. Labs and governments have warned that models trained on exploit data could help attackers find and use software flaws faster.
What to expect next
Watch whether Song shares any evidence, such as details of the white hat project or of the offer, and whether security researchers report similar approaches. Expect the post to feed the debate over open weight releases and over controls on fine tuning for offensive security. Labs that publish open models may face more pressure to explain how they would detect misuse.