Researchers react to OpenAI's release of hundreds of AI-written math proofs
The Navier-Stokes moment was always much more about the figure below than about the Navier-Stokes problem itself. The run represents a decade of mathematical progress in a week. Can't wait to point these tools at life sciences, building our next models, and alignment!
Context
If the results hold up, AI models are now producing research-level mathematics far faster than human teams.
On October 6, OpenAI published a large set of mathematical manuscripts, reported as 722 results, produced by an unreleased internal model. Through Wednesday, mathematicians and AI researchers debated what the proofs mean. Gary Marcus criticized how little OpenAI disclosed about its methods.
Deep dive
On October 6, OpenAI published 722 mathematical manuscripts produced by an internal model it has not released, grouped into 372 families of results and posted in a public GitHub repository. According to The Next Web, the model was given about 4,000 problems, and each result used about three hours of ChatGPT Pro thinking compute on average. OpenAI says it is the same model behind its Navier-Stokes result last month.
OpenAI's Mark Chen argued that the run matters more than any single problem, calling it a decade of mathematical progress in a week. Noam Brown and Ethan Mollick also described models passing top experts on some research problems.
Mathematicians quoted in news coverage were more cautious. Kevin Buzzard said that of 30 papers relevant to number theory, about seven looked impressive and only one was formally verified in Lean. NYU's Tristan Buckmaster doubted that OpenAI had checked so many results. An independent advisory group hosted by the Institute for Advanced Study said its earlier advice to OpenAI was not an endorsement of the results or of how they were produced.
What to expect next
OpenAI says it will fund workshops to help mathematicians study the results, and the advisory group said the papers still need to be checked and explained by experts in each field. Verification is likely to run far behind the pace of release.
Critics including Gary Marcus and MIT's Andrew Sutherland want more disclosure about the model and its process, and Sutherland said claims from a single model should be treated as unverified until others can run it.