Anthropic Introduces FlashProve, Open Source Provable Inference That Lets Developers Verify Which Model Produced a Response
New Anthropic research: progress on provable inference for large language models. Here, we introduce FlashProve. Building on established cryptographic techniques, FlashProve allows developers to verify whether a given response came from a given AI model. It’s substantially more efficient than prior open-source methods for doing so. These proofs have a range of applications. They can, for instance, make it harder for attackers to hijack a chat, or enable safety evaluators to verify that the model they’re testing is the same one being served to customers. For our full write-up, see here: t.co/OIZWPb6QZa Further progress in provable inference will likely be an important part of improving trust in AI deployment. To that end, we're open-sourcing FlashProve’s code, here: t.co/7ky3Ir8V06 We’re grateful to our collaborators: Alessandro Chiesa, @GiacomoFenzi, @gwberg95, @miranda__christ, Rahul Dalal (t.co/pjWKLHWXyz) and Ben Mayer (@transmissions11).
Context
Proof that a response came from a specific model could help stop chat hijacking and let safety testers confirm they test the model customers actually get.
Provable inference uses cryptographic proofs to show that a given output was produced by a given model, without trusting the server that ran it. Anthropic is the maker of Claude. The work was done with outside collaborators, including the cryptographer Alessandro Chiesa.
Deep dive
Anthropic (@AnthropicAI) posted on Nov 3 that it has made progress on provable inference for large language models, and introduced FlashProve. According to Anthropic, FlashProve builds on established cryptographic techniques and allows developers to verify whether a given response came from a given AI model. The company says it is substantially more efficient than prior open source methods for doing this.
Anthropic gave two example uses. Proofs could make it harder for attackers to hijack a chat, and they could let safety evaluators verify that the model they are testing is the same one being served to customers. The second point addresses a long running worry in AI safety: a lab could in principle show testers one model and ship another.
The company said it is open sourcing the FlashProve code and published a full write up, both linked from the post. It credited collaborators including Alessandro Chiesa, Giacomo Fenzi, Rahul Dalal and Ben Mayer.
The post does not give numbers on speed or cost in its text, so how practical the proofs are for large production models depends on details in the write up. Earlier methods for proving model outputs were usually far too slow for models of this size, which is why an efficiency gain matters.
The announcement passed 290,000 views within a few hours.
What to expect next
Watch for independent cryptographers to review the method and publish benchmarks on real model sizes. If the cost is low enough, auditors and regulators could start asking labs for proofs alongside safety reports. Other labs and cloud providers may adopt or extend the open source code, and API customers may ask for proofs that they are getting the model they pay for.