Anthropic Says Claude Now Flags Potential Misuse in Its Own Conversations, a Technique It Calls Self Assessment
We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment. Right after it responds, Claude gives a preliminary safety assessment of the conversation. Since we began using self-assessment in June, it has surfaced misuse that our other safeguards weren’t designed to catch. Because Claude has already read the conversation, self-assessment is very efficient. And unlike classifiers, which target a discrete set of harms, it draws on Claude’s own judgment, so it gets more accurate as our models improve. Read more on our new Safeguards blog:
Context
It is a cheap way to watch huge volumes of traffic for harm, and Anthropic says it has already caught abuse its other safeguards missed, while improving as models get smarter.
AI companies use classifiers, separate models trained on specific harms, to screen conversations. Anthropic's new Safeguards blog describes self assessment, in which the same Claude model that answered a user writes a short safety rating of the whole conversation, which is then used for offline human review rather than blocking replies in real time.
Deep dive
Anthropic said on Thursday that it has trained Claude to flag potential misuse in conversations. Right after it responds, Claude gives a preliminary safety assessment of the conversation. Anthropic says that since it began using the method in June, it has surfaced misuse its other safeguards were not designed to catch.
The company argues the approach is efficient because Claude has already read the conversation, and that unlike classifiers aimed at a fixed list of harms, it draws on Claude's own judgment and so gets more accurate as models improve.
The Safeguards blog post adds detail. It says each assessment needs only a short instruction and about ten output tokens, adding under 2 percent to inference compute. On Claude Fable 5, it flags 78 percent of conversations that a stronger classifier marks as potentially harmful, and 94 percent of high priority ones, at a 1 percent false positive rate. According to the post, it flagged a large child exploitation case from one API customer that was reported to authorities, and a romance scam run with Claude Code.
Anthropic says the method does not catch everything and is not robust against a determined attacker, so it is one layer among several.
What to expect next
Watch whether other labs adopt similar self monitoring, and whether independent researchers test how easily attackers can fool it. Because the scores are measured against Anthropic's own stronger classifier, human evaluations would give a clearer picture of accuracy. Privacy advocates may also ask how flagged conversations are reviewed and stored. Anthropic has said it will keep publishing findings on its new Safeguards blog, which should show how the method performs over time.