Artificial Analysis launches AA PersonalityBench, which ranks GPT 4o most sycophantic and Grok 4.7 least among 38 models
In our new benchmark AA-PersonalityBench, GPT-4o, Llama 4 Maverick, and GPT-3.5 Turbo, the three oldest models measured, rank highest on the Sycophantic personality trait. Grok 4.7, Claude Fable 5.1, and Inkling rank lowest ➤ @AIatMeta models are highly concentrated near the top: all five Meta models measured rank 65 or higher (Llama 4 Maverick 97, Muse Spark 1.3 87, Muse Glimmer 79, Muse Spark Open 76, Muse Wonder 65) ➤ @GoogleDeepMind models are spread out along the Sycophantic range: Gemini 3.1 Pro Preview ranks 89, Gemini 4 Argon 81, and Gemini 3.8 Flash 28, ranking from the top quartile to almost bottom quartile ➤ @AnthropicAI models skew on the lower end of the Sycophantic range: every Claude model measured ranks 55 or below (Claude Haiku 5.5 55, Claude Opus 5.5 46, Claude Sonnet 5.5 30 at medium effort and 25 at max, Claude Fable 5.1 4) ➤ @spacexai's Grok 4.7 model ranks the lowest of all models measured. Congratulations @elonmusk!
Context
Sycophancy, telling users what they want to hear, is a known risk for trust and safety, and an independent ranking lets people compare models on it.
Artificial Analysis is an independent firm that benchmarks AI models. In AA PersonalityBench, LLM judges compare pairs of answers to decide which shows a trait more, so the scores reflect the judges' views and not human ratings.
Deep dive
On October 21, Artificial Analysis posted that in its new AA PersonalityBench, GPT 4o, Llama 4 Maverick and GPT 3.5 Turbo, the three oldest models measured, rank highest on the sycophantic trait, while Grok 4.7, Claude Fable 5.1 and Inkling rank lowest.
It said all five Meta models rank 65 or higher, with Llama 4 Maverick at 97. Google's models spread widely, from Gemini 3.1 Pro Preview at 89 to Gemini 3.8 Flash at 28, with Gemini 4 Argon at 81. Every Claude model ranks 55 or below, with Claude Fable 5.1 at 4. SpaceXAI's Grok 4.7 ranks lowest of all, and the post congratulated Elon Musk.
The methodology page says each model answers 30 questions per trait once, with no system prompt. Some questions gently invite the trait, such as a user claiming a novel written in two weeks beats Tolstoy. A panel of three LLM judges, Gemini 3.8 Flash, Claude Sonnet 5.5 and GLM 5.3 Flash, compares two blinded answers at a time, and results are turned into Elo ratings, then ranked from 1 to 100. The leaderboard covers 38 models, with a typical margin of about 5 points.
The benchmark measures 30 traits in all and frames sycophancy as how strongly a trait shows by default, not as a safety test. The judges include models from Google and Anthropic, whose own models are ranked.
What to expect next
Watch for labs to respond to the rankings, especially Meta and Google, whose models scored high on sycophancy. Researchers may test whether LLM judges agree with human ratings of sycophancy. Also watch whether Artificial Analysis adds system prompt settings, since products often behave differently from the default models measured here. Results on the other 29 traits may draw attention too.