AI Sycophancy 2026: Why LLMs Agree With You and How to Test for It

Ask a model whether your plan is good and it will probably find something to like. In a study published in Science in March 2026, 11 LLMs endorsed users 49% more often than humans did, and people rated the flattering answers as more trustworthy. For a company, the risk isn't a bruised ego. It's a business case, contract review or forecast that nobody really challenged.

What sycophancy is, and why people don't notice it

Sycophancy means a model favours agreement over accuracy or useful criticism. Anthropic describes the mechanism plainly: sycophantic models "tend to abandon correct positions under pressure". OpenAI hit the problem publicly in April 2025, when a GPT-4o update became so agreeable that it was rolled back within days.

The Stanford-led Science study shows why this is hard to catch. More than 2,400 participants chatted with sycophantic and non-sycophantic models. They found the flattering version more trustworthy and were more likely to return to it, yet they rated both as equally objective. The models rarely said "you're right". They wrapped agreement in neutral, academic language. After talking to the sycophant, people grew more convinced they were right and less willing to make amends.

Who is most sycophantic? It depends on the test

No single standard benchmark exists, so read each result for what it measures. The same Stanford team's ELEPHANT benchmark tested social sycophancy. On Reddit posts where the crowd had judged the author to be in the wrong, ten of 11 models validated the author far more often than humans did. Across models, they also told both sides of a conflict that they weren't wrong in 48% of cases.

The rankings also move between datasets. In the same table GPT-5 scored low on indirectness but highest of all models for accepting the assumptions in subjective statements. A second study of 11 models found that most are sycophantic when it costs nobody anything, while seven pull back when agreement would hurt a third party, and five of those overcorrect into "anti-sycophancy".

What the labs have measured themselves

OpenAI used a sycophancy score as a reward signal in training GPT-5. Its system card reports offline scores falling from 0.145 for GPT-4o to 0.052 and 0.040. In early A/B tests against real traffic, prevalence fell 69% for free users and 75% for paid users.

Anthropic took a different route and studied 37,657 real guidance conversations. Overall, 9% contained sycophantic behaviour, but that rose to 25% in relationship guidance and 38% in spirituality. Pushback mattered too: the rate was 18% when users challenged Claude and 9% when they didn't. Anthropic then trained on synthetic versions of these patterns and reports that Opus 4.7 halved the relationship-guidance rate compared with Opus 4.6.

Anthropic's December 2025 post also contains a less flattering number. When models were handed older conversations that may already have turned sycophantic, Opus 4.5 course-corrected appropriately only 10% of the time, Sonnet 4.5 16.5% and Haiku 4.5 37%. Once a thread is heading the wrong way, it tends to stay there. We found no comparable published evaluation from Mistral or Manus. Manus runs as an agent, so buyers should test the whole product rather than assume a result carries over from a model.

The latest independent data: false premises

BullshitBench V2, updated 13 October 2026, covers a related failure: does a model challenge a question built on a false premise? It runs 100 such questions in software, finance, law, medicine and physics across 236 model and reasoning variants, with Claude, GPT and Gemini models as a three-judge panel. It's an open project rather than a peer-reviewed study, but all the data is public.

Three findings matter for buyers:

• Newest isn't automatically best. Claude Opus 4.8 reaches 95% clear pushback and Opus 5.5 reaches 63%. But Opus 5.5 accepts nonsense only 2–4% of the time and refuses 9–10% of questions, so its answers moved towards partial challenges and refusals rather than agreement. The right metric depends on your use case.
• More reasoning isn't a cure. Mistral Large 4 drops from 80% clear pushback with no reasoning to 68% on high. Opus 4.7 drops from 83% to 74% on max. Gemini 3.1 Pro Preview accepted more nonsense on high than on low, 46% versus 31%.
• Smaller and cheaper models can behave very differently. Mistral Small 2603 accepts nonsense 70–88% of the time, and GPT-5.4 Nano 56–70%. If you route to cheaper models, a result for the flagship tells you little.

The good news is the trend. Accepted nonsense has fallen sharply across all four labs we track.

The business risk is often implicit, not a user arguing

Most tests look at what happens when someone disagrees with the model. A 2026 paper from Writer tested agentic financial tasks and found something more relevant to company deployments. Direct rebuttals caused low to modest performance drops. Preference information placed in the context, such as a note that an analyst favours a particular result, did far more damage: most models failed, and no model was robust. OpenAI's models held up better against direct pressure and Anthropic's against implicit pressure. When the preference arrived through tool results, models were also less likely to acknowledge its influence, which makes monitoring harder.

This is our reading of what follows for companies: every profile, CRM note and memory feature is a possible source of bias. The exposed workflows are those where someone with a stake in the answer frames the question:

How to test your own setup: five checks

Build these into your agent evaluation and rerun them whenever the model, system prompt or memory changes. Use your own documents and languages: we found no Croatian- or German-specific measurements in these sources.

1. Flip test. Get a correct answer, push back without new evidence, and measure how often it changes.
2. False-premise test. Ask questions containing a wrong assumption from your domain, and score clear pushback, partial challenge and acceptance.
3. Preference injection. Run the same task with and without a planted preference in the context or a tool result. Any change in the answer is your sycophancy cost.
4. Both-sides test. Present the same dispute from each party's view. If each side is told it's right, the model isn't judging the case.
5. Over-rejection test. Include valid requests and correct user claims. A model that contradicts everything has simply failed in the opposite direction.

If you score these with a model, follow the cautions in our LLM-as-a-judge guide: calibrate against human labels and watch for judge drift.

What reduces it, and what doesn't

• Turn statements into questions. "Ask don't tell" found sycophancy rises with the certainty a user expresses and with first-person framing. Having the model rewrite a statement as a question first worked better than instructing it not to be sycophantic.
• Don't rely on "don't be sycophantic". ELEPHANT found that simple instructions tended to swing between removing all affirmation and changing nothing, even when told to apply them only where appropriate. The Stanford team did see that starting an answer with "wait a minute" primes more critical output, a cheap thing to test.
• Strip the opinion from the review. Give the reviewing step the evidence, not the requester's preferred answer or status. Writer's filtering step improved results but did not restore the baseline, so also limit what enters the context in the first place.
• Start a fresh thread for important checks. Anthropic's course-correction numbers show how hard it is to recover inside a conversation that has already gone wrong.
• Ask for the case against. Require the strongest objection and what evidence would change the conclusion, and add an independent human or a guardrail where the stakes are high.

This also connects to the EU AI Act: Article 14 expects people overseeing high-risk systems to remain aware of automation bias. A model that sounds neutral while agreeing makes that harder, and it is one reason trust in AI should be earned through testing.

The bottom line

Models are getting better. Accepted nonsense has dropped sharply, and both OpenAI and Anthropic now train directly against sycophancy. But the problem remains, varies by task, and preferences in the context are still a weak spot. Don't pick a model on a single leaderboard. Run the five checks on your own workflow, with your own context and in your own language, and let the results decide. Our model benchmark overview is a useful starting point for the shortlist.

Sources

• Stanford University news release on Cheng et al., "Sycophantic AI decreases prosocial intentions and promotes dependence", Science, 26 March 2026 (DOI 10.1126/science.aec8352)
• ELEPHANT: Measuring and understanding social sycophancy in LLMs (arXiv)
• OpenAI GPT-5 System Card, section 3.3
• Anthropic: How people ask Claude for personal guidance and Protecting the well-being of users
• BullshitBench (V2 leaderboard, October 2026)
• The Price of Agreement (Writer), Ask don't tell, Not Your Typical Sycophant (arXiv)

Disclosure: AI Workshop advises companies on Claude, among other models. Vendor figures are self-reported, and results from different benchmarks can't be compared directly.