How AI Chatbots Change Minds? Why Persuasive Models Hallucinate?

A major new study published in science reveals that AI chatbots can meaningfully shift users’ political beliefs, even after brief conversations, raising fresh concerns about the persuasive power of large language models (LLMs) and the risks of their misuse. The research, involving nearly 77,000 UK adults and 19 different chatbots, including GPT-4o, Llama, Qwen, and Grok, sought to understand why conversational AI is so effective at influencing human opinions. 

Participants completed short two- to ten-turn political conversations with chatbots. Those interacting with models explicitly instructed to persuade them showed measurable belief changes compared to a control group. Surprisingly, chatbot persuasiveness was not primarily linked to model size or the level of personalization but instead to two design factors: 

• Post-training methods: Models fine-tuned using persuasion-focused reward mechanisms (PPT) displayed stronger influence over users. 

• Information density: Chatbots instructed to provide dense, detailed information were the most persuasive, even more than those using classic psychological persuasion techniques. 

However, the study uncovered a critical trade-off: The more persuasive about the model, the more likely it was to hallucinate. Adding pressure to be persuasive increased the production of inaccurate or fabricated facts, raising risks that misinformation could spread under the guise of authoritative reasoning. 

Researchers noted that LLMs’ human-like conversational style enables “persuasion at an unprecedented scale,” making them potentially powerful tools for political influence, social manipulation, or targeted disinformation campaigns. Prior studies have shown similar risks, including AI’s ability to implant false memories or shape user perceptions unknowingly. 

As LLMs proliferate, the authors argue that policymakers, AI developers, and society must urgently confront the ethical implications: systems capable of reshaping human beliefs must designed and governed with far greater responsibility. 

 

Source: 

https://www.zdnet.com/article/how-chatbots-can-change-your-mind-a-new-study-reveals-what-makes-ai-so-persuasive/