Study: LLM conformity persists even without influential speakers
A new study reveals that Large Language Models (LLMs) alter correct answers based on repeated incorrect information, even when no speaker is presented.
Vad har hänt
Researchers have published a study on July 5 via arXiv investigating the tendency of Large Language Models (LLMs) toward conformity. The study, "Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks," reveals that "conformity" — where an LLM changes a correct answer in favour of an incorrect one — largely occurs even when no specific speaker is attributed to the incorrect response.
Key facts
”We show that most of this apparent conformity survives even after the peer is removed. The reason is a confound: standard conformity prompts mix two cues at once, the presence of a speaker and the repeated wrong answer itself.”
”Across six open-weight LLMs and seven QA and reasoning datasets, this condition alone causes harmful revision in 66.5% of initially correct cases, compared with 10.3% under a plain re-ask.”
Varför det spelar roll
Previous studies have conflated the effects of a speaker with those of repeated incorrect information. This research isolates the impact of the repeated misinformation itself. This indicates that LLMs are sensitive to external stimuli even in the absence of clear social pressure, deepening the understanding of how models process and revise information.
Vem påverkas
The study primarily affects developers and researchers in the AI field who design and evaluate LLMs. The findings are relevant for those working to minimise bias and improve the reliability of AI systems, as well as for end-users who rely on these systems for accurate information processing.
EU-status
Ej relevant för EU-status.
Mer att veta
The effect persists even when the repeated answer is paraphrased or when response options are hidden in an open context. The framework regarding how the source is presented only marginally affects this fundamental conformity.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Vilka LLM:s berörs?
Länken öppnar i nytt fönster och leder till utgivarens egen sida.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.