Increased thinking can amplify position bias in AI models
A new analysis, published on arXiv, shows that more complex reasoning paths in AI models can lead to increased position bias in multiple-choice questions.
What happened?
Research published on arXiv on 22 May 2026, titled "More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models", investigates how Chain-of-Thought (CoT) and reasoning-enhanced models affect position bias. The study shows that within reasoning-capable models, position bias per question scales with the length of the reasoning path. This was observed in thirteen different configurations of reasoning modes, including models such as DeepSeek-R1, tested on datasets like MMLU and ARC-Challenge.
Key facts
| Publikationsdatum | 22 maj 2026 |
|---|---|
| Antal modellkonfigurationer testade | 13 |
| Korrelation mellan resonemangslängd och bias (PBS) | 0.11 till 0.41 |
| Testade dataset | MMLU, ARC-Challenge, GPQA |
”within any reasoning-capable model, per-question position bias scales with the length of the reasoning trajectory.”
Why it matters
Traditionally, it has been assumed that models using CoT reasoning reduce superficial heuristic biases through more careful thinking. This study presents a different picture by demonstrating that increased thinking, measured as the length of the reasoning path, actually correlates positively with position bias. This suggests that the mechanisms leading to more complex reasoning can also intensify existing biases related to the placement of answer options.
Who is affected?
The results affect AI researchers and developers working to improve the accuracy and fairness of large language models (LLMs). Companies using or developing models for decision support or automated questioning systems are affected, as position bias can lead to incorrect or unreliable results. Users of AI systems relying on these models may be indirectly affected by potentially biased information presentation.
What else you should know
The study tested thirteen reasoning-mode configurations, including two R1-distilled 7-8B models, two base models with CoT, and DeepSeek-R1 at 671B parameters. The majority of these showed a positive partial correlation between reasoning path length and Position Bias Score (PBS), with p-values below 0.05. A truncation intervention provided causal evidence supporting the results.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Vilka bolag berörs?
The link opens in a new window and leads to the publisher's own site.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.
AI-verktyg i artikeln
Topics
Get similar news straight to your inbox
The reader's room
Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.
Sign in to submit a comment or question.
Read the article through your role
- Decide whether this affects strategy over 6–12 months or is just noise.
- Discuss with leadership: do we own the right question or does ownership need to move?
- Ask: what risk are we taking by NOT acting on this this quarter?
Generated angle — not editorial analysis of "Increased thinking can amplify position bias in AI models"