Study: LLMs generate social comparison but lack ability to detect it
A new study shows that large language models (LLMs) can generate text that evokes social comparison in readers, yet lack the ability to reliably identify these signals in text.
Vad har hänt
Researchers have introduced a new benchmark called XHS-SCoRE (Xiaohongshu Social Comparison Reader Elicitation) to assess whether text-generated posts evoke upward, downward, or neutral social comparison from the reader's perspective. The study utilised prompt-based LLM classifiers and Chinese encoder models to analyse content from the social media platform Xiaohongshu (RedNote). The results show that while LLMs can effectively create content that triggers social comparison, they fail to consistently detect this effect in existing text.
Key facts
| Benchmark namn | XHS-SCoRE |
|---|---|
| Plattform för studie | Xiaohongshu (RedNote) |
| Typ av jämförelse | Uppåtgående, Nedåtgående, Neutral |
”We introduce Xiaohongshu Social Comparison Reader Elicitation (XHS-SCoRE), a reader-grounded benchmark for detecting if a text-only Xiaohongshu (RedNote) post elicits UPWARD, DOWNWARD, or NEUTRAL/no clear social comparison from a first-person reader perspective.”
”Across prompted LLM classifiers and supervised Chinese encoder baselines, we find a consistent mismatch between generation fluency and reliable detection ability: the signal is textually learnable in-domain, but not robustly accessible to prompt-based classification.”
”A controlled pilot further shows that LLM-generated Xiaohongshu-style posts can shift perceived standing and comparison-related affect even w”
Varför det spelar roll
This discrepancy between generative capability and detection capability highlights a significant limitation in current LLMs. The ability to generate text that influences the reader's self-perception and emotions is central to the understanding of online social interaction. The fact that LLMs cannot identify such signals could lead to unintended effects when used in social media or other platforms where text can impact user well-being.
Vem påverkas
The study primarily affects developers and researchers within AI and machine learning working with LLMs and natural language processing. Users of social media platforms may be indirectly affected by content generated by LLMs, especially if it evokes social comparison without the platform's systems being able to moderate it. Companies using LLMs for content creation or moderation need to take these limitations into account.
EU-status
Not applicable for EU status. The study focuses on technical limitations of LLMs and Xiaohongshu, a Chinese platform.
Mer att veta
The controlled pilot study showed that LLM-generated posts in the style of Xiaohongshu, even if not directly detected by the models themselves, can still influence readers' perception of social standing and comparison-related emotions.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Vilka bolag berörs?
Länken öppnar i nytt fönster och leder till utgivarens egen sida.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.