Forskning· Analysis

AI Models Create Credible Disinformation – But Fail to Detect It

A new study shows that multimodal AI models can collaborate to generate credible disinformation at scale. Simultaneously, 16 tested AI models failed to detect the fabricated posts.

By the Aheadline editorial team·30 sep. 2026·2 min read·Source: arXiv cs.CL (NLP/LLM)Verifierad signalAI-generated

By Alma Berg · Policy- & EU-reporter

30 September 2026

Vad betyder det för mig?

What happened?

Researchers have developed a multi-agent system where three collaborating AI agents create misleading social media posts. The system generated over 9,000 pairs of multimodal fake news across the sectors of science, health, and entertainment. When 16 leading open and closed-source MLLMs were tested on their ability to detect the fabricated material, most performed significantly worse than human evaluators.

Key facts

Genererade nyhetspar	> 9 000 st
Testade MLLM-modeller	16 modeller
Publiceringsdatum	26 september 2026

Why it matters

The report indicates that generative AI has reached a level where it can automatically produce credible disinformation at scale. The fact that existing AI models struggle specifically with verifying the authenticity of images presents a major challenge for automated moderation on digital platforms.

Who is affected?

The findings are relevant to AI model developers, security researchers, fact-checkers, and social media operators. Regular users and authorities working to counter information operations are also affected by the identified security vulnerabilities.

Impact on the EU

The research concerns global platforms and is directly relevant to the implementation of the EU's AI Act and Digital Services Act (DSA). EU regulations increasingly demand that technology companies mitigate risks and identify AI-generated disinformation.

What else you should know

The study was published as a preprint on arXiv in September 2026. The results underscore a growing need for more advanced tools for image and text verification as generative models become widely accessible to the public.

Frequently asked questions

Quick answers about this story

Vad har hänt?

Forskare publicerade en studie som visar att multiagentsystem baserade på multimediala AI-modeller (MLLM) kan generera över 9 000 trovärdiga falska nyhetsinlägg, samtidigt som 16 ledande AI-modeller misslyckades med att tillförlitligt upptäcka dem.

När hände det?

Studien publicerades som ett preprint på arXiv den 26 september 2026.

Varför spelar det roll?

Resultaten visar att AI-modeller har stora brister i att upptäcka AI-genererad desinformation, särskilt gällande bildäkthet. Det gör automatiserad granskning på sociala medier osäker inför storskaliga desinformationskampanjer.

Vad var AI-modellernas största svaghet?

Modeller som testades misslyckades särskilt kritiskt med att bedöma bilders äkthet, vilket visar att visuell fusk är svårast för nuvarande AI att avslöja.

MLLM arXiv AI Act Digital Services Act Multimodal LLM

Original source

arXiv cs.CL (NLP/LLM)·arxiv.org

The link opens in a new window and leads to the publisher's own site.

Verifierad signal

Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.

AI-verktyg i artikeln

Topics

#Models

[ STAY UP TO DATE ]

Get similar news straight to your inbox

The reader's room

Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.

Loading comments…

How this affects you

Read the article through your role

Decide whether this affects strategy over 6–12 months or is just noise.
Discuss with leadership: do we own the right question or does ownership need to move?
Ask: what risk are we taking by NOT acting on this this quarter?

Generated angle — not editorial analysis of "AI Models Create Credible Disinformation – But Fail to Detec"