Study Reveals Factual Errors in AI-Generated Academic Texts
A new study examines the prevalence of factual errors, known as hallucinations, in texts generated by large language models (LLMs) for academic purposes.
What happened?
Researchers have conducted a study analysing how four major language models — ChatGPT, Grok, Gemini, and Copilot — perform when generating academic texts. The study evaluated the models' capabilities in areas such as reference generation, fact-based explanations, abstract generation, and text refinement. A total of 80 prompts were used across these categories.
Key facts
”Large Language models (LLMs) show extraordinary abilities, but they are still prone to hallucinations, especially when we use them for generating Academic content. We have investigated four popular LLMs, ChatGPT, Grok, Gemini, and Copilot for hallucinations specifically for acade”
Why it matters
Large language models are becoming increasingly common tools in academic research and writing. The discovery that these models generate hallucinations — plausible but incorrect or fabricated answers — highlights a critical challenge. This can undermine the reliability of AI-generated academic content and affect the integrity of research processes.
Who is affected?
Researchers, students, educators, and AI model developers are directly affected. Academic institutions must address the use of these models, while AI developers gain insights into areas that require improvement regarding factual accuracy. Users of LLMs for academic purposes must be aware of the risk of inaccuracies.
What else you should know
The study introduced a new weighted metric, the Hallucination Index (HI), to quantify the degree of hallucinations. Certain models showed better performance in specific tasks, such as Grok and Copilot in reference generation, but still exhibited flaws in overall factual accuracy.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Vilka modeller testades i studien?
The link opens in a new window and leads to the publisher's own site.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.
Get similar news straight to your inbox
The reader's room
Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.
Sign in to submit a comment or question.
Read the article through your role
- Decide whether this affects strategy over 6–12 months or is just noise.
- Discuss with leadership: do we own the right question or does ownership need to move?
- Ask: what risk are we taking by NOT acting on this this quarter?
Generated angle — not editorial analysis of "Study Reveals Factual Errors in AI-Generated Academic Texts"