Study Reveals Factual Errors in AI-Generated Academic Texts

Vad har hänt

Researchers have conducted a study analysing how four major language models — ChatGPT, Grok, Gemini, and Copilot — perform when generating academic texts. The study evaluated the models' capabilities in areas such as reference generation, fact-based explanations, abstract generation, and text refinement. A total of 80 prompts were used across these categories.

Varför det spelar roll

Large language models are becoming increasingly common tools in academic research and writing. The discovery that these models generate hallucinations — plausible but incorrect or fabricated answers — highlights a critical challenge. This can undermine the reliability of AI-generated academic content and affect the integrity of research processes.

Vem påverkas

Researchers, students, educators, and AI model developers are directly affected. Academic institutions must address the use of these models, while AI developers gain insights into areas that require improvement regarding factual accuracy. Users of LLMs for academic purposes must be aware of the risk of inaccuracies.

EU-status

Ej relevant för EU-status.

Mer att veta

The study introduced a new weighted metric, the Hallucination Index (HI), to quantify the degree of hallucinations. Certain models showed better performance in specific tasks, such as Grok and Copilot in reference generation, but still exhibited flaws in overall factual accuracy.