HarmProfile: New dataset analyses harmful outputs from 23 language models
A research team has launched HarmProfile, a new dataset containing over 80,000 validated instances of harmful content from 23 advanced language models, designed to create deeper risk profiles for AI systems.
What happened?
The HarmProfile dataset has been launched via arXiv and consists of over 80,000 validated instances where advanced language models generated harmful content. The study covers 23 language models from 13 different model families, distributed across 15 areas of harm and 57 subcategories. Instead of merely recording whether an attack is successful, researchers analyze the actual text generated by the models when security safeguards fail.
Key facts
| Antal validerade fall | Över 80 000 |
|---|---|
| Antal modeller | 23 st |
| Antal modellfamiljer | 13 st |
| Skadekategorier | 15 huvudkategorier, 57 underkategorier |
Why it matters
Traditional evaluations often focus on whether a 'jailbreak' succeeds or fails, without closely examining what the model actually generates. By mapping content, severity, and variation in model failures, researchers can create a model-specific risk profile. This allows for a deeper understanding of how different architectures and training methods influence the nature of harmful output.
Who is affected?
The results are primarily relevant for AI researchers, security experts, and language model developers who need to improve their protection mechanisms. It also provides evaluators with tools to compare risks between different model families based on actual text output.
Impact on the EU
HarmProfile has been published as an open research resource and is available to researchers and authorities within the EU. The methodology of content-centric risk profiling can provide a concrete foundation for security assessments and compliance checks, although the dataset itself is not directly linked to specific EU legislation.
What else you should know
The researchers also noted significant variations in how models respond to the same types of misuse. Certain models exhibit very high levels of severity within specific subcategories, demonstrating that single general safety tests are insufficient to capture complex risk patterns.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Hur många modeller ingår i studien?
The link opens in a new window and leads to the publisher's own site.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.
AI-verktyg i artikeln
Topics
Get similar news straight to your inbox
The reader's room
Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.
Sign in to submit a comment or question.
Read the article through your role
- Decide whether this affects strategy over 6–12 months or is just noise.
- Discuss with leadership: do we own the right question or does ownership need to move?
- Ask: what risk are we taking by NOT acting on this this quarter?
Generated angle — not editorial analysis of "HarmProfile: New dataset analyses harmful outputs from 23 la"