Meta unveils Llama 4: Moving to Mixture of Experts
Meta has announced Llama 4, introducing a Mixture of Experts architecture and a context window of ten million tokens. However, the launch has met with mixed reactions regarding the model's test results.
What happened?
Meta has unveiled Llama 4, the first model family in the Llama series to be built on a Mixture of Experts (MoE) architecture. Previous versions were dense models where all parameters were activated for each token, whereas Llama 4 uses only a subset of parameters per word. The model is built to be natively multimodal and is introduced with a context window of up to ten million tokens.
Key facts
| Lanseringsdatum | 5 april 2025 |
|---|---|
| Kontextfönster | 10 miljoner tokens |
| Arkitektur | Mixture of Experts (MoE) |
Why it matters
The model's launch has attracted significant attention in the AI industry, not least because of the shift in architecture and the immense context window. However, the release was quickly followed by debates regarding how the model's results on various benchmarks and leaderboards were evaluated, creating a divided view of its performance among industry analysts.
Who is affected?
The launch primarily affects AI developers, researchers, and companies building applications on top of open models. The extreme context length makes the model particularly interesting for organisations analysing large volumes of documents or codebases.
Impact on the EU
Llama 4 is released as open weights, making it accessible to European developers and companies. Meta must, however, continue to comply with the EU AI Act and GDPR regarding the training on European user data.
What else you should know
The transition to an MoE architecture marks a strategic shift for Meta, as it enables significantly more capable models without inference computational costs spiralling at the same rate. At the same time, the discussions surrounding benchmarks show that the industry is placing increasingly high demands on independent evaluation.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Påverkar det EU?
The link opens in a new window and leads to the publisher's own site.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.
AI-verktyg i artikeln
Topics
Get similar news straight to your inbox
The reader's room
Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.
Sign in to submit a comment or question.
Read the article through your role
- Decide whether this affects strategy over 6–12 months or is just noise.
- Discuss with leadership: do we own the right question or does ownership need to move?
- Ask: what risk are we taking by NOT acting on this this quarter?
Generated angle — not editorial analysis of "Meta unveils Llama 4: Moving to Mixture of Experts"