Meta Unveils Llama 4 – Shifts to Mixture of Experts
Meta has announced Llama 4, introducing a Mixture of Experts architecture and a context window of 10 million tokens. However, the launch has been met with mixed reactions regarding the model's test results.
What happened?
Meta has unveiled Llama 4, the first model family in the Llama series to be built on a so-called Mixture of Experts (MoE) architecture. Previous versions were dense models where all parameters were activated for every token, whereas Llama 4 only uses a smaller subset of parameters per word. The model is built to be natively multimodal and is introduced with a context window of up to 10 million tokens.
Key facts
| Lanseringsdatum | 5 april 2025 |
|---|---|
| Kontextfönster | 10 miljoner tokens |
| Arkitektur | Mixture of Experts (MoE) |
Why it matters
The model's release has drawn significant attention within the AI industry, not least because of the architectural shift and the massive context window. However, the launch was quickly followed by debates regarding how the model's results on various benchmarks and leaderboards have been evaluated, leading to a divided assessment of its performance among industry analysts.
Who is affected?
The launch primarily impacts AI developers, researchers, and companies building their own applications on top of open models. The extreme context length makes the model particularly interesting for organisations analysing very large document volumes or codebases.
Impact on the EU
Llama 4 is released as open weights, making it accessible to European developers and companies. Meta must, however, continue to adhere to the EU AI Act and GDPR regarding the training on European user data.
What else you should know
The transition to an MoE architecture marks a strategic shift for Meta, as it enables significantly more capable models without inference computational costs spiralling at the same rate. Simultaneously, the discussions surrounding benchmarks demonstrate that the industry is placing increasingly high demands on independent evaluation.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Påverkar det EU?
The link opens in a new window and leads to the publisher's own site.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.
AI-verktyg i artikeln
Topics
Get similar news straight to your inbox
The reader's room
Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.
Sign in to submit a comment or question.
Read the article through your role
- Decide whether this affects strategy over 6–12 months or is just noise.
- Discuss with leadership: do we own the right question or does ownership need to move?
- Ask: what risk are we taking by NOT acting on this this quarter?
Generated angle — not editorial analysis of "Meta Unveils Llama 4 – Shifts to Mixture of Experts"