Hugging Face enables 180B parameter MoE models with 32 GB RAM
Hugging Face is introducing expert offloading in the Transformers library. The new method makes it possible to run 180 billion parameter MoE models on computers with only 32 GB of RAM.

What happened?
Hugging Face has implemented support for expert offloading during inference for Mixture-of-Experts (MoE) models in the Transformers library. The technology makes it possible to run models with up to 180 billion parameters, such as DBRX or Qwen1.5-MoE, on consumer hardware with only 32 GB of RAM. By loading only the expert networks actually activated for a specific token directly from the storage disk, memory requirements are reduced drastically.
Key facts
| Maximal modellstorlek | 180 miljarder parametrar |
|---|---|
| Lägsta RAM-krav | 32 GB RAM |
| Mjukvarubibliotek | Hugging Face Transformers |
| Modellarkitektur | Mixture-of-Experts (MoE) |
Why it matters
MoE architectures activate only a fraction of their total parameters during each computational step, yet traditional inference has historically required all weights to be held in system memory or VRAM. Expert offloading significantly lowers the barrier to entry for experimenting with the largest open models. This democratises access to advanced AI and reduces dependence on proprietary cloud services.
Who is affected?
The solution is primarily aimed at AI researchers, open-source developers, and smaller organisations looking to evaluate or run large MoE models locally without access to expensive data centre GPUs. Developers with consumer PCs or standard workstations now gain access to models that previously required multi-card systems.
Impact on the EU
The method is distributed as open-source via Transformers and is directly available to European developers. It facilitates the local execution of open AI models, which can help organisations comply with GDPR data protection requirements by avoiding cloud-based API calls outside the Union.
What else you should know
Performance depends largely on the read speed of the storage medium, with fast NVMe drives required to minimise latency when expert weights are loaded dynamically. While inference is slower than with pure GPU storage, it makes the models practically testable without specialised hardware.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Påverkar det EU?
The link opens in a new window and leads to the publisher's own site.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.
Get similar news straight to your inbox
The reader's room
Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.
Sign in to submit a comment or question.
Read the article through your role
- Decide whether this affects strategy over 6–12 months or is just noise.
- Discuss with leadership: do we own the right question or does ownership need to move?
- Ask: what risk are we taking by NOT acting on this this quarter?
Generated angle — not editorial analysis of "Hugging Face enables 180B parameter MoE models with 32 GB RA"