Skip to content
Forskning· Analysis

Hugging Face enables 180B parameter MoE models with 32 GB RAM

Hugging Face is introducing expert offloading in the Transformers library. The new method makes it possible to run 180 billion parameter MoE models on computers with only 32 GB of RAM.

By the Aheadline editorial team·19 nov. 2026·2 min read·Source: Hugging Face BlogVerifierad signalAI-generated
Hugging Face enables 180B parameter MoE models with 32 GB RAM
Hugging Face enables 180B parameter MoE models with 32 GB RAM
Hugging Face enables 180B parameter MoE models with 32 GB RAM
By · Policy- & EU-reporter
Last updated
Vad betyder det för mig?

What happened?

Hugging Face has implemented support for expert offloading during inference for Mixture-of-Experts (MoE) models in the Transformers library. The technology makes it possible to run models with up to 180 billion parameters, such as DBRX or Qwen1.5-MoE, on consumer hardware with only 32 GB of RAM. By loading only the expert networks actually activated for a specific token directly from the storage disk, memory requirements are reduced drastically.

Key facts

Maximal modellstorlek180 miljarder parametrar
Lägsta RAM-krav32 GB RAM
MjukvarubibliotekHugging Face Transformers
ModellarkitekturMixture-of-Experts (MoE)

Why it matters

MoE architectures activate only a fraction of their total parameters during each computational step, yet traditional inference has historically required all weights to be held in system memory or VRAM. Expert offloading significantly lowers the barrier to entry for experimenting with the largest open models. This democratises access to advanced AI and reduces dependence on proprietary cloud services.

Who is affected?

The solution is primarily aimed at AI researchers, open-source developers, and smaller organisations looking to evaluate or run large MoE models locally without access to expensive data centre GPUs. Developers with consumer PCs or standard workstations now gain access to models that previously required multi-card systems.

Impact on the EU

The method is distributed as open-source via Transformers and is directly available to European developers. It facilitates the local execution of open AI models, which can help organisations comply with GDPR data protection requirements by avoiding cloud-based API calls outside the Union.

What else you should know

Performance depends largely on the read speed of the storage medium, with fast NVMe drives required to minimise latency when expert weights are loaded dynamically. While inference is slower than with pure GPU storage, it makes the models practically testable without specialised hardware.

Frequently asked questions

Quick answers about this story

Vad har hänt?
Hugging Face har introducerat stöd för expertströmning i sitt Transformers-bibliotek, vilket gör att MoE-modeller med 180 miljarder parametrar kan köras på 32 GB RAM.
När hände det?
Tekniken presenterades i ett tekniskt blogginlägg på Hugging Face Blog under våren 2024.
Varför spelar det roll?
Det sänker hårdvarukraven för att köra mycket stora öppna språkmodeller dramatiskt, vilket gör att forskare och utvecklare kan köra dem på vanliga konsumentdatorer istället för på dyra GPU-kluster.
Påverkar det EU?
Lösningen är tillgänglig globalt som öppen källkod och gör det enklare för organisationer i EU att köra modeller lokalt utan att skicka data till externa molntjänster.
Original source
Hugging Face Blog·huggingface.co

The link opens in a new window and leads to the publisher's own site.

Verifierad signal

Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.

[ STAY UP TO DATE ]

Get similar news straight to your inbox

No affiliate linksCancel anytimeGDPR-friendly
[ Frequency ]
[ What do you want to read about? ]

You'll receive updates on 2 topics.

The reader's room

Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.

Sign in to submit a comment or question.

Loading comments…
How this affects you

Read the article through your role

  • Decide whether this affects strategy over 6–12 months or is just noise.
  • Discuss with leadership: do we own the right question or does ownership need to move?
  • Ask: what risk are we taking by NOT acting on this this quarter?

Generated angle — not editorial analysis of "Hugging Face enables 180B parameter MoE models with 32 GB RA"