Llama 4

Learning Objectives

• Understand the Llama 4 model family and how Mixture-of-Experts architecture works
• Compare Maverick, Scout, and Llama 3.3 to choose the right model for different use cases
• Evaluate Llama 4's open-weight licensing and its implications for developers and enterprises

What Is Llama 4?

Llama 4 is Meta's latest generation of open-weight foundation models — the most downloaded open-weight frontier models in the world. Released in April 2025, Llama 4 introduced Meta's first Mixture-of-Experts (MoE) architecture, which dramatically improves efficiency: the models have hundreds of billions of total parameters but only activate a fraction for each token, achieving frontier performance at a fraction of the compute cost.

Meta open-sources Llama models as a strategic choice: by commoditizing the model layer, Meta reduces the cost of AI infrastructure that powers its own products (Facebook, Instagram, WhatsApp, Ray-Ban glasses) while building an ecosystem that makes Llama the default choice for developers worldwide.

The Llama 4 Family

Llama 4 Maverick — The Flagship

Llama 4 Maverick is Meta's most capable open-weight model:

• 400 billion total parameters, 17 billion active — MoE architecture with 128 experts, 1 routed per token
• 1 million token context window — matching Claude Opus 4.8 and GPT-5.6
• Multimodal — processes both text and image inputs natively
• LMArena Elo: 1,417 at launch — competitive with frontier closed models
• Available under Meta's community license (free for most commercial use)

The MoE architecture is key: although Maverick has 400 billion total parameters, only 17 billion are active for each token. This means it delivers frontier-class performance while requiring significantly less compute per inference than a dense 400 billion parameter model.

Llama 4 Scout — Extended Context

Llama 4 Scout is optimized for extremely long context scenarios:

• 109 billion total parameters, 17 billion active — smaller MoE (16 experts)
• 10 million token context window — the largest context of any major released model, 10-times Maverick's
• Fits on a single H100 GPU — practical for self-hosted deployment
• Ideal for processing massive document collections, entire codebases, or very long conversation histories

Ten million tokens is approximately 7.5 million words — enough to process an entire library of technical documentation or a full year of corporate communications in a single context.

Llama 3.3 70 Billion — The Production Workhorse

While Llama 4 gets the headlines, Llama 3.3 70 billion remains the most widely deployed open-weight model in production:

• Dense architecture (simpler to deploy than MoE)
• Proven reliability across thousands of production deployments
• Performance competitive with the earlier Llama 3.1 405 billion at a fraction of the cost
• Runs on a single high-end GPU (A100 or H100)

For teams that need a proven, efficient, well-understood model, Llama 3.3 is often the pragmatic choice.

Licensing

For individual developers and most businesses, the license is effectively free and unrestricted. The 1 million MAU threshold only affects the largest companies.

Choosing Between Llama Models

Llama 4 vs. Competing Open-Weight Models

The "Avocado" Question Is Settled

The model long rumored under the code name "Avocado" shipped on April 8, 2026 as Muse Spark, the first flagship from Meta Superintelligence Labs — and it is proprietary, not open-weight. Meta has signalled an eventual open release, but the model was built first to power Meta AI across WhatsApp, Instagram, Facebook, Messenger, and Ray-Ban glasses. In July 2026 Meta shipped Muse Spark 1.1 through a new OpenAI-compatible Model API — an API, not open weights.

Strengths

• Most downloaded open-weight frontier model — largest community, most third-party fine-tunes, broadest cloud provider support
• MoE efficiency — frontier performance with only 17 billion active parameters per token
• 1 million / 10 million token context — Maverick matches closed models; Scout offers 10-times more
• Multimodal — native text and image processing in Maverick
• Free for most commercial use — Meta community license is effectively unrestricted for the vast majority of developers
• Broad deployment options — Hugging Face, all major clouds, local via Ollama, fine-tunable with Torchtune

Limitations & Considerations

• Not fully open-source — Meta's license restricts companies above 1 million MAU; not OSI-compliant
• MoE complexity — MoE models are harder to deploy, fine-tune, and optimize than dense models; some teams prefer Llama 3.3 for simplicity
• Meta's frontier is now closed — the "Avocado" model shipped April 2026 as the proprietary Muse Spark, and Llama 4 Behemoth was deprioritized behind it; Llama still ships, but plan for the possibility that this generation has no open successor at the frontier
• Llama 4 Behemoth delays — the largest Llama 4 model remains unreleased, reportedly facing engineering challenges
• Benchmark gaps — while competitive, Llama 4 Maverick trails Claude Opus 4.8 and GPT-5.6 on SWE-bench and some reasoning tasks

Key Takeaways

• Llama 4 is Meta's open-weight frontier model family — Maverick (400 billion/17 billion MoE, 1 million context, multimodal) and Scout (10 million context, single-GPU) are the most downloaded open-weight models in the world
• MoE architecture delivers frontier performance at a fraction of the inference cost of dense models of equivalent total parameter count
• Free for most commercial use under Meta's community license; Llama 3.3 70 billion remains the go-to for proven production deployments
• Meta's shift is confirmed, not potential — "Avocado" shipped April 2026 as the proprietary Muse Spark, and Llama 4 Behemoth was deprioritized behind it; Llama still ships, but Meta's most capable model is no longer open