Llama 4

Learning Objectives

• Understand the Llama 4 model family and how Mixture-of-Experts architecture works
• Compare Maverick, Scout, and Llama 3.3 to choose the right model for different use cases
• Evaluate Llama 4's open-weight licensing and its implications for developers and enterprises

What Is Llama 4?

Llama 4 is Meta's latest generation of open-weight foundation models — the most downloaded open-weight frontier models in the world. Released in April 2025, Llama 4 introduced Meta's first Mixture-of-Experts (MoE) architecture, which dramatically improves efficiency: the models have hundreds of billions of total parameters but only activate a fraction for each token, achieving frontier performance at a fraction of the compute cost.

Meta open-sources Llama models as a strategic choice: by commoditizing the model layer, Meta reduces the cost of AI infrastructure that powers its own products (Facebook, Instagram, WhatsApp, Ray-Ban glasses) while building an ecosystem that makes Llama the default choice for developers worldwide.

The Llama 4 Family

Llama 4 Maverick — The Flagship

Llama 4 Maverick is the most capable model in the Llama 4 family (since October 15, 2026, Meta's larger Muse Spark Open sits above it as an open-weight option):

• 400 billion total parameters, 17 billion active — MoE architecture with 128 experts, 1 routed per token
• 1 million token context window — in the same class as the long-context windows of closed frontier models
• Multimodal — processes both text and image inputs natively
• LMArena Elo: 1,417 at launch — competitive with frontier closed models
• Available under Meta's community license (free for most commercial use)

The MoE architecture is key: although Maverick has 400 billion total parameters, only 17 billion are active for each token. This means it delivers frontier-class performance while requiring significantly less compute per inference than a dense 400 billion parameter model.

Llama 4 Scout — Extended Context

Llama 4 Scout is optimized for extremely long context scenarios:

• 109 billion total parameters, 17 billion active — smaller MoE (16 experts)
• 10 million token context window — the largest context of any major released model, 10-times Maverick's
• Fits on a single H100 GPU — practical for self-hosted deployment
• Ideal for processing massive document collections, entire codebases, or very long conversation histories

Ten million tokens is approximately 7.5 million words — enough to process an entire library of technical documentation or a full year of corporate communications in a single context.

Llama 3.3 70 Billion — The Production Workhorse

While Llama 4 gets the headlines, Llama 3.3 70 billion remains the most widely deployed open-weight model in production:

• Dense architecture (simpler to deploy than MoE)
• Proven reliability across thousands of production deployments
• Performance competitive with the earlier Llama 3.1 405 billion at a fraction of the cost
• Runs on a single high-end GPU (A100 or H100)

For teams that need a proven, efficient, well-understood model, Llama 3.3 is often the pragmatic choice.

Licensing

For individual developers and essentially every business, the license is effectively free and unrestricted. The 700 million monthly-active-user threshold only affects a handful of the largest companies on earth.

Choosing Between Llama Models

Llama 4 vs. Competing Open-Weight Models

The "Avocado" Question Is Settled

The model long rumored under the code name "Avocado" shipped on April 8, 2026 as Muse Spark, the first flagship from Meta Superintelligence Labs — and it launched proprietary, not open-weight. The model was built first to power Meta AI across WhatsApp, Instagram, Facebook, Messenger, and Ray-Ban glasses. In July 2026 Meta shipped Muse Spark 1.1 through a new OpenAI-compatible Model API. Open weights followed on October 15, 2026 as Muse Spark Open: 816 billion parameters, about 42 billion active, under a custom Muse Model License that requires a "Built with Muse Spark Open" notice and a separate agreement for any company over $20 million in revenue that resells the model as an API.

Strengths

• Most downloaded open-weight frontier model — largest community, most third-party fine-tunes, broadest cloud provider support
• MoE efficiency — frontier performance with only 17 billion active parameters per token
• 1 million / 10 million token context — Maverick matches closed models; Scout offers 10-times more
• Multimodal — native text and image processing in Maverick
• Free for most commercial use — Meta community license is effectively unrestricted for the vast majority of developers
• Broad deployment options — Hugging Face, all major clouds, local via Ollama, fine-tunable with Torchtune

Limitations & Considerations

• Not fully open-source — the Llama Community License requires Meta's approval above 700 million monthly active users; not OSI-compliant
• MoE complexity — MoE models are harder to deploy, fine-tune, and optimize than dense models; some teams prefer Llama 3.3 for simplicity
• Meta's top model is closed, but its open line continues outside Llama — Muse Wonder is proprietary and Llama 4 Behemoth was shelved, while the 816 billion parameter Muse Spark Open (October 15, 2026) is now Meta's largest open-weight model, under a custom license rather than the Llama Community License
• Llama 4 Behemoth delays — the largest Llama 4 model remains unreleased, reportedly facing engineering challenges
• Benchmark gaps — while competitive, Llama 4 Maverick trails the closed frontier models from Anthropic and OpenAI on SWE-bench and some reasoning tasks

Key Takeaways

• Llama 4 is Meta's open-weight frontier model family — Maverick (400 billion/17 billion MoE, 1 million context, multimodal) and Scout (10 million context, single-GPU) are the most downloaded open-weight models in the world
• MoE architecture delivers frontier performance at a fraction of the inference cost of dense models of equivalent total parameter count
• Free for most commercial use under Meta's community license; Llama 3.3 70 billion remains the go-to for proven production deployments
• Meta's shift is confirmed, not potential — "Avocado" shipped April 2026 as the proprietary Muse Spark, and Llama 4 Behemoth was deprioritized behind it; Meta's most capable model, Muse Wonder, is closed, though Muse Spark itself gained open weights on October 15, 2026