Nemotron

Learning Objectives

• Understand what the Nemotron model family is and how it fits into NVIDIA's AI strategy beyond hardware
• Compare the Nemotron models and identify their intended use cases
• Evaluate when Nemotron models are the right choice versus other open-weight alternatives

What Is Nemotron?

Nemotron is NVIDIA's family of open-weight large language models, developed by NVIDIA Research. While NVIDIA is known for GPU hardware, Nemotron represents the company's push into the model layer — competing directly with Meta's Llama, Google's Gemma, and Microsoft's Phi in the open-weight model ecosystem.

The Nemotron family serves a specific strategic purpose: demonstrating and optimizing the full NVIDIA AI stack. Models are trained on NVIDIA hardware, optimized for NVIDIA inference infrastructure (TensorRT-LLM, NIM), and designed to showcase what's possible when hardware and software are tightly integrated.

The Nemotron Model Family

Nemotron 3 (Nano, Super, Ultra)

NVIDIA's newest generation and current flagship line, built on a hybrid Mamba-Transformer architecture. The larger Super and Ultra sizes add Latent MoE — a hardware-aware mixture-of-experts (MoE) expert design — and the whole family is trained in NVIDIA's 4-bit NVFP4 format on Blackwell GPUs, with a 1-million-token context window across all three sizes:

• Nano — roughly 31 billion total parameters with about 3 billion active per token. Tuned for efficient, high-throughput inference and the first size to ship.
• Super — roughly 100 billion total parameters with about 10 billion active per token.
• Ultra — the flagship, roughly 550 billion total parameters with about 50 billion active per token. NVIDIA CEO Jensen Huang unveiled Ultra at his Computex keynote in Taipei, positioning it as the smartest US open-weights model: NVIDIA reports it leads US open-weights rankings on the Artificial Analysis Intelligence Index, generates more than 300 tokens per second, and runs roughly 30 percent cheaper than leading alternatives while delivering up to five-times faster inference.

Weights and training recipes ship free under the NVIDIA Open Model License, optimized for deployment through NIM containers and TensorRT-LLM. Nemotron 3 is aimed squarely at agentic coding, search, and long-context reasoning workloads.

Nemotron-4 340 Billion

NVIDIA's earlier flagship open model, trained on 9 trillion tokens. Available in three variants:

• Base — Pre-trained foundation model for further fine-tuning
• Instruct — Chat-optimized for direct use in conversational applications
• Reward — Specialized for scoring and filtering synthetic training data; widely used in RLHF pipelines

The Reward model is particularly notable — it has become a standard tool for teams building synthetic data pipelines, where it scores AI-generated training examples to filter out low-quality outputs before they contaminate training sets.

Llama-3.1-Nemotron-70 Billion-Instruct

A fine-tuned version of Meta's Llama 3.1 70 billion parameter model, enhanced using NVIDIA's Nemotron reward model and RLHF techniques. At release, it outperformed GPT-4o on several benchmarks — demonstrating that fine-tuning expertise can be as important as raw model scale.

This model follows a practical pattern: take the best available open-weight base (Llama), apply superior fine-tuning techniques, and produce a derivative model that exceeds the original. It runs on a single high-end GPU (H100 or A100 with 80GB memory).

Minitron (8 Billion and 4 Billion)

Smaller models derived from Nemotron-4 15 billion through pruning and knowledge distillation — NVIDIA's research into making large models smaller without proportional quality loss. These target edge deployment and cost-sensitive inference.

Nemotron-Mini-4B-Instruct

Designed specifically for on-device and edge deployment. Pairs naturally with NVIDIA's Jetson hardware platform for embedded AI applications where cloud connectivity is unavailable or undesirable.

Nemotron Elastic 30 Billion (May 2026)

The newest and most architecturally interesting addition. Nemotron Elastic is a single 30 billion-parameter reasoning-model checkpoint that contains 30 billion, 23 billion, and 12 billion-parameter nested submodels — extractable at inference time via zero-shot slicing with no further fine-tuning required. One file ships, three model sizes deploy.

The recipe was published by NVIDIA Research with three concrete wins:

• 360-times token reduction over training the three sizes from scratch — the elastic post-training run consumed roughly 160 billion tokens versus the multi-trillion-token equivalent for three independent pretraining runs
• 18.7 gigabytes for the 30 billion checkpoint under NVFP4 quantization (NVIDIA's 4-bit floating point format) — small enough to fit on a single consumer-class GPU
• Up to 16 percent higher accuracy and 1.9-times lower latency in the 23-to-30 billion configuration compared to Nemotron Nano v3's default budget control. The accuracy-and-latency Pareto frontier moves outward across all three sizes.

The intuition behind elastic slicing: reasoning tokens are high-volume but tolerant of some capacity reduction, while the final answer requires higher precision. Nested QAD (quantization-aware distillation) preserves the slicing property even after dropping to FP8 or NVFP4, so a single quantized checkpoint serves all three sizes at all three precisions (BF16, FP8, NVFP4) — nine deployment configurations from one training run.

Practical impact: deployment teams that want to A-B test a smaller model against a larger one no longer need two separate fine-tuning runs and two model artifacts. Pull the same checkpoint, slice differently per request based on task complexity. The model card and quantized variants are on Hugging Face under nvidia/NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B.

Nemotron Diffusion 14 Billion (May 2026)

The newest variant, posted to Hugging Face in late May 2026 alongside an NVIDIA Labs technical report. Nemotron Diffusion is a 14 billion-parameter language model that switches between three decoding modes — autoregressive, parallel diffusion, and a "self-speculation" mode that drafts with diffusion and verifies with autoregression — all without changing model weights or attention patterns.

The throughput numbers from NVIDIA's published benchmarks:

• 2.2-times faster than the comparable Qwen 3 8 billion-parameter baseline at matched accuracy in self-speculation mode
• 850 tokens per second on a GB200 system — a 3.3-times lift over the baseline
• 5.9-times tokens per forward pass versus standard Qwen 3 8 billion with matching accuracy, by reusing model weights to compute multiple tokens per step

Base, instruct, and vision-language variants all ship open-weight. The architecture is positioned as a path from memory-bound to compute-bound inference as GPUs continue to outrun memory bandwidth — a structural concern that has been growing as HBM capacity per accelerator climbs slower than raw compute. The model card lives at nvidia/Nemotron-Labs-Diffusion-14B.

Access

Strengths

• Frontier open-weights flagship — Nemotron 3 Ultra (~550 billion total parameters, ~50 billion active) leads US open-weights rankings while running roughly 30 percent cheaper than leading alternatives, with a 1-million-token context window across the whole Nemotron 3 line
• Tightly optimized for NVIDIA hardware — models ship with TensorRT-LLM optimizations and NIM containers out of the box
• Reward model for synthetic data — Nemotron-4 Reward is widely used beyond NVIDIA's own models for scoring synthetic training data
• Full-stack demonstration — showcases NVIDIA's training, fine-tuning, and inference capabilities end to end
• Range of sizes — from 4 billion (edge) to 340 billion (datacenter), covering diverse deployment scenarios
• Strong fine-tuning results — Llama-Nemotron-70 billion demonstrates benchmark-leading performance through fine-tuning alone
• Elastic post-training (May 2026) — Nemotron Elastic 30 billion packs three nested model sizes in one checkpoint with zero-shot slicing, collapsing what used to be three separate training runs into one
• Tri-mode decoding (late May 2026) — Nemotron Diffusion 14 billion ships a single architecture that switches between autoregressive, diffusion, and self-speculation modes for a 2.2-times throughput lift over the comparable Qwen 3 baseline at matched accuracy

Limitations & Considerations

• Ecosystem is smaller than Llama/Gemma — fewer community fine-tunes, fewer third-party tutorials, less Stack Overflow coverage
• NVIDIA hardware advantage — models are most optimized for NVIDIA GPUs; running on AMD or other hardware loses the performance edge
• Licensing varies by model — Nemotron-4 uses NVIDIA's own license (check commercial terms); Llama-Nemotron inherits Meta's license
• Large model sizes — the 340 billion parameter flagship requires significant GPU infrastructure to run
• Less brand recognition — NVIDIA is known for hardware; developers may overlook Nemotron when choosing open models

Key Takeaways

• The Nemotron 3 generation (Nano, Super, Ultra) is NVIDIA's newest open-weight family — a hybrid Mamba-Transformer architecture with Latent MoE and a 1-million-token context; the 550 billion parameter Ultra flagship launched at Computex as the top-ranked US open-weights model, running roughly 30 percent cheaper than leading alternatives
• Nemotron is NVIDIA's open-weight model family spanning edge-scale models to datacenter-class flagships — from 4 billion parameters up to the 550 billion parameter Nemotron 3 Ultra
• The Nemotron-4 Reward model has found broad adoption for scoring synthetic training data in RLHF pipelines, extending its impact beyond direct model use
• Llama-3.1-Nemotron-70 billion demonstrates that fine-tuning expertise can produce models that outperform larger competitors on key benchmarks
• Nemotron Elastic 30 billion (May 2026) ships three nested model sizes (30, 23, 12 billion) in a single checkpoint with zero-shot slicing — a 360-times token reduction over training the sizes independently, with NVFP4 quantization shrinking the 30 billion checkpoint to under 19 gigabytes
• Nemotron Diffusion 14 billion (late May 2026) is the newest variant — a tri-mode architecture (autoregressive, diffusion, self-speculation) that delivers a 2.2-times throughput lift over the comparable Qwen 3 baseline at matched accuracy, scaling to 850 tokens per second on a GB200
• Nemotron models are most compelling when deployed on NVIDIA infrastructure (NIM + TensorRT-LLM), where tight hardware-software integration delivers maximum performance