ARM AI Processors

Learning Objectives

• Understand ARM's architecture-licensing model and why it dominates mobile + edge AI
• Identify the Neoverse data-center platform's traction and the new AGI CPU's role
• Evaluate when ARM-based AI compute makes sense vs. x86 or specialized accelerators

What Is ARM AI?

ARM (majority-owned by SoftBank since 2016) is a UK chip-architecture company — but unlike Intel or AMD, ARM does not manufacture chips. Instead, it licenses CPU architectures (Cortex, Neoverse) and core IP that other companies build their own chips around. Apple's M-series, Amazon's Graviton, NVIDIA's Grace, Google's Tensor, Qualcomm Snapdragon, MediaTek Dimensity, and the SoCs in every iPhone and most Android phones are all built on ARM architecture.

For AI specifically, ARM has two strategic positions: Cortex dominates mobile and edge AI inference (where most consumer-facing AI compute happens), and Neoverse has aggressively expanded into AI server CPUs — by 2025 reaching nearly 50% market share of CPU shipments among the top hyperscalers. In March 2026, ARM took the next step by announcing AGI, its first in-house designed data-center processor.

Pricing & Access

ARM does not sell chips. ARM sells architecture licenses to chipmakers and system designers, who then design and fabricate their own silicon (typically at TSMC or Samsung).

The new AGI CPU is a meaningful strategy shift — for the first time, ARM is selling its own chip rather than just licensing the design.

Core Architecture Components

Neoverse Data Center Platform

ARM's server-CPU platform — Neoverse V (max performance) and N (efficiency) cores. Used by AWS Graviton 4, Microsoft Cobalt 100, Google Axion, Oracle Ampere, and NVIDIA Grace. By 2025, Neoverse-based chips reached billion+ deployed cores and approximately 50% market share of CPU shipments to top hyperscalers — a stunning shift from x86 dominance just five years prior.

Cortex Mobile + Edge

The dominant CPU architecture for mobile AI inference globally. Apple A-series and M-series, Qualcomm Snapdragon, MediaTek Dimensity, and Samsung Exynos all build on Cortex cores. Most everyday AI inference (speech recognition, image processing, on-device LLM) runs on Cortex-based silicon.

AGI CPU (March 2026)

ARM's first in-house data-center processor, announced March 24, 2026. Specs: up to 136 Neoverse V3 cores, built on TSMC 3nm, designed specifically for AI inference orchestration and agentic computing workloads. Lead customer is Meta (using AGI to power AI search ranking and recommendation systems). OpenAI, Cerebras, and Cloudflare also announced as launch partners. Production silicon is ready to order; volume shipments by end of 2026.

Mali G2-Ultra NX (September 2026)

ARM's first AI-native mobile GPU, announced September 8, 2026. The change is architectural rather than incremental: neural accelerators sit inside the graphics pipeline rather than alongside it, so the AI work happens where the frame is being drawn instead of in a separate block. Three techniques ship together — reconstructing a high-resolution frame from a cheaper render, generating intermediate frames to reach 120 frames per second, and combining upscaling with denoising for ray-traced scenes.

ARM reports up to 24 percent higher benchmark performance than the previous generation and 14 percent on non-AI gaming, with a demo showing roughly 70 percent less external memory traffic. Tencent Games, Unity China and the studio behind Where Winds Meet are integrating it. Every figure here is ARM's own, and no device launch date has been given.

This matters beyond gaming. Memory bandwidth, not raw compute, is the binding constraint on phone-class AI, so a pipeline that produces more pixels from less data movement is the same lever that makes on-device language models practical.

NVIDIA NVLink Fusion Integration

Neoverse platform integrates NVIDIA's NVLink Fusion interconnect — letting ARM-based head nodes pair efficiently with NVIDIA GPU clusters in AI training and inference deployments.

SoftBank Vision Fund Synergy

SoftBank also owns Ampere Computing, another ARM server-chip designer, and has an extensive Vision Fund portfolio of AI startups — making the ARM/SoftBank ecosystem strategically connected to the broader AI investment landscape.

Strengths

• Architecture ubiquity: Almost every mobile device and increasing share of data centers run ARM — ARM royalties scale with the entire computing market
• Power efficiency: ARM's RISC heritage means lower power-per-watt vs. x86 for many workloads — critical for AI inference at scale and for mobile/edge AI
• Hyperscaler tailwind: AWS, Microsoft, Google, Oracle all designing custom ARM-based server CPUs — ARM's success compounds with hyperscaler AI capex
• AGI CPU launch (2026): First-party data-center chip lets ARM capture more value per deployment than pure architecture licensing
• NVIDIA partnership: Neoverse-NVLink Fusion integration positions ARM as the head-node CPU for NVIDIA AI clusters

Limitations & Considerations

• Not a chipmaker: ARM ships architecture, not finished chips — execution depends on partner foundries (TSMC, Samsung) and customer silicon teams
• Geopolitical sensitivity: UK-based, SoftBank-owned, with Chinese subsidiary (ARM China) operating semi-independently — geopolitical risk varies by jurisdiction
• AI accelerator gap: ARM is a CPU architecture; for actual AI training acceleration, NVIDIA GPUs / TPUs / Gaudi remain the workload — ARM is the orchestrator, not the accelerator
• Royalty rate sensitivity: Single-digit-cents-per-chip royalties mean ARM revenue is highly sensitive to royalty-rate negotiations with Apple, Qualcomm, and other big customers
• AGI is new: AGI CPU production silicon is fresh as of March 2026 — real-world performance and TCO data still emerging through 2026

Best Use Cases

When to choose alternatives:

• AI training (large model) → NVIDIA H100/H200/B200 or Intel Gaudi 3 GPUs/accelerators
• Frontier inference at the largest scale → NVIDIA Grace Hopper (still ARM-based, but the GPU does the work)
• Pure-x86 compatibility required → Intel Xeon or AMD EPYC for legacy software stacks

Key Takeaways

• ARM is a chip architecture company, not a manufacturer — it licenses CPU IP that powers most mobile AI inference globally and increasingly AI server CPUs
• Neoverse server platform has captured approximately 50 percent of CPU shipments to top hyperscalers by 2025, deployed across AWS Graviton, Azure Cobalt, Google Axion, Oracle Ampere, and NVIDIA Grace
• AGI CPU (announced March 2026) is ARM's first in-house data-center processor — 136 Neoverse V3 cores on TSMC 3nm, with Meta as lead customer and OpenAI/Cerebras/Cloudflare as launch partners
• The Mali G2-Ultra NX (September 2026) is ARM's first AI-native mobile GPU, putting neural accelerators inside the graphics pipeline rather than beside it; ARM reports up to 24 percent higher benchmark performance and roughly 70 percent less external memory traffic in its own demo, which is the constraint that matters most for phone-class AI
• ARM is the AI orchestrator, not the AI accelerator — pair with NVIDIA GPUs / TPUs / Gaudi for actual training workloads
• SoftBank ownership ties ARM strategically to the broader AI investment landscape, including SoftBank's Ampere Computing and the Vision Fund AI portfolio