PrismML Bonsai

Learning Objectives

• Understand what Bonsai is: a compression of someone else's model, not a model trained from scratch
• Know what ternary weights are and why they change where a 27 billion parameter model can run
• Judge the vendor's retention claim, and know when a full-precision or cloud model is still the better choice

What Is Bonsai?

Bonsai is a family of open-weight language models from PrismML, a startup founded by Caltech researchers and led by professor Babak Hassibi. Its premise is that a capable reasoning model does not have to be large in memory. PrismML takes an existing open model and compresses its weights so aggressively that it fits on a laptop, a desktop graphics card or, in the smallest variants, a phone.

The current flagship is Ternary Bonsai 2 27B, released on September 17, 2026. It is a compressed version of Alibaba's Qwen3.8 27B, and its total footprint is 5.9 gigabytes, more than nine times smaller than the full-precision model it comes from. It keeps Qwen's 262,000 token context window and accepts images as well as text.

The family started smaller. PrismML published its first Bonsai models, at 8 billion, 4 billion and 1.7 billion parameters, in March 2026, and added its first 27 billion parameter model in July 2026, with a one-bit version for recent iPhones. Bonsai 2 keeps the same footprint as that July model; the change is a stronger base model and less lost capability.

What It Can Do

PrismML reports the scores below for Ternary Bonsai 2 27B in thinking mode against the full-precision models. Every figure is vendor-reported; the full per-benchmark results are in PrismML's whitepaper.

The pattern matters more than the total. The compressed model gives up the most on knowledge and reasoning and on vision, loses little on coding and math, and slightly beats the original on instruction following. PrismML's earlier ternary 27B model kept about 95 percent of its base model's aggregate score; Bonsai 2 raises that to 98.2 percent.

On speed, PrismML reports up to 143 tokens per second on an NVIDIA GeForce RTX 5090 and about 47 tokens per second on an Apple M5 Max, and says the model is more energy-efficient per token on an RTX 4090 than a full-precision 8 billion parameter model.

Pricing

The weights are released under Apache 2.0, and so is the base model: Alibaba published Qwen3.8 27B under plain Apache 2.0 with no revenue or user thresholds, so there is no restrictive upstream license hiding underneath. That matters because the same Qwen family ships some of its other checkpoints under custom community licenses, so read the license on the exact repository you download.

Strengths

• A 27 billion parameter model in under 6 gigabytes — runs on a single consumer graphics card or a well-equipped Mac instead of a server
• Most of the capability survives — a reported 98.2 percent of the full model's aggregate score, with coding and math nearly intact
• Genuinely open terms — Apache 2.0 on the weights and on the Qwen base, with commercial use allowed
• Private by default — running locally means documents and code never leave the machine
• Fast local inference — fewer bits per weight means more tokens per second on the same hardware
• Broad hardware support — NVIDIA GPUs through CUDA and Apple Mac, iPhone and iPad through MLX, using PrismML's own low-bit kernels

Limitations and Considerations

• Benchmarks are vendor-reported — PrismML chose the suite and ran the evaluations, and no independent test had confirmed them at release
• It is a compression, not a new model — Bonsai inherits Qwen3.8 27B's knowledge, strengths and blind spots, and cannot exceed it on the whole
• The losses are uneven — knowledge and vision fall furthest, so check the tasks you care about rather than trusting the aggregate
• Not a frontier model — a 27 billion parameter model, even uncompressed, trails the large closed models on hard reasoning and long agentic work
• Custom kernels — the speed figures depend on PrismML's own low-bit kernels, so community runtimes may not match them
• A young company — PrismML raised a seed round of about 22 million dollars, so the pace of future releases depends on a small team

Best Use Cases

When to choose alternatives:

• Maximum capability → a closed frontier model from OpenAI, Anthropic or Google through its API
• The uncompressed original → Qwen3.8 27B itself, if you have the memory and want every point of accuracy
• Server-side serving at scale → a full-precision open model on a standard inference stack

Getting Started

1. Find the Bonsai models on Hugging Face under the prism-ml organization, and pick the format for your hardware — GGUF for NVIDIA and general use, MLX for Apple devices
2. Check that your machine has at least 6 gigabytes of free GPU or unified memory for the 27B model, and more for long contexts
3. Load it with a runtime that supports the format, and compare its answers with a cloud model on a few of your own tasks
4. Confirm the license on the repository you downloaded before building a product on it

Key Takeaways

• Bonsai is PrismML's family of compressed open-weight models; the flagship Ternary Bonsai 2 27B (September 17, 2026) is Alibaba's Qwen3.8 27B shrunk to 5.9 gigabytes
• It stores each weight as minus one, zero or one, about 1.76 bits per weight, which is what lets a 27 billion parameter model run on a laptop or a single graphics card
• PrismML reports it keeps 98.2 percent of the full model's aggregate benchmark score, up from about 95 percent for its earlier ternary 27B model — a vendor-reported figure
• Both the Bonsai weights and the Qwen base are under Apache 2.0, so commercial use is permitted
• It is a compression, not a new frontier model: it cannot beat the model it came from, and knowledge and vision lose the most