A strange thing happened to local AI models this year. Every few months, someone released a "2-bit" build of a good open model, and every few months, people who actually stress-tested those builds found the same thing: the benchmark table looked fine, but the model fell apart the moment you asked it to reason through a hard problem for more than a few steps.
On September 17, 2026, a company called PrismML released Bonsai 2 27B, and the claim is different this time. Not "a smaller model that runs on your laptop." That was July's release, and it was genuinely novel but had a real gap. The new claim is "near-lossless": 98.2% of the full-precision model's benchmark average, at 5.9GB instead of 54GB.
If that claim holds, it changes the math for anyone paying a monthly subscription for AI assistance. So let's look at what the numbers actually say, where independent testers found problems, and how you can run it yourself in about ten minutes.
A quick disclosure before we start: I have not run this model on my own hardware. Everything in this article comes from PrismML's model card and whitepaper, their press release, and public testing by community members, all linked inline. Treat the vendor numbers accordingly.
The one-sentence version
PrismML took Alibaba's open Qwen3.8-27B, replaced almost every weight with one of three values (-1, 0, or +1), and shipped it under Apache 2.0. The result scores 84.78 across 14 thinking-mode benchmarks where the original FP16 model scores 86.32. The file is 5.9GB instead of 54GB.
That is roughly 9 times smaller with less than 2% of the benchmark score lost. For comparison, a conventional "2-bit" quantization of the same model (IQ2_XXS) lands at 72.59, more than 12 points behind, at 9.4GB. The compressed model is both smaller and smarter than the conventional approach.
Why ternary weights are different from "2-bit quantization"
Here is the part worth understanding, because it explains why this release matters more than the usual quantization churn.
Conventional quantization squeezes model weights into fewer bits. The problem is that below 4 bits, the standard methods don't degrade gracefully. They collapse selectively: the model still answers trivia and chit-chat fine, which is what most casual testing checks, but sustained multi-step reasoning falls off a cliff.
PrismML's model card gives the cleanest example of this. The conventional 2-bit build of Qwen3.8-27B scores 88.93 on MMLU-Redux, a knowledge benchmark, which looks respectable. On AIME26, a competition math benchmark, it scores 57.5. On LiveCodeBench, it scores 56.4. Same model, same "2-bit" label. If you tested it with a few chat questions, you would never notice.
Bonsai 2 holds exactly the benchmarks where conventional low-bit methods collapse: 95.83 on AIME26 (the full-precision model scores 94.58, so it actually edges it out) and 90.07 on LiveCodeBench (baseline: 90.05, statistically level). Math as a category drops only from 97.06 to 96.57, and coding actually comes in level with the baseline at 89.42 versus 89.07.
The mechanism, per the whitepaper: each weight is one of three values, with one shared FP16 scale factor per group of 128 weights. That works out to 1.72 bits per weight. The weights are also stored after a Hadamard rotation, a mathematical transform that spreads outliers evenly so no single weight carries information that ternary values cannot represent. The vision tower stays unquantized FP16 in the MLX build, or ships as an optional 0.63GB add-on for the GGUF build, because images degrade faster than language under extreme compression.
The previous Bonsai release (July, based on Qwen3.6-27B) retained 94.6% of baseline. This one retains 98.2% on the same kind of evaluation. That 3.6-point jump is the actual news.
The speed and hardware numbers
This is where it gets practical, because the whole point is running on hardware you already own:
- RTX 5090 (32GB): about 130 tokens per second generating, with prompt processing near 3,900 tokens per second on the PQ2_0 packing.
- Apple M5 Max laptop: about 47 tokens per second.
- Apple M5 Pro laptop: about 28 tokens per second, drawing 27.5W on the GPU rail. For reference, the NVIDIA cards in the same table draw 300 to 455W of board power.
- Entry-level L4 (24GB, 72W): about 32 tokens per second.
That M5 Pro number matters more than it looks. The FP16 baseline of this model is 54GB, which does not fit on any consumer laptop at all. The meaningful statement is not "28 tokens per second is fast," it is that a 27B-class reasoning model runs interactively on an everyday laptop, which was impossible two months ago.
Context window is 262K tokens, inherited from the base model. The architecture is about 75% linear attention, which is what keeps long context affordable on-device. PrismML measured the ternary build holding a 100K-token context at 14.7GB total memory, versus about 25.6GB for a conventional 4-bit build of the same model.
Where it does not hold up
Now the part the launch posts leave out.
The score drops are not evenly distributed. Math and coding are near-parity. But knowledge and reasoning fall from 85.55 to 79.86, vision falls from 71.36 to 66.19, and agentic tool calling drops from 76.74 to 74.92. OCR Bench v2 is the worst single number: 56.88 against a baseline of 60.99. If your use case is "read this screenshot and extract text," this model is not the one.
The community testing of the July generation is also worth weighing, even though it tested the older model, not Bonsai 2. A well-documented run on r/LocalLLaMA put the first Ternary Bonsai 27B through Terminal-Bench 2.0, an agentic coding benchmark, on an 8GB laptop GPU. It scored 7.9%, below a 9B model at conventional 4-bit quantization that fit on the same card. The 1-bit variant could not finish a task at all, rambling past 14,000 tokens without emitting a stop. To their credit, PrismML said upfront that agentic coding was not a strong target of that release and put an agentic-tuned variant on the roadmap. The 74.92 BFCL v3 tool-calling score of Bonsai 2 suggests improvement, but nobody has published independent agentic-harness numbers for Bonsai 2 yet as far as I can find. If your plan is "wire this into an agent framework and let it run," wait for independent results.
One more honest caveat: every headline number here is PrismML measuring PrismML. The methodology looks sound (EvalScope plus vLLM on H100, identical infrastructure across all variants, full per-benchmark tables published), and the whitepaper is public. But these are still vendor-reported numbers until independent replications land. Watch the PrismML-Eng/Bonsai-demo repository and the r/LocalLLaMA threads for those.
How to run it yourself
Here is the part most writeups skip, and the single biggest gotcha: stock llama.cpp will not run these files. The ternary kernels live in PrismML's llama.cpp fork. Worse, stock llama.cpp loads the Q2_0 variant without warning and produces garbage, because it has no Hadamard activation runtime. Use the fork's prebuilt binaries.
The steps, from the model card:
Step 1: Get the fork. Grab a prebuilt binary from the PrismML-Eng/llama.cpp releases page for your platform, or build it:
git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j
# On a Mac, drop -DGGML_CUDA=ON; Metal is the default backend
Step 2: Download the model. The GGUF repo has two packings. PTQ1_0 is 5.95GB and packs the ternary values densely. PQ2_0 is 7.21GB and stores each value in a 2-bit slot:
hf download prism-ml/Ternary-Bonsai-2-27B-gguf \
Ternary-Bonsai-2-27B-PQ2_0.gguf --local-dir .
Step 3: Run it.
./bin/llama-cli -m Ternary-Bonsai-2-27B-PQ2_0.gguf \
-ngl 99 -fa on -c 32768 \
--temp 1.0 --top-p 0.95 --top-k 20 \
-p "Explain quantum computing in simple terms." -n 256
-ngl 99 offloads every layer to GPU; use 0 for CPU-only. Context goes up to 262,144 tokens with the -c flag if your memory allows.
Two practical notes from the model card. First, which packing is faster depends on your GPU: PQ2_0 wins on H100, A100, and Blackwell cards plus all prompt processing, while PTQ1_0 wins on Ada-generation cards and anywhere memory is tight. Second, this is a reasoning model that thinks by default at "xhigh" effort. If responses feel slow, drop the reasoning effort to medium for shorter answers, and use the instruct-mode sampling settings (temperature 0.7, top_p 0.80) when you turn thinking off.
Apple Silicon users have a native MLX build (prism-ml/Ternary-Bonsai-2-27B-mlx-2bit) at 8.6GB including the vision tower, measuring around 28 tokens per second on an M5 Pro in the model card's Metal numbers.
So: cancel the subscription?
Here is the decision checklist, honestly framed:
- Run it locally if: your work is math, coding help, document analysis over long context, or anything privacy-sensitive where sending data to an API is a dealbreaker. 98.2% of a strong 27B model, free, offline, at 28 to 130 tokens per second on hardware you own, is a legitimately new option.
- Stay on the subscription if: you need frontier-model quality, heavy agentic multi-step work (unproven on this model so far), strong vision/OCR, or zero friction. A 27B model at 1.72 bits per weight is remarkable engineering, but it is not Claude or GPT-class, and PrismML does not claim it is.
- The realistic middle path: use Bonsai 2 for the high-volume, private, and offline work, and keep the subscription for the tasks that genuinely need a frontier model. This is the pattern that usually wins, and the pattern the "local AI will replace everything" and "local AI is a toy" camps both miss.
The most interesting number in the whole release is not on any benchmark. It is intelligence density: benchmark score per gigabyte. Bonsai 2 lands at 0.469. The conventional 2-bit build manages 0.199. FP16, 0.053. Even if you never run this model, the direction is clear: the gigabytes-per-IQ exchange rate is improving fast, and your laptop is the beneficiary.
I write about AI infrastructure and developer tools every week. Subscribe, it's free, and it tells me this kind of deep-dive is worth making.
What is your setup for local models? Running anything on a laptop GPU, or is it still all cloud for you? Tell me in the comments.
Sources: PrismML announcement, GGUF model card, MLX model card, press release, community Terminal-Bench testing of the July release, and the Bonsai whitepaper.
