Bonsai 2 27B: Run a 27B AI Locally on Your Laptop

Most of the model launches I've covered this year chase the same goal: a bigger context window, a higher SWE-bench score, another trillion parameters.

PrismML is going the other direction.

On September 17, 2026, the team released Ternary Bonsai 2 27B, a compressed version of Qwen3.8 27B that fits in 5.9GB and runs on a laptop.

By intensely compressing how it stores information, PrismML made Bonsai 2 27B over 9 times smaller than the original Qwen version. Yet, it still retains 98.2% of its original performance and the 262k token context window, processing text and images.

In this article, I'll cover everything new with Bonsai 2 27B, looking at the new features, exploring the benchmarks, and breaking down pricing and access.

If you want to run the first Bonsai model yourself, we have a step-by-step walkthrough in our Bonsai 27B local setup tutorial.

TL;DR

• Bonsai 2 27B compresses Qwen3.8 27B to 5.9GB while keeping 98.2% of its aggregate benchmark performance.
• The real story is where capability is retained: math and coding hold at parity, while knowledge and vision take the biggest hit.
• It runs locally on NVIDIA GPUs via CUDA and Apple devices via MLX, reaching 143 tokens/second on an RTX 5090.
• It's Apache 2.0 licensed with weights on Hugging Face, and third-party providers currently host it for free.
• Worth it if you need on-device coding agents or private document work; look elsewhere if you rely on fine-grained knowledge recall or high-stakes OCR.

What Is Bonsai 2 27B?

Bonsai 2 27B is PrismML's flagship compressed multimodal model, built by applying ternary quantization end-to-end across the language model of Qwen3.8 27B.

Where the full-precision Qwen3.8 27B occupies 54GB in FP16 (the uncompressed, full-size version of the model), Bonsai 2 27B lands at 5.9GB for text only, a footprint small enough to sit on a consumer GPU or a MacBook.

According to PrismML, the compression uses ternary weights limited to −1, 0, and +1, paired with FP16 group-wise scaling. That works out to 1.72 true bits per weight on PrismML's own accounting, versus the 16 bits of the FP16 reference.

The model keeps the 262K-token context window and multimodal text-and-image input of its base.

The headline result is the retention number.

Across a suite PrismML describes as 14 benchmarks in six skill categories, evaluated in thinking mode with EvalScope and vLLM on an H100, Bonsai 2 27B averages 84.78 against the FP16 baseline's 86.32, or 98.2%.

For a sub-2-bit model, that is a much smaller gap than conventional quantization delivers.

Bonsai 2 27B Key Features

What you can do with Bonsai 2 27B comes down to running 27B-class capability in places a 27B model normally can't go.

Here are the features that define it.

Run a 27B multimodal model on a laptop

Bonsai 2 27B fits in 5.9GB, which puts a 27B-class model within reach of consumer hardware instead of a datacenter.

The low-bit representation is applied across the whole language model, not just a few layers, so the footprint reduction is real rather than cosmetic.

For practitioners, this changes what "local" means.

Instead of dropping down to an 8B model for on-device work, you can run something with the reasoning and coding ability of a 27B model and still have RAM left over.

The catch is that it is still a 27B model architecturally, so very low-end hardware will feel the memory pressure.

262K-token context for long-document work

The model carries a 262K-token context window, which is enough to hold a full codebase, a long legal contract, or a stack of research PDFs in a single prompt.

That matters most for retrieval-augmented generation setups where you want to keep large amounts of context resident rather than chunking aggressively.

Community reports flag one caveat here: some users note recall and coherence degradation at the extreme end of the window, past roughly 200K tokens.

If you are pushing near the limit, test recall on your own data before trusting it in production.

Coding agents and tool calling

Bonsai 2 27B is built to hold up in agentic loops, where small errors compound over many steps.

PrismML demonstrated its ability to drive coding agents through Cline and run computer-use workflows, both on an RTX 5090.

This is the area where compression usually falls apart, because a single degraded step can derail an entire agent run.

Bonsai 2 27B scores 90.07 on LiveCodeBench and 74.92 on BFCL v3, close to the FP16 numbers, which is what makes local agent work plausible here rather than aspirational.

Multimodal text-and-image input

The model accepts images alongside text, covering visual question answering, document understanding, and OCR.

This opens up private multimodal workflows like debugging from a screenshot or analyzing a scanned document without sending it to the cloud.

Vision is also where the compression cost shows most clearly.

On MMMU-Pro, Bonsai 2 27B scores 75.49 against the FP16 model's 81.73, and on OCR Bench v2 it drops to 56.88 against 60.99.

For high-stakes document processing, complex diagrams, or low-resolution text, I would treat vision as the weakest link and verify results.

How Does Bonsai 2 27B Perform on the Benchmarks?

The aggregate story is 84.78 versus 86.32 for FP16, but the aggregate understates what actually matters.

Conventional sub-4-bit builds collapse selectively on exactly the benchmarks that demand sustained reasoning, and Bonsai 2 27B is designed to hold those.

The comparison PrismML draws is against a conventional 2-bit build of the same base model, Qwen3.8-27B IQ2_XXS, which scores 72.59 aggregate.

Bonsai 2 27B outscores it by more than 12 points at less than two-thirds the size, and comes within 0.4 points of a much larger 4-bit build (UD-Q4_K_XL at 85.18) while being a third of its footprint.

Reasoning and math

Math is where Bonsai 2 27B is nearly indistinguishable from the full-precision model.

It scores 95.83 on AIME26 and 98.80 on MATH-500, against FP16 scores of 94.58 and 99.80.

The contrast with conventional quantization is stark: IQ2_XXS falls to 57.50 on AIME26, a 37-point collapse that casual testing on easy questions would completely miss.

On broader knowledge and reasoning, the picture is softer.

Bonsai 2 27B scores 89.09 on MMLU-Redux against FP16's 91.46, and third-party reviews put the composite knowledge-and-reasoning category around 79.9 versus 85.6 for FP16.

Fine-grained factual recall is where ternary compression costs you most.

Coding and agentic workflows

Coding holds at parity, and in places slightly exceeds the base model. Bonsai 2 27B scores 95.12 on HumanEval+ and 90.07 on LiveCodeBench, against FP16 scores of 93.29 and 90.05.

The composite coding average at 89.42 versus 89.07 for FP16 is a small win for the compressed model.

On agentic tool calling, BFCL v3 sits at 74.92 against 76.74 for FP16, a modest gap.

For comparison, when we reviewed Qwen3.8-Max, the far larger 2.4T model scored 86.1 on OSWorld-Verified for agentic computer use, so Bonsai 2 27B is not competing with cloud flagships on raw agent capability.

It is competing on running the agent locally at all.

Instruction following

Instruction following is one of the few categories where Bonsai 2 27B edges out the full-precision model outright.

It scores 91.31 on IFEval versus 91.50 for FP16, and 74.00 on IFBench prompt-loose against FP16's 71.00.

Throughput and energy efficiency

Speed is part of the benchmark story here because it determines whether local agent loops are usable.

Bonsai 2 27B reaches up to 143 tokens/second on an NVIDIA GeForce RTX 5090 and 46.8 tokens/second on an M5 Max.

On energy, PrismML reports 0.581 mWh per token on an RTX 4090, which it claims is 40% more efficient than an 8B model running at full precision.

For coding assistants, higher throughput means faster edit-debug loops; for background assistants on a laptop, lower energy per token means longer battery life without constantly calling the cloud.

Bonsai 2 27B Pricing and Availability

Bonsai 2 27B is released under the Apache 2.0 license, so the weights are free to download and use commercially.

PrismML positions it as a self-hosted model first, and its official docs focus on local serving rather than a per-token API.

The weights are available today on Hugging Face in the Bonsai 2 collection.

GGUF builds land around 5.93 to 5.95GB for the ternary PTQ1_0 variant, with an MLX build that runs larger. Note that Ternary Bonsai 2 is 27B only, with no 8B, 4B, or 1.7B companions in this generation.

How to Access to Bonsai 2 27B

Bonsai 2 27B is open-weight under Apache 2.0, so you run it yourself rather than calling a proprietary API.

It runs on NVIDIA GPUs via CUDA and on Apple devices (Mac, iPhone, iPad) via MLX, using PrismML's custom low-bit kernels.

The main routes are:

• Self-host with PrismML's llama.cpp fork: pull the GGUF weights from the Hugging Face collection and serve them; the default local API base is http://localhost:8080/v1.
• Self-host on Apple Silicon with MLX: serve the MLX build with a local API base of http://localhost:8081/v1, keeping in mind older Macs may hit RAM limits since the MLX variant is larger than the GGUF.
• Hosted inference: reach it through Together AI as Prism-ML/Ternary-Bonsai-27B or through Puter, both currently free.

PrismML exposes an OpenAI-style REST interface, so it slots into existing tooling that expects a chat-completions endpoint.

For a full local walkthrough of the first model, see our general Bonsai local setup guide.

Final Thoughts

Bonsai 2 27B is a bet that the interesting frontier is intelligence per gigabyte, not raw capability.

Closing the retention gap from the first generation's 95% to over 98% is what makes the "near-lossless" framing credible rather than marketing, and the density chart puts it well ahead of conventional low-bit builds of the same base.

I would reach for it if you are building on-device coding agents, private document analysis, or hybrid systems that keep sensitive work local.

I would be more cautious for knowledge-heavy QA and high-stakes OCR, where the compression cost is real and measurable.