Bonsai 27B fits on a phone. But does it run well?

On July 14, PrismML released Bonsai 27B, a compressed version of Qwen3.6 27B available under the Apache 2.0 license. The binary build occupies 3.9 GB and fits an iPhone 17 Pro Max. The ternary version is 5.9 GB in theory but about 7.2 GB in its deployed GGUF language-model pack, making it a laptop and GPU option.

But fitting the weights into memory is only the first part of running a model on a phone. The early Hacker News discussion and r/LocalLLaMA thread quickly moved to the less glamorous questions: runtime support, low-bit kernels, inference speed, and growing context.

How a 27B model gets down to 3.9 GB

The binary model uses -1 or +1 weights with group-wise scaling, for an effective 1.125 bits per weight. The ternary model adds zero and has an ideal rate of 1.71 bits. Current kernels use a 2-bit slot for each ternary value, bringing the deployed rate to 2.125 bits and its resident GGUF footprint to roughly 7.2 GB.

PrismML estimates that the original model needs roughly 54 GB at 16-bit precision and around 18 GB in a conventional 4-bit build. At 3.9 GB, the binary release leaves some of the phone's usable allocation for the runtime, activations, and KV cache.

In PrismML's own 15-benchmark evaluation, the company says the ternary version retains 95% of the full-precision model's overall score, while the binary version retains about 90%. Math and coding hold up best. On the binary model card, instruction following falls from 78.47 to 65.74, vision from 72.61 to 59.57, and agentic tool calling from 80.00 to 66.03. Tool calling has the largest absolute drop, but the other two categories also lose roughly 13 points.

These are the company's benchmarks, so independent testing will matter.

Fitting in memory is not the same as running well

Low-bit weights do not automatically produce efficient inference. The runtime needs kernels that operate on the packed representation. If it first unpacks the weights into a wider format, the model may take less disk space without delivering the expected speed.

That distinction is already visible in the Reddit discussion. One tester reported good throughput with the ternary model on an Nvidia desktop GPU. Another found a 1-bit GGUF build unexpectedly slow in llama.cpp, raising the question of whether it was using a dedicated low-bit path or paying an unpacking cost.

PrismML highlights Apple support through MLX and Nvidia support through CUDA, using custom low-bit kernels. The experience may therefore differ sharply across an iPhone app, an M-series Mac, llama.cpp, and a generic GGUF frontend.

The phone demonstration needs similar care. PrismML labels its iPhone 17 Pro Max multimodal demo as using cached and prefilled image context, which narrows what that particular demo proves. Its model card provides a more useful text-inference measurement: 11 tokens per second on an iPhone 17 Pro Max, settling to 10.8 during a continuous battery-drain run. These are still vendor measurements. Independent tests with uncached prompts, growing context, and memory pressure would tell us more about the experience inside a real app.

The useful comparison is not 27B against itself

PrismML mainly compares Bonsai with full-precision and compressed versions of the same model. That measures retained capability, but it is not quite the decision facing a developer.

At roughly the same memory budget, the alternative may be a smaller 7B-to-9B model using a mature 4-bit runtime. It may start faster, support more frontends, leave more space for context, and run more consistently on the phone. The useful test is whether Bonsai 27B completes real tasks better within the same memory, latency, and energy limits.

Tool calling is especially relevant because PrismML positions the model for local agentic workflows. The launch post says it stays "within a few points of full precision." That roughly describes the ternary model at 74.01 against an 80.00 baseline, although six points is already doing some work. It does not describe the phone-sized binary build at 66.03. A local agent that produces good text but becomes less reliable across repeated structured calls may not benefit much from the larger parameter count.

Bonsai 27B still moves an important boundary. A 27B-class model in a phone-sized package was not a normal deployment option before this release. The next proof is less dramatic: a repeatable mobile setup, uncached workloads, and comparisons against smaller models that developers could already run well.