PrismML's Bonsai 27B Squeezes a 27-Billion-Parameter Model Onto Your Phone — at 3.9GB, Fully Offline

The race to put a genuinely capable AI model on a phone just cleared a milestone. PrismML has released Bonsai 27B, a 27-billion-parameter multimodal model compressed so aggressively that its smallest build is just 3.9GB — small enough to run entirely on-device, offline, on a modern smartphone. The model is free to download under a permissive Apache 2.0 license on Hugging Face, and it lands as one of the first 27B-class systems to run on hardware you carry in your pocket.

The trick is extreme low-bit quantization. Bonsai 27B is built on Alibaba's Qwen3.6-27B (27.8 billion parameters) and ships in two compressed forms: a 1-bit binary build at roughly 1.125 bits per weight that fits in 3.9GB — about 14× smaller than the full-precision original — and a 1.58-bit ternary build at 5.9GB, around 9× smaller. For reference, the same model at standard 16-bit precision would need on the order of 54GB, which is why running it locally has been a non-starter until now.

What makes those numbers hold up is how the model was shrunk. Rather than training at full precision and rounding down afterward — the usual post-training quantization that "bleeds quality" at very low bit-widths — PrismML trained Bonsai as a native low-bit architecture from the ground up, in the style of BitNet-style quantization-aware training. The model learns with binary or ternary constraints baked into the forward pass while keeping full-precision gradients in the backward pass, so its internal representations are optimized for that tiny numeric space from the start. The payoff: PrismML says the 1-bit build keeps more than 90% of full-precision benchmark performance, and the ternary build more than 95%.

It is also more than a text toy. Bonsai 27B is multimodal — it accepts images alongside text, so it can read screenshots, documents, and camera input — and it's tuned for tool use, planning, coding, and agentic execution rather than just chat. It carries a full 262K-token context window, and PrismML pairs it with speculative decoding for a claimed 1.37× decode speedup on the CUDA serving path. On a phone, the company reports around 11 tokens per second on an iPhone 17 Pro for the 1-bit build — slow next to a data center, but genuinely usable for an offline assistant that never sends your data anywhere.

The strategic significance is bigger than the file size. On-device inference means no per-token API bill, no network dependency, and no user data leaving the handset — an answer to both the cost and privacy problems that hang over cloud AI. If a 27B multimodal model with real reasoning and vision can live entirely on a phone, the default location of everyday AI starts to shift from someone else's servers back to the device in your hand. PrismML, whose stated mission is simply "concentrating intelligence," is betting that the next frontier isn't only bigger models in the cloud — it's smarter compression that brings the good ones home.