Bonsai 27B Guide: How a 27B Model Runs Locally

What Just Shipped

On July 14, 2026, PrismML — a startup founded by Caltech researchers with backing from Khosla Ventures and continuing support from Samsung — released Bonsai 27B, built on top of Qwen 3.6 27B. It's the third release in their Bonsai family, following an 8B model in March that first proved 1-bit compression could work at meaningful scale.

Visit :- Prismml News

The core technique is aggressive weight compression. Instead of the standard 16-bit precision most models use, Bonsai compresses every weight down to either:

• Ternary (3 values: -1, 0, +1) — the higher-quality variant, landing at 5.9GB

• Binary (2 values: -1, +1) — the smallest variant, landing at 3.9GB, small enough for a phone

For comparison: a 27B model at standard 16-bit precision needs roughly 54GB of memory. Even a typical "compressed" 4-bit version needs about 18GB — still too large for most phones and many laptops. Bonsai's binary variant is roughly 14 times smaller than the original.

What makes this notably different from typical quantization: PrismML compressed the entire model end-to-end — embeddings, attention layers, and the output head — with no higher-precision "escape hatches" for sensitive layers. Most compressed models keep certain layers at higher precision as a quality safety net, which increases file size. Bonsai skips that trade-off entirely.

Both variants are multimodal (they can process images, not just text), support a 262K-token context window, and are released free under the Apache 2.0 license — meaning you can use them commercially with no licensing fees.

The Benchmarks — How Much Capability Do You Actually Keep?

This is the number that matters most: does compressing a model this aggressively wreck its actual usefulness? Based on PrismML's own 15-benchmark suite (independently corroborated by Decrypt's hands-on testing), the answer is largely no.

What this means practically:

Math and coding hold up remarkably well — you're not losing much capability there even in the smallest 1-bit variant. Where you feel the compression most is agentic/tool-calling and instruction following, where the gap widens to 6-14 points. If your use case is heavy multi-step tool orchestration, expect the local model to need more hand-holding than a frontier cloud model would.

Speed: On an NVIDIA RTX 5090, Bonsai hits up to 163 tok/s (1-bit) and 134 tok/s (ternary). On an Apple M5 Max, it reaches 87 tok/s (1-bit) and 58 tok/s (ternary). On an actual iPhone 17 Pro Max, Decrypt's independent testing measured roughly 11 tokens per second — usable for real conversations, noticeably slower than a cloud API response, but running with zero internet connection and zero per-token cost.

Step-by-Step: How to Run Bonsai 27B

Option 1 — On iPhone (1-bit variant)

The easiest path for iPhone is PrismML's companion app rather than manual setup:

1. Download "Locally AI" from the App Store — this is the iOS app PrismML has listed as their official on-device runner

2. Inside the app, search for and download the 1-bit Bonsai 27B model (the file is roughly 3.9GB, so use Wi-Fi)

3. Requires an iPhone with sufficient RAM headroom — PrismML built this specifically to clear the memory budget of an iPhone 17 Pro class device, since phones typically only expose 40-50% of total RAM to any single app

4. Once downloaded, the model runs fully offline — no account, no internet required after setup

Option 2 — On Mac (Ternary variant, via MLX)

Mac users get the higher-quality ternary variant, which runs natively through Apple's MLX framework:

1. Install MLX if you don't already have it (pip install mlx in Terminal, requires Python)

2. Pull the model weights directly from Hugging Face: prism-ml/Ternary-Bonsai-27B (search PrismML's Hugging Face collection)

3. Bonsai runs natively on Apple Silicon (M-series chips) — no additional conversion needed since PrismML ships MLX-ready weights

4. Expect roughly 58 tok/s on an M5 Max; lower-end M-series chips will run slower but functional

Visit Source

Option 3 — On Windows / NVIDIA GPU (via CUDA)

For developers wanting maximum speed or agentic workflow testing:

1. PrismML provides custom low-bit CUDA kernels built specifically for Bonsai's hybrid-attention architecture — standard llama.cpp-style loaders won't get you the full speed benefit

2. Pull weights from their GitHub repo (PrismML-Eng/Bonsai-demo) which includes setup scripts

3. An RTX 5090 is the reference hardware PrismML benchmarked against (163 tok/s), but the model will run on more modest GPUs at reduced speed

4. This path is best if you're planning to wire Bonsai into an actual agentic pipeline or tool-calling workflow, since GPU serving gives you the most headroom

Visit :- Github

Option 4 — Quick Test Without Installing Anything

If you just want to try it before committing to a full local setup, PrismML is currently offering a free, limited-time developer preview API — check their docs at docs.prismml.com for access. There's also a browser-based WebGPU demo on Hugging Face Spaces that runs directly in-browser with no download at all, useful for a 5-minute sanity check before deciding whether to invest in the full local setup.

Local vs. Cloud — When Does This Actually Make Sense?

Being honest about the trade-off matters here, since "free and private" doesn't automatically mean "right for every use case."

Use Bonsai 27B (local) when:

• You're prototyping agentic workflows and want to iterate without burning API credits on every test loop

• You're working with private or sensitive data that shouldn't leave the device — local execution means zero network exposure

• You want an assistant that keeps working with no internet connection

• You're building a hundred-step agent loop where cloud API costs would compound quickly — local inference has zero marginal cost per call

• You're testing tool-calling or MCP integration patterns before committing to a production cloud setup

Stick with cloud models (Sonnet 5, Opus 4.8, GPT-5.6, etc.) when:

• You need the absolute highest reasoning ceiling — the compression gap in agentic/tool-calling tasks (6-14 points) is real and matters for production-grade autonomous systems

• Your product needs guaranteed uptime and support SLAs that a self-hosted model can't offer

• You're not deploying to a device with enough memory headroom — the ternary variant still needs a reasonably modern Mac or GPU

The hybrid approach PrismML themselves suggest is architecturally interesting for builders: route routine, privacy-sensitive, or high-volume steps to a local model like Bonsai, and reserve your cloud frontier model calls for the genuinely hard reasoning steps. This collapses the cost-per-task of an agentic system significantly, since most steps in a real agent loop are routine, not frontier-difficulty.

The Bigger Picture

Two developments make this release worth watching beyond the immediate "cool, I can run AI on my phone" novelty.

Apple is reportedly in early talks with PrismML. According to CNBC reporting cited by Decrypt, Apple is evaluating PrismML's compression technology for potential on-device use, and PrismML's CEO confirmed a compressed Gemma model is already next in their pipeline, with larger frontier models to follow. If this compression approach gets adopted more broadly, it could meaningfully change how much on-device AI capability ships in consumer hardware going forward.

The intelligence-per-gigabyte trend is accelerating. PrismML frames this as a "Pareto shift" — extracting dramatically more capability per gigabyte of memory than previous compression approaches. Whether or not Bonsai specifically becomes your go-to local model, the broader trend it represents — genuinely capable models shrinking fast enough to run on consumer hardware — is one worth tracking if you're making build-vs-buy decisions on your AI infrastructure.

AIWerse Verdict

Bonsai 27B represents a genuine step forward in what's practically possible on consumer hardware — not a marketing exaggeration, but a real 27B-class model that fits in a phone's memory budget while keeping the vast majority of its reasoning capability intact. For developers and builders experimenting with agentic workflows, it's a legitimate free tool worth adding to your stack, especially for prototyping, privacy-sensitive use cases, or high-volume routine tasks where cloud API costs would otherwise compound.

It won't replace frontier cloud models for your hardest production reasoning tasks — the benchmark gaps in tool-calling and instruction-following are real. But as a free, offline, genuinely capable local option, Bonsai 27B earns a spot in any serious builder's toolkit.

Try the browser demo first if you're unsure, then commit to a full local install once you've confirmed it fits your workflow.

FAQs

Is Bonsai 27B actually free?

Yes. Both the ternary and 1-bit variants are released under the Apache 2.0 license, which permits commercial use with no licensing fees. You can download the weights directly from Hugging Face or GitHub at no cost.

How do I run Bonsai 27B locally?

Use PrismML's "Locally AI" app for iPhone, MLX for Mac (Apple Silicon), or their custom CUDA kernels for NVIDIA GPUs on Windows/Linux. See the step-by-step section above for exact instructions per platform.

Is Bonsai 27B better than Qwen 3.6 27B?

No — Bonsai 27B is a compressed version of Qwen 3.6 27B, so the original uncompressed model is still more capable (85.0 vs 80.5/76.1 average benchmark score). Bonsai's value isn't beating the original — it's retaining 90-95% of that capability at a fraction of the memory footprint, small enough to actually run on consumer hardware including phones.

What's the difference between 1-bit and ternary Bonsai?

1-bit Bonsai uses only two weight values (-1, +1) and is the smallest at 3.9GB — built specifically to fit on phone-class memory budgets. Ternary Bonsai adds a third value (0) for slightly more expressive power, landing at 5.9GB with noticeably better quality — the better choice if you're running it on a laptop rather than a phone.

Can Bonsai 27B run without internet?

Yes, once downloaded, both variants run entirely offline with no internet connection required. This is one of the core advantages over cloud-based AI models.

What phone do I need to run Bonsai 27B?

PrismML's benchmarks and testing were done specifically on an iPhone 17 Pro Max class device, since phones typically only expose 40-50% of total RAM to any single app, and clearing that memory ceiling for a 27B-class model is what makes this release notable. Older or lower-RAM phones may not have enough headroom to run it.

Is Bonsai 27B good for coding?

Reasonably strong — the ternary variant scores 86.0 on coding benchmarks (HumanEval+, MBPP+) versus 88.7 for the uncompressed original, a relatively small gap. For agentic coding workflows requiring extensive iteration, cloud frontier models still hold an edge, but for local prototyping and quick code generation, Bonsai performs well.

Related News & Updates

• Claude Sonnet 5 vs Opus 4.8: Which Should Power Your Stack in 2026?

• Cloudflare AI Crawler Rules: Avoid Blocking Google in 2026

• AI Guides

Junaid Nawaz is the founder of AIwerse and a developer focused on AI tools, agentic workflows, and builder-focused tech. He covers AI model releases, coding tools, and platform updates for developers and teams building with AI. You can follow AIwerse on X (@AIwerse).