Compression method
OpenBonsai
An Apache-2.0 pipeline that compresses dense language models to ternary weights {−1, 0, +1} in the Bonsai (PTQ1_0) format, so a capable model fits on the device that runs the browser. Compatible with Prism ML’s Ternary Bonsai runtime, whose own compression pipeline is closed.
Why it matters
There are two ways to get a capable model inside the browser: train a tiny one, or shrink a strong one. OpenBonsai is the second route, done in the open, so anyone can compress their own fine-tune instead of waiting for someone else’s release.
How it works
openbonsai run turns a BF16 model into a PTQ1_0 GGUF in four stages: Rotate, Round (sequential GPTQ), Refine (ScaleQ/CAT-Q, research only and skipped by default) and Distill. Calibration uses the model’s own answers to math and code prompts, following ScaleQ-1.58 (Wang et al., Intel Labs China).
The same pipeline compressed Qwen3-8B with no architecture-specific code, and the output drops into Prism ML’s llama.cpp fork. Your own merged fine-tune can be compressed the same way, using its answers as calibration and distillation data.
Ternary inside the browser
WebBrain also converted Prism ML’s Ternary Bonsai 2 vision models to ONNX for WebGPU. The first graph took minutes per prompt because prefill ran token by token on the CPU (about 0.13 s per prompt token for 2B on an RTX 5090); the rewritten graph handles 1k prompt tokens in 1.3 s and 4k in 3.3 s.
Ternary models are disabled as WebBrain’s decision provider for now, because they gave confident false answers.
Results so far (Qwen3.5-9B)
| Model | GSM8K subset, thinking off | HumanEval, thinking off |
|---|---|---|
| OpenBonsai Qwen3.5-9B | 65/100 | 34/50 |
| Prism ML Ternary Bonsai 2 9B | 77/100 | 45/50 |
From OpenBonsai’s PyTorch harness, not Prism’s runtime. The HumanEval figure is probably optimistic: the setting was chosen from four runs on the same 50 problems. With thinking on: 32/50 and 14/25 for OpenBonsai, 46/50 and 24/25 for Prism’s model.
Models
-
openbonsai-qwen-3.5-9bQwen3.5-9B compressed to ternary with OpenBonsai 202 downloads · 4 likes -
Ternary-Bonsai-2-2B-ONNX-WebGPU-VisionPrism ML’s Ternary Bonsai 2 2B, rewritten for WebGPU with vision 373 downloads · 1 likes -
Ternary-Bonsai-2-4B-ONNX-WebGPU-VisionPrism ML’s Ternary Bonsai 2 4B, rewritten for WebGPU with vision 109 downloads · 2 likes
Caveats
- A research preview, not a finished product, with a measurable quality gap to the uncompressed base and to Prism ML’s releases.
- Code and non-English output are still the main weakness.
- Independent project, not affiliated with or endorsed by Prism ML. Bonsai is Prism ML’s product.