EvalSignal 019 / Independent model evaluation
Fine-tune in BF16, then compress with OpenBonsai. Plus: MiniCPM-V 4.7 and LFM3-VL-2B push the 1B and 2B multimodal frontier.
This issue has two stories about useful AI on ordinary hardware: bringing your own specialization into a tiny ternary model, and choosing between two compact models that read screens and select tools.
Prism ML's Ternary Bonsai releases demonstrated an appealing idea: a 9B model can occupy about 2GB when its weights are constrained to three values, -1, 0 and +1, plus scaling factors for groups of weights. That makes the compression recipe valuable to anyone who wants to deploy a specialized model locally.
Prism released ternary checkpoints while keeping its compression recipe private. OpenBonsai makes an alternative Apache-2.0 compression pipeline available for that job. It takes BF16 models through calibration, ternary rounding and scale distillation, then exports a GGUF compatible with Prism's runtime.
The main benefit is bringing your own BF16 fine-tune into the ternary Bonsai ecosystem. For a supported base model, fine-tune in BF16, merge the adapted weights, then compress that specialized checkpoint into Bonsai-compatible ternary weights. Your fine-tune acts as the teacher; its own answers, including answers to prompts from your task, guide calibration and distillation. The goal is to retain the specialization you trained while shrinking the deployment footprint.
| Release detail | Reported result |
|---|---|
| First published model | Qwen3.5-9B, text only |
| BF16 weights | Approximately 18GB |
| Ternary model file | 1.997GB, about 9 times smaller |
| Compression hardware | One RTX 5090; about 5 hours estimated from measured stages |
| Serving format | PTQ1_0 GGUF, using Prism's llama.cpp fork |
These are project-reported compression results. Model-file size excludes runtime overhead and the KV cache, which stores the conversation's attention state.
Extreme compression can erase useful behavior. OpenBonsai draws on the Attend to Your Own Thoughts approach from Intel Labs China: let the original model generate reasoning traces, then use those activations to guide compression. The aim is to preserve the pathways used during problem solving. Its pipeline also distills scaling factors against the original model's outputs.
The current checkpoint is a research preview. Its model card reports coding improvements over the first release, together with model-selection and data-overlap caveats. Math and coding still trail the BF16 original and Prism's Ternary Bonsai 2 9B. The useful contribution today is an inspectable pipeline, a downloadable checkpoint and published failure analysis that others can reproduce and improve.
For deployment, use the documented Prism llama.cpp fork: upstream llama.cpp does not support this PTQ1_0 file. Published runtime validation used the fork's CPU build; the RTX 5090 figure above describes compression work. The released checkpoint is text only, so a local agent still needs a separate vision model for screenshots.
The next goal is to extend this BF16-to-ternary workflow to Muse Glimmer and Flint. Supporting their architectures and measuring how much task capability survives compression are the next steps. They are roadmap targets; the published Qwen3.5-9B checkpoint is the working example today.
Two recent releases are resetting expectations for tiny multimodal AI: MiniCPM-V 4.7 and LFM3-VL-2B. At just 1B and 2B parameters, they bring screen reading and agent tool use into small local models. Both push their size classes forward, opening up new possibilities for on-device assistants that can understand a screen and help decide what to do next. Their latest routing and vision results show where that progress is strongest.
MiniCPM-V 4.7 here is the 1B checkpoint. The LFM3-VL-2B results use the decision-focused checkpoint logged as lfm3-vld-2b, based on Liquid AI's architecture. Both ran locally in Q4 K M quantization on NVIDIA RTX 5060 Ti hardware.
The routing suite used 100 first-turn prompts and 100 seeded scenarios; 89 scenarios were scored after excluding 11 developer-mode cases. Exact action means the next tool name and arguments both match the reference. The broader tool-family match also credits the right tool with different arguments, or an appropriate direct answer that avoids an unnecessary tool call.
| Metric | MiniCPM-V 4.7 (1B) |
LFM3-VL 2B |
|---|---|---|
| Structured calls | 97/100 | 87/100 |
| Exact action | 2/89 | 1/89 |
| Tool-family match | 27/89 30.3% | 32/89 36.0% |
| Anti-patterns Lower is better | 4/89 | 2/89 |
LFM3's broader routing score is 5.7 percentage points higher. MiniCPM gets one more exact action. Both exact-match counts are very low: choosing a plausible tool still leaves substantial work on its arguments. Structured-call coverage counts emitted tool calls; some prompts appropriately received direct prose answers.
The traces also recorded one context-length error for MiniCPM and six output-length limits for LFM3. Those integration details matter when a small model is expected to complete an action reliably.
The vision suite presented the same 100 held-out screenshots and the same six-section observation prompt to both models. A full pass requires the requested structure and all required content, with no critical hallucination. Mean rubric averages the percentage of available rubric points earned across the screenshots. The original grader expects numbered headings; latency below is the mean time per screenshot.
| Original grader | MiniCPM-V 4.7 (1B) |
LFM3-VL 2B |
|---|---|---|
| Full passes | 0/100 | 4/100 |
| Mean rubric | 9.4% | 44.7% |
| Screenshot latency | 1.30s | 2.20s |
| Allocated VRAM | ~2.4GB | ~2.8GB |
MiniCPM's 0/100 needs an explanation. It supplied the six named sections in order, but dropped the leading digits. The grader consequently failed to find the sections. When header recognition was broadened to accept those names, the same saved responses reached 37/100 full passes and a 68.2% mean rubric score. This was a scoring change, with no new model generation.
That normalized score reveals much stronger screenshot reading than the original format score suggests. The source reports it only for MiniCPM; LFM3's 4/100 and 44.7% above use the original grader. A final ranking under the more tolerant parser needs the same rescoring for both models.
MiniCPM's advantages were especially visible in UI text and form feedback. LFM3 was more competitive on scene context and uncertainty. Both completed all 100 screenshot cases without HTTP errors or timeouts.
The small weight files also need vision projectors: MiniCPM pairs about 0.64GB of weights with a 1.11GB projector; LFM3 pairs 1.25GB with a 379MB projector. The measured VRAM totals above include the running configuration, making them more useful for hardware planning than weight size alone.
MiniCPM-V 4.7 is an impressive choice for capability per parameter. At roughly half LFM3's model size, it produces rich screenshot descriptions and responds faster in this setup. It is especially attractive when size and speed matter, provided the integration accepts and validates its section headings. Its full running VRAM footprint is closer to LFM3's because of the larger vision projector.
LFM3-VL-2B is our quality-first pick for the structured agent workflow tested here. It has the higher tool-family match and the better score under the original six-section vision contract. Its extra parameters buy stronger routing and observation-format compliance. That recommendation concerns the tested agent contract; MiniCPM's higher normalized vision-content score remains an important counterpoint.
Both push the practical frontier of their respective size classes: MiniCPM shows how much a 1B multimodal model can do, while LFM3 makes the case for spending more parameters on structured agent behavior. Keep exact-action validation in the loop for either model.
OpenBonsai project and BF16 fine-tuning workflow · Released checkpoint and current model card
OpenBonsai release report · Frozen tool-routing methodology and results · Vision results and header-normalization analysis
