Tiny Vision Models Compared III: LFM3-VLD and MiniCPM-V 4.7 Under WebBrain's Strict Browser Rubric

A tiny multimodal model can have sharp visual perception and still fail a brittle agent contract. In our newest 100-case browser viewport shootout, webbrain-one/lfm3-vld-2b and openbmb/MiniCPM-V-4.7-1B went head-to-head on dual NVIDIA RTX 5060 Ti GPUs. LFM3-VLD-2B proved disciplined, adhering strictly to the six-section contract with 4 full passes and a 44.7% mean rubric score. MiniCPM-V 4.7 scored 0 strict passes under our exact numbered regex because it dropped the leading digits 1) to 6) in favor of markdown labels; yet when scored on its actual section content, MiniCPM shot up to 37 strict passes and a 68.2% mean score at blazing 1.3-second latency.

The expanded compact vision table

Following our earlier Tiny Vision Models Compared II report, we evaluated the newest generation of compact multimodal models released in late 2026.

Both new models were run entirely offline in 4-bit quantization (Q4KM GGUF) using llama-server on dedicated NVIDIA GeForce RTX 5060 Ti GPUs. Every model evaluated in this benchmark encountered the identical 100 held-out browser screenshots (ranging from clean landing pages to dense analytics dashboards, multilingual checkouts, and occluded modals), the exact production system prompt, and the deterministic scoring rubric.

All 100 cases scored with zero HTTP errors or timeouts. A strict pass requires full compliance with WebBrain's six-section observation schema without critical hallucinations.

The MiniCPM anomaly: why a 1B model scored 0% and 68% at the same time

When MiniCPM-V 4.7 finished its 100 cases, the initial score report looked disastrous: 0 strict passes, 9.4% mean rubric score.

Yet when we inspected the raw output files, the descriptions were astonishingly vivid. Consider case 001.png (a standard account sign-in screen):

Page purpose: WebScene sign-in dialog.
Visible text: "Welcome back", "Use your account to continue", "Email", "you@example.com", "Password", "Enter password", "Sign in", "Need help?"
Inputs: Email field with value "you@example.com", Password field with label "Enter password", blue "Sign in" button, blue "Need help?" link.
State signals: None visible.
Blockers: None.
Unknowns: None.

MiniCPM identified every single visible UI element, extracted the verbatim button text, recognized the password placeholder, and correctly verified that no modals or error banners blocked the viewport.

Why did the benchmark assign it 0%?

WebBrain's production vision prompt requests six numbered sections:

1. 1) Page purpose:
2. 2) Visible text:
3. 3) Inputs:
4. 4) State signals:
5. 5) Blockers:
6. 6) Unknowns:

To prevent cross-section hallucinations (e.g. counting an error message found in a photo background as a UI blocker), WebBrain's production grader isolates each section using a strict regex:

const marker = /^\s*(?:#{1,6}\s*)?(?:\*\*)?([1-6])\s*[).:-]\s*(?:\*\*)?/gm;

MiniCPM-V 4.7 provided all six sections, with the exact heading names, in the exact order—but omitted the numeral prefix 1) to 6). As a result, the parser found zero numbered sections, scoped checks evaluated to empty strings, and critical checks failed.

When we normalized the section headers to recognize Page purpose:, Visible text:, and Inputs: regardless of whether a leading digit was present, MiniCPM's score transformed:

• Strict passes: 0/100 → 37/100
• Mean rubric: 9.4% → 68.2%
• Speed: 1,299 ms average on an RTX 5060 Ti

At 641 MB for the quantized weights, MiniCPM-V 4.7 demonstrates visual acuity close to Qwen 3.5 2B (39 passes) with half the parameters and five times faster inference. But for autonomous agents that rely on structured scrapers, it highlights an important lesson: format compliance is as critical as perception.

LFM3-VLD-2B: disciplined structure from a decision specialist

Liquid AI's LFM3 architecture brings a hybrid linear computational foundation to compact multimodal tasks. We tested webbrain-one/lfm3-vld-2b, a 2B checkpoint paired with its multimodal vision projector (mmproj-LFM3-VL-2B-VLDR-F16.gguf).

Unlike its predecessor LFM2.5-VL-3B—which completely collapsed on the numbered prompt structure—LFM3-VLD-2B followed the six-section format faithfully across all 100 test queries.

• Strict passes: 4/100 (in consent banners, data tables, navigation maps, and photo understanding)
• Mean rubric score: 44.7%
• Average latency: 2,199 ms on RTX 5060 Ti
• Errors: 0

Examining LFM3-VLD's responses across difficulty tiers shows steady reliability:

LFM3-VLD was fine-tuned primarily as a Vision-Language Decision (VLD) model—aimed at compact tool routing, next-action selection, and action ground-truthing in WebBrain's browser loop (where it scored a joint #1 36.0% loose routing match in our parallel tool-routing benchmark).

When repurposed as an end-to-end screen describer, it reliably identifies major form inputs, active tabs, and primary blockers, even if it leaves out minor background snippets that our 100-case rubric rewards.

Category breakdown

Our 100-case benchmark spans twenty real-world UI categories:

In categories dominated by fine-grained UI text reading (Checkout, Form validation, Search results, and Charts), MiniCPM-V 4.7's vision encoder exhibits higher text fidelity, correctly capturing validation errors and currency amounts.

In contextual reasoning and restraint (Uncertainty calibration, Modal detection, and Photo scenes), LFM3-VLD is more cautious, declining to guess unreadable fields and accurately flagging overlay states.

Hardware footprint on NVIDIA RTX 5060 Ti

Both evaluations were conducted on dual consumer NVIDIA GeForce RTX 5060 Ti (16 GB) graphics cards running native llama-server binaries with full layer offloading (-ngl 99):

• MiniCPM-V 4.7 1B:
• Base model: 641 MB (Q4KM)
• Multimodal projector: 1.11 GB (F16)
• Total VRAM allocated: ~2.4 GB
• Prompt evaluation: ~32 ms/token (multimodal projection ~700 ms)
• Generation speed: ~28 tokens/sec
• Total latency per screenshot: 1.30s
• LFM3-VLD-2B:
• Base model: 1.25 GB (Q4KM)
• Multimodal projector: 379 MB (F16)
• Total VRAM allocated: ~2.8 GB
• Prompt evaluation: ~35 ms/token (multimodal projection ~1.2s)
• Generation speed: ~128 tokens/sec
• Total latency per screenshot: 2.20s

Both models operate well within the memory headroom of consumer desktop GPUs and can comfortably co-exist with a local language model or background browser automation instance.

Practical takeaways for agent builders

1. Guard against format fragility in small vision models: Small models under 2B parameters frequently drop structural conventions (such as list numbering or specific prefixes) when their attention is split between visual encoding and textual generation. Agent frameworks should prefer regex parsers that match both numbered and named section headers.
2. Specialized vs. Generalist trade-offs: LFM3-VLD-2B is optimized as an action router—a role where it excels at deciding click/type targets. MiniCPM-V 4.7 is a strong general-purpose visual reader whose OCR and visual grounding punch well above its 1B weight class.
3. Local vision is viable for consumer hardware: Both models achieve sub-2.5 second screenshot observations locally on an RTX 5060 Ti with under 3 GB of VRAM, making cloud vision APIs unnecessary for local browser agents.