Spark-X2.6, LFM3, MiniCPM-V 4.7, and WebBrain VLD Models: Compact Agent Routing on NVIDIA RTX 5060

We evaluated Spark-X2.6-4B, Liquid AI LFM3-1.7B, MiniCPM-V-4.7-1B, WebBrain-Compass-VLD-Tiny-v5, and LFM3-VLD-2B locally on dual NVIDIA GeForce RTX 5060 Ti GPUs using official revision-pinned Q4KM GGUF checkpoints. The benchmark executed WebBrain's frozen September 7 Compact contract across 100 first-turn routing prompts and 100 seeded multi-turn scenarios.

Spark-X2.6-4B leads the evaluated models on strict exact-action matching with 11/89 (12.4%). webbrain-compass-vld-tiny-v5 and lfm3-vld-2b lead the looser tool-family match with 32/89 (36.0%).

Comparison: the newly evaluated models

The suite contains 100 first-turn prompts and 100 seeded scenarios. Eleven Dev scenarios are outside Compact mode, leaving 89 scored scenarios. Strict means the model's next tool call matches the reference tool name and arguments. Loose matching additionally credits tool-name matches with divergent arguments and terminal prose that answers the user question without calling redundant tools. Structured-call coverage measures whether the model emitted valid JSON tool calls.

All models used the same frozen inputs, quantization (Q4KM), deterministic sampling seeds (3407), and concurrency=1 on dedicated RTX 5060 Ti hardware. MiniCPM-V-4.7-1B and Liquid AI LFM3-1.7B executed in parallel across the two available RTX 5060 Ti GPUs, followed by Spark-X2.6-4B, WebBrain-Compass-VLD-Tiny-v5, and LFM3-VLD-2B on CUDA2. SHA-256 digests were verified before inference.

What the responses show

Tool calling and inspection preferences

Spark-X2.6-4B and LFM3-1.7B achieved 100/100 structured tool call validity in first-turn routing, demonstrating strong adherence to OpenAI function calling schemas. Both WebBrain-Compass-VLD-Tiny-v5 and MiniCPM-V-4.7-1B achieved 97/100, directly answering three conversational queries without invoking unnecessary tools, while LFM3-VLD-2B achieved 87/100, resolving 13 conversational queries with direct terminal text.

As observed in earlier compact model tests, small models display a heavy bias toward tree inspection when faced with web navigation tasks:

• Spark-X2.6-4B called get_accessibility_tree on 75/100 first-turn prompts, down from 78/100 in Spark-X2.5-4B. It matched 11/89 strict actions and 29/89 loose actions, matching its predecessor's strict score while demonstrating solid anti-pattern resistance (only 2 anti-pattern verdicts).
• LFM3-1.7B showed a marked improvement over the LFM 2.5 generation: its strict score jumped to 10/89 (compared to 4/89 for LFM2.5-2.6B), while its first-turn navigation calls reached 27/100 with only 1 anti-pattern throughout the entire scenario suite. At 190ms average per-step latency on RTX 5060 Ti, it was the fastest model in the lineup.
• MiniCPM-V-4.7-1B showed higher diversity in initial tool selection (calling fetchurl, click, readpage, and inspect_viewport) and reached 27/89 (30.3%) loose tool-family matching, but only achieved 2 strict matches due to argument divergence, alongside 4 anti-patterns and 1 context-length error.
• WebBrain-Compass-VLD-Tiny-v5 achieved a top loose match score of 32/89 (36.0%), outperforming Spark-X2.6-4B (29/89) and base MiniCPM-V-4.7-1B (27/89). It also led all models on first-turn exact reference match with 8/100, while exhibiting the lowest tree-querying dependency (50/100). Fine-tuned specifically for browser GUI grounding and decision routing, it produced 7 strict scenario matches (more than triple base MiniCPM-V's 2) and favored direct navigation, form input (type_ax), and clicks over passive tree querying.
• LFM3-VLD-2B achieved the joint-top loose match score of 32/89 (36.0%), matching WebBrain-Compass-VLD-Tiny-v5. Fine-tuned with the Pegasus VLD method on Liquid AI's hybrid convolution/attention backbone, it exhibited near-flawless anti-pattern avoidance (only 2/89 across the entire scenario suite) and 0 errors, demonstrating that task-specific decision grounding substantially elevates Liquid AI's compact routing capability (up from 19/89 to 32/89).

Generational progression: Spark, LFM, MiniCPM, and WebBrain VLD

Comparing the new releases against their direct predecessors in WebBrain's frozen benchmark reveals notable architectural refinements:

1. Spark-X2.6 vs Spark-X2.5: Spark-X2.6-4B maintains top-tier strict execution (11/89) and clean 100% structured call fidelity, with reduced tree-inspection bias and improved prompt evaluation throughput on modern Ada/Blackwell tensor cores.
2. Liquid AI LFM3 vs LFM2.5: LFM3-1.7B more than doubles LFM2.5-2.6B's strict score (10/89 vs 4/89) while requiring 35% fewer parameters and running at nearly 500 tokens/sec total throughput in llama.cpp.
3. WebBrain-Compass-VLD-Tiny-v5 vs base MiniCPM-V 4.7: While both share the same compact 1B architecture and 672MB Q4KM weights, WebBrain's task-aligned VLD fine-tune dramatically improves tool precision: strict matching increases from 2/89 to 7/89, and loose matching reaches 32/89.
4. LFM3-VLD-2B vs base LFM3-1.7B: Applying WebBrain's VLD decision training to Liquid AI's 2B hybrid architecture increases loose tool matching by +14.7 percentage points (36.0% vs 21.3%), while maintaining an exceptionally clean execution profile (2 anti-patterns, 0 errors). Both WebBrain VLD fine-tunes now tie for first place among all evaluated compact models in tool-family routing.

Complete comparison with historical compact models

The table below compiles all evaluated models on WebBrain's Compact frozen contract, sorted by strict score, then loose score:

Hardware setup and methodology

Runs were conducted locally on Ubuntu 24.04 with dual NVIDIA GeForce RTX 5060 Ti GPUs (16GB VRAM each). MiniCPM-V-4.7-1B ran on CUDA2 (port 18080) and LFM3-1.7B ran on CUDA1 (port 18081) simultaneously, followed by Spark-X2.6-4B, WebBrain-Compass-VLD-Tiny-v5, and LFM3-VLD-2B on CUDA2.

The serving backend was llama.cpp build b11960 with CUDA acceleration, Flash Attention enabled, sampling seed 3407, Act temperature 0.15, Ask temperature 0.3, top-p 1, top-k 0, min-p 0, and repetition penalty 1. Native Jinja chat templates were used with reasoning/thinking disabled to benchmark core agent routing.

Download the complete comparison data. Raw per-case traces, logs, and evidence audits are stored under test/llm/analysis/2026-10-29-spark-lfm-minicpm-compact/.