EvalSignal 007 / Independent model evaluation
Definitely not Opus.
When vision workloads reach scale, even a fraction of a cent per request becomes material. In this price tier, Qwen currently has a clear cost advantage.
Running locally is preferable when you have the hardware. But GPUs get busy, local throughput has limits, and some workloads require more parallelism than one machine can provide.
Whether you are distilling a dataset, categorizing images, extracting visual records, running OCR, or processing a large collection of screenshots, hosted Qwen models are an inexpensive option through OpenRouter.
But which Qwen should you choose?
There are dense and MoE architectures, instruct and thinking variants, dedicated vision models, and newer general-purpose models. Is newer automatically better? Does bigger mean more accurate? Is thinking worth the extra tokens?
We tested all six.
EvalSignal ran 600 image requests across a frozen 100-case vision suite built from real interface screenshots. Every endpoint received the same images, instructions, temperature, token budget, and scoring rubric.
All 600 requests completed without an API error. Total cost: $0.53.
A strict pass required the complete structured response, all important visual facts, and no critical contradiction.
Mean rubric gives partial credit when a model identifies only some expected facts. Latency is the mean end-to-end response time. Output tokens are the billed completion tokens across all 100 cases.
| Model | Strict | Rubric | Latency | Output | Cost |
|---|---|---|---|---|---|
| Qwen3-VL-32B Instruct |
69 | 91.9% | 4.6s | 22.2K | $0.022 |
| Qwen3-VL-30B-A3B Instruct / MoE |
68 | 91.3% | 2.2s | 17.1K | $0.024 |
| Qwen3.5-35B-A3B Generalist / MoE |
67 | 90.8% | 5.2s | 55.4K | $0.099 |
| Qwen3-VL-30B-A3B Thinking / MoE |
62 | 88.9% | 6.3s | 68.5K | $0.188 |
| Qwen3-VL-8B Thinking |
61 | 86.7% | 6.4s | 75.5K | $0.180 |
| Qwen3-VL-8B Instruct |
48 | 84.0% | 2.1s | 14.9K | $0.021 |
Qwen3-VL-32B Instruct produced the best score, but its advantage over the 30B-A3B MoE was only one strict pass and 0.6 rubric points.
The MoE model answered in 2.2 seconds instead of 4.6 seconds. When thousands or millions of images are waiting, that difference compounds.
Choose the dense 32B when maximum accuracy per request matters. Choose the 30B-A3B MoE when throughput matters too.
At 8B, thinking increased strict passes from 48 to 61 and improved advanced-case performance from 35% to 55%.
But output grew from 14.9K to 75.5K tokens. Cost increased from $0.021 to $0.180, while latency rose from 2.1 to 6.4 seconds.
At 30B, thinking produced the opposite result. Strict passes fell from 68 to 62, cost increased from $0.024 to $0.188, and latency rose from 2.2 to 6.3 seconds.
Thinking can rescue a small model. At 30B, it made the model slower, more expensive, and less accurate.
Qwen3.5-35B-A3B supports text, images, and video, but finished behind both dedicated Qwen3-VL models.
It consumed 55.4K output tokens and cost roughly four times as much as the two leaders. A broader multimodal model is not automatically the best choice for a vision-heavy workload.
Eighteen cases failed across all six models.
Form validation and modal overlays were especially difficult: even the best models reached only a 20% strict-pass rate. Multilingual OCR peaked at 40%.
Budget vision is useful, but difficult overlays, hidden controls, validation errors, and ambiguous interface states remain unresolved.
Lowest absolute cost: Qwen3-VL-8B Instruct
The cheapest measured run, but its difficult-case reliability was substantially lower.
Best value and maximum quality: Qwen3-VL-32B Instruct
The highest score cost only $0.001 more per 100 cases than the 8B model.
Best speed at high quality: Qwen3-VL-30B-A3B Instruct
Almost the highest score, roughly half the latency of the quality leader.
Best cost, speed, and quality mix: Qwen3-VL-30B-A3B Instruct
The strongest overall operating profile for large parallel vision workloads.
Best for combined vision and textual intelligence: Qwen3.5-35B-A3B or Qwen3.6-35B-A3B
Our measured results belong to Qwen3.5. Public benchmarks suggest that Qwen3.6 delivers broadly similar visual performance while improving textual, coding, and agentic capabilities.
For a new mixed text-and-image deployment, we would choose Qwen3.6. See the public benchmark results.
For vision-only processing at scale, choose Qwen3-VL-30B-A3B Instruct.
For maximum accuracy and surprisingly strong value, choose Qwen3-VL-32B Instruct.
When the same workload also requires substantial textual intelligence, choose Qwen3.6-35B-A3B.
Pricing, availability, and latency represent an August 22, 2026 snapshot. OpenRouter routing and prices may change.
