EvalSignal field note
Another very good week for open-source AI. Three releases landed close together — Qwen3.8-27B, DeepSeek-V4-Flash-0731, and Poolside’s Laguna S 2.1 — and together they reshuffled the local-agent stack.
Scope note. GLM-5.3 also shipped this month and, by Zhipu’s own numbers, sits in Fable 5 territory and above Kimi K3. It is not a model you run on your own machine, so we are leaving it out of the head-to-head. Same reason DeepSeek V4 Pro doesn’t appear here. This issue is about weights you can actually pull down and serve offline.
The short version: Qwen3.8-27B is the monster. It is the strongest model you can currently run on a single 24 GB consumer GPU, and it is the only one of the three that can see.
Meta’s Muse Glimmer (Aug 10, 30B, Apache 2.0) was the story five days earlier. It beat Qwen3.6-27B and Gemma 4-31B across most agentic benchmarks — MCP Atlas 75.5, DeepSearch QA 74.6, Gaia2 43.3, AIME 2026 94.7. But the wins were narrow and uneven: Qwen3.6-27B still held OSWorld-Verified 75.6 vs 65.9, TerminalBench 2.1 60.7, and SWE-Bench Verified 77.2.
Qwen 3.8 does not win narrowly. It wins where local agents actually break.
NVIDIA’s Nemotron 3 line is the other reference point, and it is not a happy one. On the open-weight leaderboards, Nemotron 3 Ultra sits around 48 while Qwen sits around 52 — six months later, with vastly more compute behind it. Nemotron’s real story is throughput (NVIDIA claims 7.5x over Qwen3.5-122B in high-volume serving), not capability.
| Benchmark | Qwen 27B | Laguna S | DeepSeek | Winner |
|---|---|---|---|---|
| Terminal-Bench 2.1 | 73.0 | 70.2 | 82.7 | DeepSeek |
| DeepSWE | 42.2¹ | 40.4 | 54.4 | DeepSeek |
| SWE-bench Pro | 61.7 | 59.4 | — | Qwen |
| NL2Repo | 42.3 | — | 54.2 | DeepSeek |
| Toolathlon Verified | — | 49.7 | 70.3 | DeepSeek |
| Agents' Last Exam | 20.4 (Pass@1) | — | 25.2 | DeepSeek² |
All numbers are vendor-reported from official model cards and launch posts.
¹ Qwen labels its run DeepSWE 1.1, so the three DeepSWE figures are not guaranteed to be the same evaluation setup. ² Agent benchmarks use different harnesses and scaffolds across vendors. The direction is meaningful; two-point gaps are not.
DeepSeek beats Qwen by 9.7 and Laguna by 12.5. Qwen beats Laguna by only 2.8 — and that is the number worth sitting with, because these are radically different objects. Qwen is a 27B dense model. Laguna is 118B total / ~8B active MoE. Laguna was already remarkable for its active-parameter count; Qwen just erased most of its margin from a much smaller footprint.
DeepSeek’s jump is its own kind of strange. The V4-Flash-Preview scored 61.8 on Terminal-Bench 2.1. The 0731 checkpoint scores 82.7 with identical architecture and parameter count — 284B total, 13B active, 1M context. Only the post-training was redone. DeepSeek says 0731 supersedes the preview outright, and the agentic gains are the reason.
That should also make you slightly more skeptical of the absolute numbers. A 21-point move from post-training alone, measured on an unreleased internal harness, is the kind of result that usually shrinks under independent evaluation.
| Benchmark | Score |
|---|---|
| Terminal-Bench 2.1 | 73.0 |
| SWE-bench Pro | 61.7 |
| NL2Repo | 42.3 |
| DeepSWE 1.1 | 42.2 |
| QwenSWEBench | 79.0 |
| LiveCodeBench v6 | 90.3 |
| GPQA Diamond | 89.2 |
| HLE | 30.8 |
| IFBench | 79.5 |
| CoWorkBench | 70.7 |
Against its own predecessor at the same nominal dense parameter count:
| Benchmark | Qwen 3.6 | Qwen 3.8 | Delta |
|---|---|---|---|
| Terminal-Bench 2.1 | 63.4 | 73.0 | +9.6 |
| SWE-bench Pro | 53.5 | 61.7 | +8.2 |
| DeepSWE | 13.3 | 42.2 | +28.9 |
| OSWorld-Verified | 63.9 | 84.3 | +20.4 |
The DeepSWE line is not a typo. Whatever changed in Qwen’s agentic post-training, it changed a lot.
Qwen3.8-27B is natively multimodal. The model card lists image and video input alongside text, and describes it as a 27B causal LM with a vision encoder. Its agentic vision results:
| Benchmark | Score |
|---|---|
| OSWorld-Verified | 84.3 |
| AndroidWorld | 81.9 |
| WebArena-Verified | 64.8 |
| Vision2Web | 62.9 |
| SWE-MM | 38.6 |
Laguna S 2.1 lists its modality as text-to-text. DeepSeek V4 Flash 0731 ships as a text-generation model. If your agent needs to look at a screen — and every browser or desktop agent eventually does — there is currently one answer in this size class.
Capability per parameter is only half the local story. The other half is memory bandwidth, and it decides which of these models is actually usable on your box.
Qwen3.8-27B footprint (27.78B params, 262,144-token native context, Apache 2.0):
| Quantization | VRAM |
|---|---|
| BF16 | ~54 GB |
| FP8 | ~27 GB |
| Q6_K (near-lossless) | ~21 GB |
| Q4_K_M | ~16 GB |
| Q3_K_M / IQ3 | ~13 GB |
24 GB is the fit-first minimum. 32–48 GB is the comfortable range once you want real context length — and with a 262K window, KV cache is not a rounding error.
Dense is fast where bandwidth is fast, and painful where it isn’t. On a 5090, a 27B dense model at 4-bit is close to ideal: the weights fit, the bandwidth is there, and you get interactive speeds. On a DGX Spark (GB10), the same model is memory-bandwidth-bound — the unified memory holds it comfortably, but dense inference has to stream every parameter per token and the box simply doesn’t feed it fast enough. This is the well-documented Spark failure mode, and it is exactly where Laguna’s ~8B active MoE wins: it reads a fraction of the weights per token, and Poolside sized its NVFP4 release to fit a single Spark deliberately.
And the hardware market is not helping. RTX 5090 median US pricing hit ~$4,700 in August, roughly 135% over the $1,999 MSRP, driven by the memory shortage rather than gaming demand. A model this good on a single 32 GB card is not going to cool that off.
Single 5090 (or any 24–32 GB consumer GPU) → Qwen3.8-27B. Best capability that fits, and you get vision for free. Q4_K_M if you’re tight, Q6_K if you have the headroom.
Single DGX Spark → Laguna S 2.1 for throughput, Qwen3.8-27B if you need vision. Laguna’s MoE architecture is the right shape for Spark’s bandwidth profile and Poolside tuned the NVFP4 build for exactly this box. Run Qwen only if the task requires screenshots, and expect it to be slow.
Multiple DGX / serious local cluster → DeepSeek V4 Flash 0731, or GLM 5.3 if you’ll accept a non-local fallback. 0731 is the strongest pure text/code agent brain here by a clear margin.
Every number in this issue is vendor-reported, and several come from harnesses that have not been released. Qwen’s DeepSWE run is explicitly a different version than DeepSeek’s and Poolside’s. Treat the rankings as directional and the point gaps as noise until independent runs land. We will revisit these on the frozen agentic web suite, where every model gets the same browser states, the same 41-tool schema, and the same prompt.
Model cards and official announcements
Qwen3.8-27B-FP8 (Hugging Face) · DeepSeek-V4-Flash GA post · Introducing Laguna S 2.1 (Poolside) · Muse Glimmer (Meta AI Research) · Muse-Glimmer-30B (Hugging Face)
Benchmark analysis and verification
Qwen 3.8 benchmarks: what’s actually verified (Yotta Labs) · Qwen 3.8 benchmark scores explained (Emergent) · Qwen3.8-27B specs & verdict (Kingy) · DeepSeek V4 Flash 0731 benchmarks & pricing (byteiota) · DeepSeek V4 Flash 0731 agent economics (NxCode) · Laguna S 2.1 coverage (VentureBeat) · Laguna S 2.1 analysis (MarkTechPost) · Muse Glimmer benchmarks (Artificial Analysis) · GLM-5.3 vs Kimi K3 vs DeepSeek V4
Running it locally
Qwen3.8 local guide (Unsloth) · DeepSeek-V4 local guide (Unsloth) · Qwen3.8-27B VRAM requirements, 13–54 GB · Qwen3.8-27B hardware guide (Kingy) · Dense 27B/31B on DGX Spark — real benchmarks (GitHub) · Choosing an inference engine on DGX Spark · DeepSeek V4 Flash on LM Studio
Hardware market
RTX 50 series median prices, August 2026 (TechPowerUp) · NVIDIA GPU price report, August 2026 (BuySellRam)