EvalSignal 015 / Independent model evaluation

NEX N2.5 Pro is cheap. Is it actually a better agent planner?

One hundred frozen browser-planning calls cost about $0.15. Every request completed, and 95 produced a native structured tool call. The tradeoff appears in the next decision: NEX N2.5 Pro did not match the first-action precision of the stronger saved reference.

What NEX N2.5 Pro is

Nex AGI positions NEX N2.5 Pro as a multimodal agentic model for long-horizon computer use, browsing, coding, and visually grounded workflows. The released configuration uses a Qwen3.5 mixture-of-experts architecture with roughly 397 billion total parameters, 60 text layers, 512 routed experts, and 10 experts selected per token. Its configured context window is 262,144 tokens.

Mixture-of-experts routing reduces the computation used for each token; it does not make the full checkpoint small. Our test used the hosted FP8 route on OpenRouter. We did not load the model locally, benchmark GPU throughput, or test images. This result is strictly about text-based first-tool planning.

The price is the headline

That observed run cost was roughly one seventh of the saved MiniMax M3 run. Provider pricing, caching, tokenization, and output length can change the total, but the gap is large enough to matter in agent loops that make many planning calls.

What the frozen test measures

The suite contains 100 saved first-turn browser-agent prompts using the same historical Sonnet 4.6 system prompt and 41-tool schema. The model sees the same browser states and instructions, then we record only its first response. Tools are not executed, so this measures routing discipline rather than end-to-end task completion or recovery.

Structured calls asks whether the response contained a native machine-readable tool call.

Exact ideal requires both the archived reference tool and every argument to match.

Ideal name checks only the selected tool name.

Sonnet alignment checks agreement with the saved Sonnet 4.6 first choice, including agreement not to call a tool. It is a consistency reference, not an independent correctness oracle.

The result

The operational layer worked. There were no transport failures, every parsed call used a known tool name, and no response hit the output limit. The weaker signal was judgment: producing a valid call did not consistently mean choosing the best first action.

Cheap, but not the strongest planner

Both rows use the same frozen prompt, tool schema, and archived scoring references. NEX N2.5 Pro emitted more structured calls, but MiniMax M3 won the action-quality columns and was faster at the median: 3.10 seconds versus 7.23 seconds.

Cost and latency were observed on provider routes at different dates, so they are not clean architecture comparisons. The planner scores are the stronger apples-to-apples signal: scale and call frequency did not translate into first-action leadership.

The inspect-first habit

NEX N2.5 Pro selected get_accessibility_tree 59 times. Inspection is often sensible, but the model also inspected when the frozen reference called for direct navigation or clarification. It aligned with Sonnet on all eight form cases while matching the archived ideal tool on none of them. On ambiguity cases, it matched Sonnet once in eight and the ideal tool twice.

This is the practical limitation. A recoverable agent loop may tolerate an extra inspection call, especially at this price. A latency-sensitive workflow or a system that expects decisive routing will pay for that caution in extra turns.

Bottom line

NEX N2.5 Pro makes a strong economic argument, not a benchmark-winning one. It is inexpensive, operationally reliable, and disciplined about native tool-call syntax. Those qualities make it worth testing in high-volume agents where a controller can recover from conservative first actions.

For workflows where the first decision matters more than call price, the saved MiniMax M3 result remains stronger. The right use of NEX N2.5 Pro is not "largest model wins." It is "the planner is cheap enough that an extra turn may be acceptable."

Sources

NEX N2.5 Pro model card  |  OpenRouter route  |  Full benchmark and methodology

Prices and provider routes can change. The suite records only the first response; tools were not executed, and no vision inputs were tested.