The headline Nemotron 3.5 Super VL benchmark result is 63.44% on Hermes Bench, ahead of the 56.33% NVIDIA reports for its own bigger Nemotron 3 Ultra. It also scores 87.05% on LangChain evals and 80.8% on CharXiv chart reasoning. Every one of those numbers comes from NVIDIA. Below, I explain what each test actually measures in plain English, so you know what the scores mean for your work.
📺 Watch: NVIDIA Nemotron 3.5 Super VL Is INSANE! (FREE AI Model)
🔥 Get the Agent OS as a free bonus: AI Profit Boardroom members get the full Agent OS zip, prompt libraries, daily tutorials and weekly live coaching calls. → Get inside
I'm going to keep this simple. A benchmark is just a test. The score only matters if the test looks like the job you want done. If you want to put this model to work inside a real agent setup, that's what we do inside the AI Profit Boardroom.
Nemotron 3.5 Super VL Benchmark Results Explained
Here's every benchmark NVIDIA published in its technical blog on 15 October 2026, with what each one tests and what it was compared with.
| Benchmark | What it tests | Super VL | Compared with |
|---|---|---|---|
| Hermes Bench | How well a model works as an agent inside Hermes: following steps, using tools, finishing tasks | 63.44% | Nemotron 3 Ultra: 56.33% |
| LangChain evals | Agent tasks run through LangChain, the popular framework for chaining model calls and tools | 87.05% | Nemotron 3 Ultra: 84.3% |
| CharXiv | Reading and reasoning about charts from scientific papers | 80.8% (84% with Python code interpretation) | Nemotron 3 Nano Omni: 63% |
| OCRBench v2 | Reading text out of images: documents, signs, scans, screenshots | Among the top open models | No exact number given |
| SWE-bench | Fixing real bugs in real code repositories | Beats Nemotron 3 Super | No numbers given |
| Terminal-Bench | Getting jobs done in a command-line terminal | Beats Nemotron 3 Super | No numbers given |
| MILU 1.5 | General knowledge in Indian languages | Leads | Sarvam-105B, Gemma-4-31B-It |
| Indic GPQA | Hard graduate-level science questions in Indian languages | Leads | Sarvam-105B, Gemma-4-31B-It |
| Indic AIME | Competition maths in Indian languages | Leads | Sarvam-105B, Gemma-4-31B-It |
| Indic FrontierMath | Very hard research-level maths in Indian languages | Leads | Sarvam-105B, Gemma-4-31B-It |
| Indic MBPP | Writing basic Python programs from prompts in Indian languages | Leads | Sarvam-105B, Gemma-4-31B-It |
A note on that last block. NVIDIA says the model leads Sarvam-105B and Gemma-4-31B-It on the Indic benchmarks, but the text doesn't give the scores. So I'm not going to make any up. If you want the head-to-head with Google's model, I wrote up Nemotron 3.5 Super VL vs Gemma 4.
What the Agent Scores Tell You
Hermes Bench and LangChain evals are the two that matter most if you run agents. They test whether the model can take a goal, use tools, and finish the job. The interesting bit is the comparison. Super VL is the smaller model, distilled from Nemotron 3 Ultra, yet NVIDIA reports it beating Ultra on both. That's a 7-point gap on Hermes Bench and roughly 3 points on LangChain. The full Super-vs-Ultra breakdown is in Nemotron 3.5 Super VL vs Nemotron 3 Ultra.
What the Vision Scores Tell You
CharXiv and OCRBench v2 are the vision tests. CharXiv asks the model questions about real charts, the kind of thing you'd find in a report or slide deck. Jumping from 63% on Nano Omni to 80.8% is a big move, and 84% when it can write Python to work things out. OCRBench v2 is about pulling text out of images. NVIDIA puts it among the top open models there. For you, these two say: screenshots, PDFs, charts and receipts are where this model earns its keep.
What the Coding Scores Tell You
SWE-bench and Terminal-Bench test coding and command-line work. NVIDIA says Super VL beats Nemotron 3 Super on both setups, but publishes no numbers in the text. So treat that as "better than the last Super", not as a claim about frontier coding models.
📺 Watch: OpenClaw + Nvidia Nemotron 3 Super + Ollama is INSANE!
Speed Benchmarks
Speed matters as much as accuracy for agents, because agents make lots of calls. Here's what NVIDIA reports.
| Speed test | Setup | Result (per NVIDIA) |
|---|---|---|
| Single-GPU throughput | Agentic tasks vs open models its size | Up to 5x higher |
| Long-input generation | Agentic tasks at 100K input tokens, Baseten endpoint | Over 600 tokens per second |
| Coding task completion | 10,000 coding tasks on B200 | 4x faster |
| Speculative decoding | DGX Spark with LiLiCorr | Up to 2.3x faster generation |
| Multi-user serving | Dynamo on 16 B200 GPUs at 50 tokens/s per user | Up to 8x as many users, 85% lower time to first token |
Why is it fast? Three reasons. It's mixture-of-experts, so only 12B of its 120B parameters work on each token. It uses a hybrid Mamba and attention architecture built for long inputs. And it has multi-token prediction, where an internal drafter guesses several tokens and checks them in one pass. NVIDIA also ships NVFP4 checkpoints with quantization-aware distillation, which run 4-bit on Blackwell GPUs.
NVIDIA also cites customer results: Scale AI (Skydance) reporting 40x lower inference cost and 5x lower latency, Ramp expecting up to 50% higher throughput and 10x lower latency, and AI21 and Alice reporting a 41% accuracy improvement. In my video I also mentioned one company's claim of 10,500 tokens per second across users on a single DGX Station. That's their own case study claim, not something I've measured.
📺 Watch: New FREE Nvidia Nemotron 3 Super + Ollama
The Big Caveat: These Are NVIDIA's Numbers
Every benchmark score on this page comes from NVIDIA. That's normal for a launch, but it matters. The company that built the model picked the tests, ran them, and wrote them up. No independent replication of the full benchmark set exists yet.
That doesn't mean the numbers are wrong. It means you shouldn't treat them as settled. Wait for others to run the same tests, and more importantly, run your own.
The One Independent Check So Far
There is one outside evaluation worth knowing about. EXO Labs ran the model on a 128GB DGX Spark and put it on the Pareto frontier for accuracy against task time. In plain English: for that hardware, nothing they tested gave you more accuracy for the same time, or the same accuracy faster. It's one evaluation on one machine, but it's not NVIDIA marking its own homework.
How to Test It Yourself
Here's the bit most people skip. Public benchmarks tell you how a model does on someone else's tests. What you actually care about is how it does on your jobs. That's the idea behind Goldie Bench: run the same real tasks through every model and compare the output side by side.
Here's a simple way to do it with this model:
- Pick five real tasks. Use things you actually do: read a receipt, summarise a PDF, explain a chart, fix a screenshot of an error, build a small app.
- Get access free. Pick Nemotron 3.5 Super VL in the OpenCode model drop-down, or use the :free variant of nvidia/nemotron-3.5-super-vl on OpenRouter (rate limits apply).
- Run the same prompts through Super VL and the model you use now.
- Score three things: did it get it right, how long did it take, and how much did you have to fix.
- Keep it for what it wins. Slot it in for those jobs and keep your other model for the rest.
When I did this, I asked it inside OpenCode to build a beautiful to-do list app and make it entertaining and engaging. It came out working. My take: it's not a frontier model and it won't replace Opus 5.5, but it's free, fast, and it reads images and files well. That lines up with what the benchmarks point to.
My full setup for swapping models in and out across Hermes, OpenClaw, OpenCode and Claude is in the Agent OS guide. And for everything else about the model, start with the main Nemotron 3.5 Super VL guide.
If you want help working out which models belong in your business, book a free AI strategy session.
FAQ
What is the best Nemotron 3.5 Super VL benchmark score?
The standout is 63.44% on Hermes Bench, beating Nemotron 3 Ultra's 56.33%, per NVIDIA. Its 87.05% on LangChain evals and 80.8% on CharXiv are also strong.
Are the benchmarks independently verified?
No. All the scores are NVIDIA's own. The only outside check so far is EXO Labs' evaluation on a 128GB DGX Spark, which put it on the accuracy and task-time Pareto frontier.
Does it beat Nemotron 3 Ultra?
On Hermes Bench and LangChain evals, NVIDIA says yes, even though Super VL is smaller and was distilled from Ultra.
How fast is it?
NVIDIA reports over 600 tokens per second on agentic tasks at 100K input tokens on Baseten, and up to 5x higher single-GPU throughput than open models its size.
Want to run your own tests inside a ready-built Agent OS, with a 30-day roadmap, 4 weekly coaching calls and daily tutorials? Join 3,400+ members in the AI Profit Boardroom for $69/mo. Or talk it through first on a free AI strategy session. Either way, treat every Nemotron 3.5 Super VL benchmark as a starting point, then test it on your own work.
