Skip to content
Some content is members-only. Sign in to access.

Cheapest to Connect: Nemotron 3.5 and the Standardized AI Network

Nvidia repeats infrastructure history — win by lowering the cost of connection, from RTX desktops up to GB200 racks

By KAPUALabs

Nvidia's Nemotron 3.5 release is best read as an infrastructure play, not a model-leaderboard play. Nvidia announced an open-weight lineup 4 aimed at always-on enterprise agents 4. It has three models 3,13,16: Super VL, Lightning and the Bolt models 14,15. The thesis running through the line is that a mid-sized model, packaged to run anywhere from a data-center rack to a single local GPU, can do work that used to need a much larger one. Mixture-of-experts designs, aggressive quantization and open licensing recur across the line, and each lowers the cost of putting an agent into production.

The systemic view matters here. We have seen this pattern before in the history of infrastructure: the winning network is usually the one that is cheapest to connect to, not the one with the biggest central switch. The evidence below supports that reading on architecture and footprint. It is much thinner on independent validation of performance.

The Flagship Claim: Parity With a Much Larger Predecessor

Super VL is the largest of the new models 5. It has 120B total parameters with 12B active at a time 8,17,25,27,29,30, in a mixture-of-experts design 18,30. The 12B-active figure is among the best-corroborated facts in the material, with six sources 8,17,25,29. Super VL handles text, images and video 23 and offers a context window of up to 1M tokens 23,27,30. It is built for end-to-end agent workflows and tool interaction 30.

The strategic claim is that this model approaches Nemotron 3 Ultra, a 550B-parameter model with 55B active 2,24,29. Nvidia says Super VL matches Ultra on instruction-following and tool-use accuracy 6,7. A separate description puts it at about a fifth of Ultra's size on multimodal agent work 25. A post-trained Nemotron 3.5 Super is said to have roughly 4.6 times fewer active parameters than Ultra 29.

A tension worth naming

The sources disagree on wording. Some say Super VL matches Ultra. Others say it outperformed Ultra on the Artificial Analysis Intelligence Index 27. That result is reported as 26 points versus 23 for Ultra 27,30, and the article calls it the highest in its segment 30. The two statements can both be true if Super VL ties on agentic accuracy and leads on the aggregate index. They are different metrics, though, and the 26-point score is stated as run on B200 hardware 27,30. "Parity" is the safer summary. "Superiority" is a narrower, index-specific result.

How much weight the performance figures can bear

Most of the other performance claims come from the vendor. Nvidia says Super VL beats Opus 5, GPT 6.1 Sol, Luna and Astra on document analysis. This is the most widely repeated competitive claim, with five sources 6. A related claim, that it completes tasks faster than those models, holds only when the model is post-trained for specific agentic tasks 5,7.

Throughput is reported at 2.34 times that of Qwen3.5-122B-A10B on GB200, at 4,331.88 tokens per second per GPU 29. On H100 it is 988.87 tokens per second per GPU, 2.33 times the Qwen3.5 result 29. One blog author, citing the model card, gives 89.52 on GPQA-Diamond 23 and 71.10 on SWE-bench Verified 23. The post-trained model is reported at 51.78% on SWE-bench Pro 29. These figures are single-source, so they show direction and do not settle anything.

The training recipe fits the efficiency story. Nemotron 3.5 Super was initialized from Nemotron 3 Super 29, itself a 120B model 1,23. It used 1T tokens of large-scale distillation 29 and roughly 3.8 million post-training samples 10. Nvidia is refining an existing 120B base, not scaling up.

Where the Economics Show Up: Deployment Footprint

Memory arithmetic shows what "12B active" buys and what it doesn't. All 120B parameters must still be stored, so BF16 weights come to about 240 GB 23. Single-GPU deployment at BF16 is therefore not possible 23. Nvidia recommends 4x H100, which provides 320 GB 23, or 2x GB200 23. The NVFP4 version is about 60 GB 23. The active-parameter design helps speed, and quantization is what shrinks the memory bill.

The integration surface is conventional, which is a strength. The model is documented for vLLM and SGLang serving 23 and distributed through Hugging Face 23. It is also available as a NIM microservice 20 and reported to run on RTX PRO and Jetson 10. Standard serving paths mean less integration debt for an enterprise adopting it.

Pushing the Same Logic Down to Local Hardware

The smaller models extend the approach. Bolt 74B-A9B is a 74B-parameter MoE model with 9B active parameters 9,12. It is optimized for tool calling and multistep tasks 28 and aimed at local agentic work 7. Native 2-bit quantization lets it fit in 24GB 10. The smaller Bolt E4B targets a footprint of about 4 GB and runs on GeForce RTX 31. It was evaluated on 21 benchmarks 31. It ships in multiple checkpoint formats, including GGUF and drafter variants, via NVIDIA NIM 31. Its parameter count and active parameters were reportedly not disclosed 12.

Lightning is a 30B-total, 3B-active model 29. In one reported benchmark it was fastest per GPU in every row 26, with 151 t/s decode at 3 concurrent requests 26.

The naming and scope in the sources are somewhat noisy. One source calls the lightweight model "Bolt" but is unsure of the exact variant 4. Another says Nemotron 3.5 "shipped at 30B parameters" 12. That fits Lightning but not Super VL's 120B. An October 15 Nvidia blog is cited as describing an 816B MoE with 42B active parameters 21, and the material does not tie that to a named Nemotron 3.5 model. The year of the October 15 release date is also not stated 11. These points are unresolved. They don't undercut the main picture.

Openness as the Adoption Mechanism

Open licensing is the second pillar. Super VL ships with open weights 19,23, open datasets 10,19 and a training recipe 19. The OpenMDW-1.1 license allows distribution, according to four sources 23, and allows distillation 23. It describes outputs as unrestricted 23, and redistribution requires a license copy and notices 23. Companies can also continue training the model for specialized engineering tasks 30.

Adoption evidence is thin. Nvidia names EY, Ramp, CrowdStrike, Perplexity, Salesforce and Skydance as users 5. This is vendor attribution with no independent usage detail. Cline offers free access 30. One source notes that models built with Nemotron include closed models 22. That suggests the open stack also serves as an upstream for proprietary products. This is my interpretation, not something the source states.

What This Implies

Nvidia is competing on deployment economics, not headline scale. A 120B model with 12B active, said to match a 550B/55B predecessor, sits beside 24GB and 4GB models for local hardware. Together they keep agent workloads on Nvidia silicon from RTX and Jetson up to H100 and GB200, as an interpretation of the reported footprints. A license that permits distillation and redistribution invites the ecosystem to build on the line instead of around it. That is the logic of a standardized network: lower the cost of connection, and volume follows.

The weakest link is independent validation. The strongest comparative claims trace to Nvidia or to single posts. These include the document-analysis results against named rivals 6, the post-training speed claims 7 and the throughput multiples 29. Only the architecture and parameter facts are broadly corroborated 8,17,25,29. Enterprise buyers should run their own reliability testing on their own workloads before treating the parity claim as settled.

Takeaways

More from KAPUALabs

See all
| Free

AI Benchmarks Are Now a Measurement-Governance Problem

By KAPUALabs
/
| Free

Efficiency Gains or Moved Inefficiency?

By KAPUALabs
/
| Free

Nvidia Bull vs. Bear: Recovery Odds Meet Insider Selling

By KAPUALabs
/
| Free

NVIDIA at $216: The Bull and Bear Cases

By KAPUALabs
/