Cloud-dependent AI productivity agents carry real costs. Data leaves your infrastructure, latency compounds across API calls, and customization options stop wherever the provider draws the line.
NVIDIA’s Nemotron 3.5 family positions itself as a structural alternative, offering open-weight models that organizations can run, fine-tune, and operate on private hardware. The flagship is Nemotron 3.5 Super VL, a 120-billion-parameter multimodal model built for specialized, always-on agents.
NVIDIA says it completes agentic tasks up to five times faster than comparable open models, citing performance above 600 tokens per second on an Artificial Analysis agentic-task evaluation run through a Baseten endpoint. That figure is NVIDIA’s own claim and has not been independently confirmed.
What the Model Does
Super VL is built for complex, document-heavy workflows rather than conversational text generation.
This is not a chatbot. Super VL is engineered for parsing documents, reading charts and tables, executing multistep reasoning chains, and calling external software tools.
Its mixture-of-experts architecture is central to the throughput numbers. Out of 120 billion total parameters, only roughly 12 billion activate per token, cutting per-token computation substantially compared to a dense model of equivalent capacity.
NVIDIA reports the model was trained on approximately 26 trillion tokens. Post-training used roughly 3.8 million samples and reinforcement learning across dozens of environments.
Organizations NVIDIA identifies as working with or customizing the model include Scale AI, EY, AfterQuery, Hebbia, AI21, Alice, and Mercor, with use cases ranging from credit-agreement review to merger-and-acquisition due diligence.
NVIDIA claims up to five-times higher throughput than other open models in its size category for agentic tasks, measured on an Artificial Analysis agentic-task evaluation via a Baseten endpoint. The result has not been independently verified.
The Local Play
The Bolt models bring agentic AI down to hardware most organizations already own or can realistically deploy.
If the flagship is built for server rooms, the Bolt models are built for everywhere else. Nemotron 3.5 Bolt 74B-A9B carries 74 billion total parameters but activates only about 9 billion per token.
Native 2-bit quantization fits it into a 24GB memory footprint, targeting NVIDIA RTX and DGX Spark systems. NVIDIA says it sustains at least 80 tokens per second.
It is available now through Hugging Face, OpenRouter, ModelScope, and NVIDIA NIM. Runtime support includes llama.cpp, vLLM, Ollama, Unsloth, and LM Studio.
Going smaller still, Bolt E4B targets a roughly 4GB footprint for voice assistants, interactive characters, and physical-AI systems running on platforms including NVIDIA Jetson. NVIDIA lists its availability for October 23.
Running these models locally can give your organization more control over sensitive data, but the privacy and compliance benefits are not automatic. Configuration, access controls, logging practices, and your broader security posture all determine whether “local” actually means “protected.”
What the Real Test Looks Like
Benchmark numbers tell one story; sustained real-world workloads will tell another.
Benchmark throughput is one data point. What actually matters for enterprise AI is whether these models hold up outside controlled conditions. That means sustained multi-hour workloads, tool-use reliability under concurrency, graceful failure recovery, and answer quality after fine-tuning on proprietary data.
If the performance claims hold across those conditions, Nemotron 3.5 could meaningfully shift how organizations approach deploying agents on private infrastructure. Until independent evaluations confirm the numbers, treat NVIDIA’s figures as a starting point for your own testing, not a purchasing decision.
