Imagine attending a one-hour meeting and having an AI tool turn the entire conversation into text without you having to type a single word. That’s the kind of work AI transcription tools are helping people handle, and Nvidia has just introduced a new model to make this even easier.
On Wednesday 28/10/2026, Nvidia released Nemotron 3.5 Transcribe, a free, open-weight AI model that can convert speech into text in up to 40 languages. The company also released version 2 of Granary, its open-source speech dataset, which developers can use to build and improve speech AI tools.
The checkpoint nvidia/nemotron-speech-streaming-en-0.6b on Hugging Face combines a cache aware FastConformer encoder with an RNNT decoder, and is tuned for both streaming and batch workloads on modern NVIDIA GPUs.
Model design and architecture
Nemotron Speech Automatic Speech Recognition is a 600M parameter model based on a cache aware FastConformer encoder with 24 layers and an RNNT decoder. The encoder uses aggressive 8x convolutional downsampling to reduce the number of time steps, which directly lowers compute and memory costs for streaming workloads. The model consumes 16 kHz mono audio and requires at least 80 ms of input audio per chunk.
Runtime latency is controlled through configurable context sizes. The model exposes 4 standard chunk configurations, corresponding to about 80 ms, 160 ms, 560 ms and 1.12 s of audio. These modes are driven by the att_context_size parameter, which sets left and right attention context in multiples of 80 ms frames, and can be changed at inference time without retraining.
But what makes this New Model Stand out?
According to Nvidia, Nemotron 3.5 Transcribe can translate 31 languages to and from English, process audio recordings that last up to an hour, and identify up to eight different speakers in a conversation.
The AI can listen to a recorded meeting, turn the conversation into written text, and distinguish between different people speaking. This can save time for anyone who regularly spends hours listening to recordings and typing out what was said.
Nvidia also claims that its model performs better than competing tools on several benchmarks, including Hugging Face’s Open ASR Leaderboard, a platform used to compare the performance of automatic speech recognition systems.
Throughput and Concurrency on Modern GPUs
The cache aware design has measurable impact on concurrency. On an NVIDIA H100 GPU, Nemotron Speech ASR supports about 560 concurrent streams at a 320 ms chunk size, roughly 3x the concurrency of a baseline streaming system at the same latency target. RTX A5000 and DGX B200 benchmarks show similar throughput gains, with more than 5x concurrency on A5000 and up to 2x on B200 across typical latency settings.
Equally important, latency remains stable as concurrency increases. In Modal’s tests with 127 concurrent WebSocket clients at 560 ms mode, the system maintained a median end to end delay around 182 ms without drift, which is essential for agents that must stay synchronized with live speech over multi minute sessions.
So, Who can Benefit from this Technology?
Developers building tools for businesses could find it particularly useful. For example, a company could use it to develop an app that transcribes business meetings, while healthcare developers could explore ways to support medical transcription.
Nvidia says developers can combine the model with the updated Granary dataset and tools such as NeMo Speech to customise their applications. This means they can work on making their AI tools recognise industry-specific terms and jargon in different languages.
However, there is one limitation: Nemotron 3.5 Transcribe works with recorded audio, not live audio streams. So, developers looking to build tools that transcribe conversations as they happen will need to consider other options.
As more businesses look for ways to automate repetitive tasks, speech AI is becoming an increasingly useful part of workplace technology. Nvidia’s latest release gives developers another tool to explore, especially when building applications that need to understand and transcribe conversations across multiple languages.
