Available through Hugging Face and ModelScope, Nemotron 3.5 Transcribe combines capabilities that developers might otherwise handle through separate speech recognition and speaker identification systems. Users can control its output through natural-language instructions, allowing the same recording to produce different results depending on the task.
For example, a meeting recording can be converted into a transcript identifying each participant, translated into English or used to answer questions about what was discussed. Developers can also choose whether transcripts preserve verbal hesitations and spoken numbers or produce cleaner text with standardized formatting.
The model supports word-level timestamps and contextual vocabulary hints, which can help it recognize technical expressions, medical terminology and unfamiliar names. Nvidia recommends supplying possible spellings when necessary while instructing the system to use them only when supported by the recording.
Despite its broad language coverage, not every capability works across all supported languages. Audio question answering was trained in English, and speaker identification supports fewer languages than transcription.
Nvidia says the model delivers competitive accuracy while requiring relatively modest computing resources. Although Nemotron 3.5 Transcribe contains 30 billion parameters, approximately four billion are activated during each generation step, allowing it to operate on a single Nvidia GPU.
Supported hardware ranges from Jetson AGX Thor and DGX Spark systems to Blackwell data center GPUs. Developers can deploy the model using vLLM or run it locally through NeMo-Speech.cpp.
On Hugging Face's English Open ASR Leaderboard, Nvidia reports that Nemotron 3.5 Transcribe achieved a 5.33% average word error rate, placing it first among the open-weight models evaluated. The comparison does not include proprietary API models.
Separate testing by Artificial Analysis produced a 2.81% word error rate, compared with 2.77% for Mistral's Voxtral Small. Nemotron approached that result while activating substantially fewer parameters.
Speaker identification is another area where Nvidia reports strong performance. On David AI's multilingual benchmark, the model recorded an 18.26% speaker attribution error rate, the lowest among 14 evaluated systems across 21 languages.
In VoiceArena testing involving conversations with overlapping speech, Nemotron also led the evaluated systems in speaker-aware transcription accuracy. It omitted 8.32% of spoken words, compared with a benchmark median of approximately 31%.
Nvidia additionally reports that the model can process 100 hours of recorded audio in under three minutes using an RTX PRO 6000 Blackwell Server Edition GPU with NVFP4 precision. That result reflects high-volume batch processing under specific hardware and workload conditions, not the ability to transcribe conversations in real time.
Alongside the model, Nvidia released Granary v2, providing developers with multilingual speech data and tools for preparing their own training collections.
The public dataset manifest covers 463,062 hours of recordings spanning 48 languages and 16 source collections. A separate account describes the broader Granary v2 collection as containing approximately 844,000 hours. Nvidia distinguishes the publicly documented dataset from the complete mixture used to train Nemotron 3.5 Transcribe.
The accompanying processing pipeline generates several versions of each transcript, including verbatim text that retains filler words and normalized versions that remove them. It also produces translations, audio descriptions, vocabulary hints and question-and-answer examples derived from recordings.
Automated quality checks help identify inconsistent transcripts and potentially fabricated content. Public manifests provide information about data origins and quality indicators, although developers must obtain the underlying recordings separately and comply with their respective licensing requirements.
Nvidia is also providing NeMo Speech fine-tuning recipes for organizations that need more reliable recognition of specialized vocabulary.
In a medical transcription experiment involving synthetic speech, accented English recordings and medical audio, Nvidia found that customizing the model substantially improved recognition accuracy.
On a benchmark involving speakers excluded from the training data, the word error rate fell from 22.44% with the strongest prompting approach to 12.17% after fine-tuning, a relative reduction of 45.8%.
A broader evaluation showed improvement from 30.8% with the baseline approach to 11.8% after fine-tuning. Nvidia reports that the training process required approximately 14 GPU hours on a single GB300 system.
The improvements came with limitations. The specialized model performed slightly worse on general English recordings, and a separate medical speech model remained more accurate when recognizing brand-name medications. Nvidia recommends evaluating customized systems against at least 100 verified real-world recordings before deployment.
Nemotron 3.5 Transcribe also builds on Nvidia's earlier speech recognition releases. Its Nemotron 3 Diarization model, introduced in September, could distinguish speakers during live conversations but required a separate transcription system. The newer model integrates speaker identification with transcription, translation and audio understanding, although it sacrifices real-time processing.
The release follows other recent developments in voice AI, including Mistral's Voxtral 3, ElevenLabs' Realtime system and Salesforce's Launa-Omni model. Those products address different aspects of speech processing and conversational AI, including real-time voice interactions.
For developers, Nvidia's offering provides a way to combine several recorded-audio processing tasks within one adaptable model. Its open weights, accompanying datasets and fine-tuning tools offer options for organizations building transcription applications, particularly where specialized terminology matters.
The benchmark results indicate strong performance under the reported test conditions, but accuracy remains dependent on language, recording quality and the specific task. Nvidia advises developers to verify names, speaker assignments and generated answers against the original audio, particularly when transcripts are used in professional or sensitive settings.
This analysis is based on reporting from the decoder.
Image courtesy of Nvidia.
This article was generated with AI assistance and reviewed for accuracy and quality.
