Nvidia’s open transcriber tops English ASR at 5.33% WER

NVIDIA’s new open-weight Nemotron 3.5 Transcribe posts a 5.33% average word error rate (WER), first among open-weight models on Hugging Face’s English Open ASR Leaderboard with API models excluded, and it takes a plain-language instruction along with the audio, the company said in an October 28 NVIDIA announcement. If your team transcribes recorded audio full of domain terms, NVIDIA’s own advice is to try context in the prompt before fine-tuning. In its medical case study, naming the language and script cut WER from 30.8% to 23.8%, and fine-tuning brought it to 11.8%; the model handles recorded audio only and does not do live streaming.

One checkpoint covers transcription in 40 languages, translation of 31 of them to or from English, speaker labels in 13, and questions about the audio, which were trained in English only. It takes recordings up to an hour long and tells apart up to eight speakers. The weights are on Hugging Face and ModelScope under the OpenMDW-1.1 license, The News Stack reports, and run on a single NVIDIA GPU, served with vLLM or locally with NeMo-Speech.cpp.

Prompt first

The same meeting recording can come back as a labelled transcript, an English translation or an answer, depending on the instruction: “Transcribe this recording and label each speaker,” “Translate this audio into English,” or “What decisions were made in this meeting?” Prompts also choose between a verbatim transcript with fillers and a cleaned-up one, and set how numbers are written. Word-level timestamps are supported; Agora used them in a classroom demo that shows a recorded lesson’s transcript next to the replay.

For unfamiliar vocabulary, NVIDIA’s suggested healthcare prompt shows the pattern:

Transcribe this medical discussion in the original language. Possible terms: HbA1c, metformin. Use these terms only when supported by the audio. Preserve standard medical spelling and write numbers in conventional written form.

The model card carries recommended presets, including reasoning for audio questions, the full prompt guide and a task-by-language matrix, and that matrix is the place to look before planning around a language: support differs by task.

When the prompt runs out

The case study used Eka Care’s open benchmark of 3,619 clips from Indian doctors and medical students, which has no training split. So the adaptation data was built elsewhere: about 153 hours of synthetic speech generated from medical text and public medicine catalogs, 29 hours of accented English from Common Voice and 63 hours of medical recordings from MultiMed. Fine-tuning ran through NVIDIA’s NeMo Speech tools.

The headline 11.8% is on the full test set, which includes the 855 clips used for tuning. On held-out speakers, NVIDIA puts the gain at 45.8% relative to the best prompt; The News Stack gives the underlying figures as 22.44% WER with that prompt and 12.17% after fine-tuning.

Teams that want to build their own training set can start from Granary v2, the data release behind the model. Its public manifest covers 463,062 hours across 48 languages from 16 source corpora, up from 25 languages in Granary v1, and NVIDIA notes these figures describe the release, not Transcribe’s actual training mix. Each record carries three transcript forms (tn_raw, itn_raw, itn_clean), from fully verbatim to “I paid €25”, plus translations, biasing terms, provenance and a quality flag. The audio itself is downloaded separately under each corpus’s original license. The NeMo Curator pipeline that produced it, with Ray and NVIDIA Dynamo backends, can also be run on your own recordings.

What the leaderboards measure

Besides the English 5.33%, the model card’s Open ASR comparison gives 4.62% mean WER across seven languages and 10.98% on long-form audio, both the lowest among the open-weight models evaluated there. On the same English leaderboard, NVIDIA’s own Canary-Qwen-2.5B was billed as record-setting at 5.63% when it was released in June 2025, per the NeMo Speech repo. The new model is a 30B mixture-of-experts with about 4B parameters active per token, and The News Stack points out that traditional transcription models are often under a billion parameters.

On Artificial Analysis’s index, NVIDIA says the model ties for first among open-weight models at 2.8% AA-WER with about 4B active parameters, against 24B for the model it ties. The News Stack names that model as Voxtral Small and gives the figures as 2.81% for Nemotron against 2.77% for Voxtral, adding that the comparison counts active parameters, which says nothing about runtime or memory. On AA-AgentTalk, voice-agent speech that makes up half the index, Transcribe scores 1.8% against 2.3% for the next-best open-weight model. Mistral’s Voxtral and AssemblyAI’s Universal 3 Pro are the other promptable speech-to-text models The News Stack lists.

Speaker attribution is where NVIDIA tops the charts outright, though the runners-up are mostly its own. On David AI’s diarization track, 21 languages and 14 systems, Transcribe’s 18.26% diarization error rate edges NVIDIA’s dedicated Nemotron 3 Diarization at 18.38%. On VoiceArena’s English full-session benchmark with 2 to 8 speakers, it leads 14 systems at 32.4% cpWER, a word error rate scored after speaker labels are matched, ahead of NVIDIA Multitalker Parakeet with Nemotron 3 Diarization at 34.2%. It also drops the fewest words there: 8.32% of reference words, against a median of about 31%.

Hardware and live audio

Supported hardware runs from Jetson AGX Thor and DGX Spark to Blackwell data-center GPUs. On one RTX PRO 6000 Blackwell Server Edition at NVFP4 precision, NVIDIA measured 100 hours of audio processed in under three minutes, using 30-second clips at high concurrency.

For live audio, NVIDIA’s separate option is Nemotron-3.5-ASR-Streaming-0.6B, released in June with 40 languages and latency adjustable from 80 ms to 1 s. Whichever model ends up in the pipeline, NVIDIA’s own instruction for Transcribe is to spot-check names, speaker changes and answers against the recording.