Nvidia’s open transcriber tops English ASR at 5.33% WER

Nemotron 3.5 Transcribe, an open-weight model NVIDIA released on October 28, averages 5.33% word error rate on Hugging Face’s English Open ASR Leaderboard, the best of any open-weight model there. It also puts transcription, speaker labels, translation and audio Q&A into one checkpoint that you steer with a plain-language prompt, NVIDIA wrote in its announcement. If your recordings are full of domain terms, NVIDIA’s own guidance is to try putting context in the prompt before you fine-tune. The model is built for recorded audio up to an hour long and does not handle live streams.
Prompt first, fine-tune second
The same checkpoint can turn one meeting recording into different outputs depending on the instruction: “Transcribe this recording and label each speaker,” “Translate this audio into English,” or “What decisions were made in this meeting?” For jargon, NVIDIA’s example prompt for a medical discussion lists candidate terms (“Possible terms: HbA1c, metformin”). It tells the model to use them only when the audio supports them and asks for standard medical spelling and written-form numbers. The prompt can also ask for a verbatim or a cleaned-up transcript and set how numbers are formatted, The News Stack notes.
NVIDIA’s medical case study shows how far each step gets you. It uses Eka Care’s open benchmark of 3,619 clips recorded by Indian doctors and medical students. Naming English and the Latin alphabet in the prompt cut WER on the full set from 30.8% to 23.8%. Fine-tuning with NeMo Speech then brought it to 11.8%, using about 153 hours of synthetic speech, 29 hours of accented English from Common Voice and 63 hours of medical recordings from MultiMed. The full set includes the 855 clips used for tuning. On held-out speakers, The News Stack reports, WER fell from 22.44% with the best prompt to 12.17% after fine-tuning, the 45.8% relative cut NVIDIA cites.
For training data, NVIDIA also released Granary v2: the curation pipeline and multilingual data behind part of Transcribe’s training. It produces each transcript in verbatim, normalized and clean forms, the same styles you can request with a prompt. The public manifest covers 463,062 hours across 48 languages, and NVIDIA says those figures describe the dataset release, not Transcribe’s training mix. Our count from the dataset card finds that 82.2% of those hours are VoxPopuli parliament audio with MOSEL pseudo-labels (machine-generated transcripts). Half the languages, 24 of 48, draw only on FLEURS and/or Common Voice 26, and those two corpora hold 4,283 hours in total across all languages. You download the source audio separately under each corpus’s own license.
The language matrix limits what a prompt can ask for. The model transcribes 40 languages, translates 31 of them to or from English, and labels speakers in 13, with up to eight speakers per recording. Audio question answering is trained in English. We checked those counts against the model card’s matrix, and they match. Word-level timestamps are supported too: Agora used them to show a recorded class’s transcript alongside its replay. NVIDIA itself advises spot-checking names, speaker changes and answers against the recording.
On speaker attribution, NVIDIA reports the lowest diarization error rate among 14 systems on David AI’s 21-language leaderboard: 18.26%, slightly ahead of its own dedicated Nemotron 3 Diarization model at 18.38%. Sortformer speaker cues reach the language model together with the acoustic features, so the model can tag each turn while it generates the transcript, rather than relying solely on a later pass. On VoiceArena’s English full-session benchmark, it leads the same number of systems at 32.4% cpWER.
What it takes to run
The weights are on Hugging Face and ModelScope under the OpenMDW-1.1 license, which The News Stack reports. NVIDIA says the model runs on a single NVIDIA GPU, from Jetson AGX Thor and DGX Spark up to Blackwell data-center GPUs, served with vLLM or locally with NeMo-Speech.cpp. It is a 30B mixture-of-experts model with about 4B parameters active per token, or about 4.3B counting the audio encoder, per the model card. Traditional transcription models are often under a billion parameters.
NVIDIA’s headline speed figure is 100 hours of audio in under three minutes on one RTX PRO 6000 Blackwell Server Edition in NVFP4, a 4-bit number format, measured on 30-second clips at high concurrency. According to the model card, that figure comes from the optimized vLLM build inside the Nemotron Transcribe NIM, with DFlash2 speculative decoding. The card says those optimizations are not yet part of the open-source release and expects the open vLLM integration to be roughly two times slower.
We inspected the NeMo-Speech.cpp repository as of October 30 but did not run it. Local inference is shipped code, but there is no pre-built GGUF file. You download the 67.5 GB BF16 checkpoint and convert it yourself: to NVFP4 for Blackwell, or to IQ4_NL for other GPUs, the CPU and Vulkan. The default settings fit a GPU with 32 GB of memory. The 24 GB profile caps recordings at 29 minutes and uses an IQ4_NL decoder. CPU and Vulkan run without speculative decoding (this is from the docs; we did not measure it). Speaker labels and translation work there, but phrase boosting does not: the docs say speech_contexts phrases and the HTTP prompt field are not supported.
For live audio, NVIDIA’s NeMo documentation already lists a separate model, Nemotron-3.5-ASR-Streaming, described as “Real-time streaming, 40 languages.”
Our analysis
What we checked
On November 1 we pulled the live Open ASR Leaderboard (the October 30 version), Artificial Analysis’s speech-to-text page data and the model card, and recomputed the averages.
The English claim holds: no open-weight model beats 5.33%. But Qwen3-ASR-1.7B trails by only 0.07 points, and the average includes two private datasets held by the leaderboard maintainers; on the nine public sets alone it is 4.70%. Two other claims are narrower than NVIDIA’s post makes them. The long-form “lowest” holds only in the model card’s own comparison table. And Artificial Analysis lists Nemotron second among open-weight models, behind Voxtral Small, where NVIDIA describes a tie for first.
| Board | NVIDIA’s claim | Live figure | Live rank | Nearest open-weight rival |
|---|---|---|---|---|
| Open ASR, English | 5.33%, first open-weight | 5.33% | 9th of 63, all eight above are API models | Qwen3-ASR-1.7B, 5.40% |
| Open ASR, multilingual | 4.62% | 4.69% | 5th, four above are API models | Qwen3-ASR-1.7B, 6.56% |
| Open ASR, long-form | 10.98%, lowest open-weight | 10.52% | 4th | parakeet-ft-v1, 9.81% (ahead) |
| Artificial Analysis AA-WER | 2.8%, tied first | 2.8% | 2nd open-weight | Voxtral Small, 2.8% (ahead) |
- The long-form leader among open-weight models, freetranscribe/parakeet-ft-v1, is a CC-BY-4.0 fine-tune of NVIDIA’s own parakeet-tdt-0.6b-v3. It was created on October 13 and added to the board on October 26.
- On Artificial Analysis, Voxtral Small’s unrounded score is 2.77%. We found no 2.81% figure for Nemotron, the number The News Stack gives, in the page data. The only Nemotron host record there is the DeepInfra endpoint, which scores 3.4%.
Up next
Claude model wrote a no-login Linux root exploit
An Ubuntu or Red Hat server with a multipath TCP listener faces a no-login root exploit written by an unreleased Claude model, and per Golem only Debian had shipped a patch. Are your Linux servers patched automatically or by hand?
Related stories
- Nemotron 3.5 Super VL gives NVIDIA’s 120B agent model eyes
- Nvidia launches a smaller, faster Nemotron model and a router to put it to work
- WSJ: Nvidia’s board is preparing to oust Jensen Huang
- Kimi K3.1 Pro weights land on HuggingFace at 2.8T params
- Tencent ships Hy4 at Hy4 preview’s API price
- FLUX 3 Image Klein’s 5.6B count leaves out its encoder
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

