Accuracy
How accurate are Blumify transcripts?
We measured it, on 6.3 hours of podcasts, talks and earnings calls that have human transcripts. The most accurate model transcripts are 93.1% accurate on average: +2.9 points more than standard-model ones, and 29% fewer mistakes.
- Most accurate model
- 93.1%
- accurate · private transcripts, with bought credits or Unlimited
- Standard model
- 90.2%
- accurate · Free and Unlimited, made from the audio
- Videos with YouTube captions
- 94.3%
- accurate · every plan: the video's own captions, checked on 11 videos
Accuracy here is 100% minus the word error rate: the share of words that are wrong, missing or added, compared with a human transcript.
Results by type of recording
| Type of recording | Most accurate model | Standard model | Most accurate model is better by |
|---|---|---|---|
| Podcasts2.74 hours | 93.3% | 87.9% | +5.5 points |
| Talks1.53 hours | 97.4% | 95.9% | +1.5 points |
| Earnings calls2.05 hours | 88.6% | 86.9% | +1.7 points |
| Average | 93.1% | 90.2% | +2.9 points |
Videos that have captions use them, on every plan: YouTube's automatic captions were 94.3% accurate against the creators' own human captions.
What each plan uses
- Blumify Free and Unlimited use the video's own captions when there are any, and otherwise transcribe the audio with our standard speech model.
- The most accurate model is a choice for a private transcript when you have bought credits or Unlimited: it transcribes English audio with the most accurate speech model we tested, and uses AI credits. Audio in other languages uses the standard model, which handles more languages.
How we measured
- Public test sets with human transcripts: Rev16 (podcast episodes), TED-LIUM long-form (full talks) and Earnings-21 (company earnings calls), 6.3 hours in all, on 5 October 2026.
- Each recording went through the same steps as a real Blumify transcript: cut into 10-minute pieces, transcribed, and joined.
- In 2 of the podcast episodes, the ads in the recording are not the ads in the human transcript: podcasts insert ads when you download them, so the recording and the transcript got different ones. No model can match a transcript of ads that are not in the audio, so we left those stretches out, 14 to 16% of the words in those episodes, for every model alike. (Corrected on 6 October 2026: the first version of this page counted those ads as errors.)
- The measure is word error rate, the standard one, as on the Hugging Face Open ASR Leaderboard. Both texts were normalized with Whisper's English normalizer (case, punctuation, numbers, filler words), and notes such as <laugh> were removed from the human transcripts.
- The most accurate model is Deepgram Nova-3; the standard model is OpenAI Whisper large-v3 turbo. We also tested NVIDIA Parakeet TDT 0.6B v3 and OpenAI Whisper large-v3; neither was more accurate than Deepgram Nova-3 on average.
- The caption figure compares YouTube's automatic captions with the creators' own human captions on 11 videos (46,008 words), a different set of recordings.
What to keep in mind
- The human transcripts keep every repeat and false start. A transcript that tidies those up counts as wrong here, even when a reader would not mind. On the earnings calls, about 4 in 10 of the words the most accurate model “missed” are repeats such as “we we”.
- 6.3 hours is a small sample. Treat differences of a point or two as close.