In speech-to-text, a hallucination is transcript text the model produces that was never spoken. Unlike an ordinary recognition error, it reads as fluent and grammatical, invented out of silence, background noise, or a quirk of the model. The system reports it with the same confidence it gives a correct transcript, and that false confidence is what makes it dangerous.
Why hallucination happens
Hallucination happens because end-to-end recognizers, and especially Whisper-style decoders built on language-model architectures, are trained to produce fluent text. That training is why they read so naturally, and on audio with no clear speech, a long pause, music, or steady noise, the model can still emit plausible words to fill the gap.
Whisper became well known for repeating a phrase or generating a stray sentence over silent stretches. The model is doing exactly what it was trained to do, predict likely text, even when there is nothing to transcribe. The same failure mode appears across generative AI, where hallucination refers broadly to confident, fabricated output.
Why hallucination is dangerous
A garbled error is obvious and easy to catch. A hallucination is not. It reads as a real sentence, so a person skimming the transcript or a downstream system parsing it gets no signal that the words are fabricated.
A hallucination can also hide inside a healthy-looking word error rate, because a few invented sentences may barely move a score averaged over a long file. In medical, legal, or compliance work, a confident fabrication is far worse than a visible mistake, since someone may act on words no one ever said.
How to reduce hallucination
The most direct guard against silence-triggered hallucination is voice activity detection: detect and skip non-speech regions so the model never runs on silence or noise in the first place.
Beyond that, check per-segment confidence scores and timestamps for spans that do not line up with real audio, avoid feeding long silent padding, and prefer provider settings or models that suppress output on non-speech. When accuracy is critical, sample transcripts against the source audio rather than trusting the aggregate score.
Frequently asked questions
What is a hallucination in speech-to-text? It is transcript text the model outputs that was never spoken. It looks like normal, fluent language, which is why it is harder to spot than a garbled misrecognition.
Why does the model hallucinate on silence? Models trained to produce fluent text keep predicting likely words even when the audio has no speech. Pauses, music, and steady noise are common triggers for invented output.
How do you reduce or prevent hallucination? Run voice activity detection to skip non-speech audio, inspect low-confidence segments and timestamps, avoid long silent padding, and choose models or settings that suppress output on non-speech regions.
How is a hallucination different from a normal transcription error? A normal error is a wrong or garbled word you can usually catch by ear or context. A hallucination is a fluent, confident sentence with no basis in the audio, so nothing in the text flags it as false.