Voice activity detection (VAD) decides which parts of an audio signal contain speech and which are silence or background noise. It acts as a gate in front of, or inside, a speech-to-text system, marking the speech regions so the recognizer spends effort only on audio worth transcribing.
How voice activity detection works
Classic voice activity detection is cheap and rule-based. It measures signal energy, zero-crossing rate, and spectral shape, then applies a threshold to call each short frame speech or not.
It runs in microseconds and needs no training data, which is why it shipped inside telephony codecs and softphones for decades. The weakness shows up in noise: a fixed energy threshold cannot tell a raised voice from a slammed door.
Neural VAD replaces the threshold with a small network that outputs a speech probability for every frame, typically one every 10 to 30 milliseconds. Trained on labeled audio, it holds up far better in background chatter, music, and low signal-to-noise ratio conditions.
Either approach exposes the same tension: too sensitive and it labels noise as speech, too strict and it clips quiet or trailing words.
A brief history of voice activity detection
VAD predates modern speech recognition. Its logic goes back to time-assignment speech interpolation, a technique first used on transatlantic telephone cables around 1960 to carry more calls by reusing the silent gaps in one talker's speech for another's.
Digital VAD later became a named part of speech codecs. The ITU-T G.729 Annex B and the GSM standard both pair a voice activity detector with discontinuous transmission, sending comfort noise instead of encoding silence to save bandwidth.
Where voice activity detection fits
VAD sits at the front of a streaming pipeline and feeds several stages. It drives endpointing and turn detection, both of which depend on knowing when speech starts and stops.
It lets a system skip silence, which cuts compute cost and lowers the risk of a hallucination on non-speech audio. For long files it segments audio into utterances a batch model can process cleanly, and it is the first stage of most speaker diarization pipelines.
Why voice activity detection matters
Good VAD is quietly load-bearing. Clip the start or end of words and accuracy drops before the recognizer even runs. Leak noise through and the model wastes effort or transcribes garbage.
In a voice agent the VAD decision also adds to response latency, because the system cannot act on speech it has not yet detected. VAD quality tracks the audio's signal-to-noise ratio closely, which is why front-end noise suppression often improves recognition indirectly, by handing the detector a cleaner signal.
Frequently asked questions
What does VAD stand for? VAD stands for voice activity detection. It is the component that separates speech frames from silence and noise, usually at the very front of a speech-to-text or voice-agent pipeline.
What is the difference between VAD and endpointing? VAD is a frame-by-frame speech-or-not decision. Endpointing sits on top of it and decides when a whole utterance has ended, usually by watching for a sustained gap of silence after VAD stops flagging speech.
Is VAD the same as noise suppression? No. VAD detects where speech is; noise suppression tries to remove unwanted sound. They often run together, since cleaner audio makes the detector's job easier.