Turn detection is the task of deciding when one speaker has finished their turn and another, often a voice agent, should begin. It goes beyond detecting raw silence, weighing acoustic and linguistic cues to judge whether a person has actually finished a thought rather than simply paused mid-sentence. It is the conversational-timing problem at the heart of voice AI.
How turn detection works
Turn detection builds on voice activity detection and endpointing but adds meaning, modeling the turn-taking that people manage without thinking. A silence-only approach ends the turn the instant audio goes quiet, which fails on natural speech like "my number is 555, uh..." where the pause is not the end.
Better turn detectors read prosody, the rhythm and pitch of speech, together with the words recognized so far. Many now use a small model that judges whether an utterance is grammatically and semantically complete, so it holds the turn open through a thinking pause and releases it the moment the thought lands.
Why turn detection matters for voice agents
Turn detection is the difference between an agent that talks over you and one that leaves dead air. Set it too eager and the agent interrupts; set it too passive and every exchange drags.
It is one of the central user-experience problems in building a voice AI agent, and it works with barge-in, which handles the opposite case of a user cutting in while the agent speaks. On phone calls the stakes rise, because there is no visual cue, so the model has only sound and text to decide whether the caller is done.
How turn detection relates to endpointing
Endpointing and turn detection overlap and are sometimes used interchangeably, but a useful line separates them. Endpointing is the acoustic decision that speech has ended so a recognizer can finalize a transcript segment. Turn detection is the conversation-level decision about whose turn it is.
In transcription products the phrase "speaker turn detection" usually means marking where the active speaker changes, which is closely tied to speaker diarization. Good voice agents tune both endpointing and turn detection together, because a tight latency budget depends on ending turns promptly without cutting people off.
Frequently asked questions
What is speaker turn detection in transcription? Speaker turn detection is the detection of the points where the active speaker changes in a recording. In transcription this feeds speaker labels and is closely related to speaker diarization, which groups the audio by who spoke when.
How is turn detection different from endpointing? Endpointing detects the end of a stretch of speech acoustically. Turn detection is the higher call about when the floor should pass, using prosody and word content so it does not end on every pause.
Why does my voice agent keep interrupting me? Its turn detection is likely too aggressive, ending the turn on short silences. A detector that waits for grammatical and semantic completion, or a longer pause tolerance, usually fixes it.