Turn-taking

A voice conversation is a negotiation over who holds the floor. The server runs voice activity detection (VAD) and turn endpointing for you; this page covers the events it emits, the knobs that tune it, and the escape hatches when you want manual control.

The default loop

1. The user speaks. The server emits user-started-speaking, streams transcript deltas, then user-stopped-speaking.
2. The endpointer decides the turn is over and the model responds: bot-llm-started, bot-tts-started, bot-started-speaking, agent audio, bot-stopped-speaking, turn-complete.
3. If the user starts talking while the agent is speaking, the agent is interrupted — playback stops and the model yields the floor (barge-in).

You don't configure any of this to get a working conversation. You tune it when the defaults don't fit the room.

Interruption sensitivity

interruption_sensitivity on the agent config controls how readily user audio barges in: "default", "high", or "low".

• high — the agent yields at the first hint of user speech. Good for fast, cooperative assistants in quiet rooms.
• low — the agent holds the floor through short interjections and background noise. Good for phone calls, speakerphones, and noisy environments where echoes and "mm-hm" backchannels shouldn't cut the agent off.
• default — the provider's tuned middle ground.

Providers map this to their own VAD parameters. On Ultravox you can additionally set turn_endpoint_delay_seconds (0–5.0) in model_options to directly stretch or shrink how long the endpointer waits after the user stops before declaring the turn over — longer values tolerate slow, thoughtful speakers; shorter values feel snappier.

Noise cancellation

audio.noise_cancellation: true on the agent config applies background-voice cancellation to inbound audio before it reaches the model. Use it when the microphone will hear more than one voice — cafés, open offices, TV in the background. It reduces both false barge-ins and transcription of bystanders.

Manual turn boundaries

If your app knows better than the endpointer when a turn ends — push-to-talk, a hardware button, a form submit — send the boundary yourself:

await session.set_muted(False)     # open the floor
# … user holds the talk button …
await session.set_muted(True)
await session.activity_end()       # "the turn is over, respond now"

await session.activityEnd();

try await session.sendActivityEnd()

activity-end marks the end of the user's turn; it does not end the session. Combined with mute, this gives you full push-to-talk semantics: unmute while the button is held, then mute + activity-end on release.

Text turns have explicit boundaries by nature — send_text(...) is always a complete turn.

Silence handling

What should happen when the user says nothing for 45 seconds? Declare it, don't poll for it — a server-side silence-timeout hook runs even if your process dies mid-call:

from cosmo_ai.hooks import SilenceTimeout, Say, EndCall

agent = client.agent(
    instructions="…",
    hooks=[
        SilenceTimeout(timeout_seconds=30, action=Say(prompt="Gently check if the caller is still there."), max_count=2, reset_mode="on_user_speech"),
        SilenceTimeout(timeout_seconds=90, action=EndCall(farewell="I'll let you go — call back anytime.")),
    ],
)

The client observes each firing via the user-speech-timeout session event, which reports the silence duration, the trigger count, and the action the server already took.

Recommended defaults by use case