Lifecycle

A Cosmo Realtime session progresses through a predictable sequence of events. Understanding the sequence lets you build accurate UI states and avoid race conditions.

State diagram

 agent.start() called
      │
      ▼
   connecting           (session-start POST + LiveKit room join)
      │
      ├──── ready ──────────────────────────────────► connected
      │                                                  │
      │                         ┌───────────────────────┤
      │                         │                        │
      │                    user speaks                user sends text
      │                         │                        │
      │                    user-started-speaking     (no event)
      │                         │                        │
      │                    user-stopped-speaking          │
      │                         │                        │
      │                    bot-llm-started ◄─────────────┘
      │                         │
      │                    (optional) tool-call
      │                         │
      │                    (optional) tool-dispatch-started
      │                         │
      │                    (optional) tool-result
      │                         │
      │                    bot-started-speaking
      │                         │
      │                    transcript (streaming, assistant)
      │                         │
      │                    bot-stopped-speaking
      │                         │
      │                    turn-complete
      │                         │
      │                    ◄────┘  (loops back to listening)
      │
      ├──── handshake rejected ─────────────────────► disconnected(handshake_failed)
      │
      └──── end() / session-ended / transport drop ─► disconnected

Phase by phase

1. connecting → ready

agent.start() performs the session-start POST and joins the LiveKit room. The first meaningful data-channel event is ready, which fires once the model session is established on the server side.

session.on('ready', ({ sessionId }) => {
  console.log(`Session ready: ${sessionId}`);
});

The connection lifecycle moves idle → connecting → connected. Transport-connected and agent-ready are distinct: the room can be up before the agent is ready to receive input.

Server-side VAD — voice activity detection, the model's "is the user talking right now?" check — fired. This is informational; the mic track keeps streaming regardless. Use it to show a "listening" indicator.

3. user-stopped-speaking

VAD detected silence. The model treats the turn as (probably) complete — tune this with interruption_sensitivity (see Turn-taking).

4. bot-llm-started

The model began generating. Use this to show a "thinking…" indicator before audio starts.

5. Tool call (optional)

If the model calls a tool, three events fire:

tool-call — model decided.
tool-dispatch-started — server handler began.
tool-result — handler returned.

All three share tool_call_id. After the final tool-result, the model continues generating. (Client-executed tools additionally surface tool-invocation — see Tools.)

6. bot-started-speaking → transcript → bot-stopped-speaking → turn-complete

The agent's audio starts. Transcript deltas stream. Audio ends. turn-complete marks the end of the assistant's turn and resets internal state for the next turn.

TypeScript events

RealtimeSession supports both callbacks and async iteration over the wire-level stream:

const session = await agent.start();

session.on('ready', ({ sessionId, rejectedTools, maxSessionSeconds }) => { /* session live */ });
session.on('lifecycle', (state) => { /* { kind, disconnectReason?, detail? } */ });
session.on('agent_state', (state) => { /* 'idle' | 'listening' | 'thinking' | 'speaking' */ });
session.on('transcript', (event) => { /* { turnId, role, text, isFinal, append } */ });
session.on('model_text', (event) => { /* { text, isFinal } */ });
session.on('tool_call', (event) => { /* { toolCallId, name } */ });
session.on('tool_result', (event) => { /* { toolCallId, ok, summary } */ });
session.on('turn_complete', ({ role }) => { /* end-of-turn marker */ });
session.on('reconnecting', ({ secondsRemaining }) => { /* upstream model rotation */ });
session.on('session_ending_soon', ({ secondsRemaining, reason }) => { /* countdown */ });
session.on('session_ended', ({ reason }) => { /* clean server teardown */ });
session.on('error', (error) => { /* { code, message } | null */ });

Or consume everything in order:

for await (const event of session) {
  // wire frames verbatim; `{type: 'unknown'}` for unrecognized types;
  // a `session-ended` item is always the final one
}

Python events

Python sessions are a single typed async stream — match on the event class:

from cosmo_ai import (
    RealtimeReady,
    RealtimeTranscriptDelta,
    RealtimeToolCall,
    RealtimeToolResult,
    RealtimeTurnComplete,
    RealtimeReconnecting,
    RealtimeError,
    RealtimeSessionEnded,
)

async with agent.start() as session:
    async for event in session:
        match event:
            case RealtimeReady():
                print("live, session:", event.session_id)
            case RealtimeTranscriptDelta():
                print(f"[{event.role.value}] {event.text}")
            case RealtimeToolCall():
                print("tool:", event.name)
            case RealtimeTurnComplete():
                print("turn over:", event.role.value)
            case RealtimeSessionEnded():
                print("ended:", event.reason)

Speech and processing markers (RealtimeUserStartedSpeaking, RealtimeBotStartedSpeaking, RealtimeBotLlmStarted, RealtimeBotTtsStarted, and their *Stopped counterparts) arrive on the same stream. Unrecognized frames surface as UnknownEvent and never terminate the stream.

Swift events

Swift sessions expose one typed AsyncThrowingStream — iterate it from exactly one task:

for try await event in session.events {
    switch event {
    case .ready(let ready):            print("live, session:", ready.sessionId)
    case .transcript(let delta):       print(delta.text)
    case .modelText(let text):         print("model text:", text.text)
    case .turnComplete(let turn):      print("turn over:", turn.role)
    case .userStartedSpeaking:         break
    case .botStartedSpeaking:          break
    case .botLlmStarted:               break
    case .toolCall(let call):          print("tool:", call.name)
    case .toolResult(let result):      print("done:", result.ok)
    case .reconnecting:                break
    case .error(let err):              print("error:", err.code, err.fatal)
    case .sessionEnded(let ended):     print("over:", ended.reason ?? "")
    case .unknown(let rawType, _):     print("unrecognized:", rawType ?? "?")
    default:                           break
    }
}

sessionEnded is always the final element; the sequence finishes after it. Start failures throw from RealtimeSession.start(_:config:) instead of appearing on the stream.

Connection state

In addition to the application-level events above, every SDK exposes the same formal connection lifecycle:

idle → connecting → connected ↔ reconnecting → disconnected

with the same typed end reasons once disconnected: client_ended (you called end()), client_closed (you tore down locally without the wire goodbye), handshake_failed (the server rejected the session start), server_ended (the server hung up on purpose), transport_error (the connection died).

Swift (typed AsyncStream)

Task {
    for await state in session.states {
        switch state {
        case .idle:              print("not connected")
        case .connecting:        print("session-start in flight")
        case .connected:         print("LiveKit room is live")
        case .reconnecting:      print("transient drop — LiveKit is recovering")
        case .reconnected:       print("recovered — same session, same tracks")
        case .disconnected(let reason):
            switch reason {
            case .clientEnded:   print("end() was called — wire end frame sent")
            case .clientClosed:  print("close() was called locally")
            case .handshakeFailed(let status, _): print("session-start rejected: HTTP \(status ?? 0)")
            case .serverEnded(let reason): print("server ended: \(reason ?? "no reason")")
            case .transportError(let message): print("transport error: \(message)")
            }
        }
    }
}

Swift's .reconnected is distinct from a fresh .connected so consumers can skip first-connect initialization after a transient recovery. .disconnected is terminal and the stream finishes — start a new session to reconnect.

Python (callback on agent.start())

from cosmo_ai import SessionState, SessionStateKind

def on_state(state: SessionState) -> None:
    if state.kind is SessionStateKind.DISCONNECTED:
        print(f"closed: {state.disconnect_reason}")   # DisconnectReason enum
    else:
        print(f"state: {state.kind.value}")

session = await agent.start(on_state_change=on_state)

TypeScript (getter + event)

session.on('lifecycle', (state) => {
  // state.kind: 'idle' | 'connecting' | 'connected' | 'reconnecting' | 'disconnected'
  if (state.kind === 'disconnected') {
    // state.disconnectReason: 'client_ended' | 'client_closed' |
    //   'handshake_failed' | 'server_ended' | 'transport_error'
    console.log('closed:', state.disconnectReason, state.detail);
  }
});

session.state;  // or read the current SessionLifecycleState synchronously

(The TypeScript engine additionally exposes a browser-UX transport_state axis with permission/ready/disconnecting phases; the lifecycle machine above is the cross-SDK vocabulary.)

Reconnect

The SDK does not automatically re-mint a session. Reconnect is layered:

Layer 1 — LiveKit's transient recovery (automatic, owned by LiveKit). Network blips, ICE renegotiation, signal-server reconnects all happen inside the LiveKit SDK without our involvement. The session_id, room, and tracks survive. While LiveKit is recovering, all three SDKs surface the in-progress state:

TypeScript: lifecycle kind becomes 'reconnecting', then back to 'connected' on success.
Swift: session.states yields .reconnecting, then .reconnected on success.
Python: on_state_change fires with SessionStateKind.RECONNECTING, then CONNECTED on success.

Surface this for "Reconnecting…" UI; no action required.

Layer 2 — LiveKit gave up (session is gone). If LiveKit's own retry budget exhausts, the lifecycle goes disconnected with reason transport_error and the event stream finishes with its terminal session-ended item. The session object is done in every SDK — mint a fresh session, optionally resuming the conversation:

session = await agent.start(resume_session_id=old_session_id)

See Reconnects for a full recipe.

The application-level reconnecting event (the server emits it while rotating the upstream model session) is distinct from both layers — the transport stays up, the session survives, and no lifecycle state changes. Show a brief indicator and stay put.

Pitfalls

bot-started-speaking fires before the first audio frame reaches the speaker. Do not use it to gate mic muting — use real AEC instead (see Audio).
turn-complete resets the turn ID in the TypeScript SDK. If you accumulate transcript items by turnId, flush on turn_complete.
The server does not send user-started-speaking for text turns — there is no audio to detect. bot-llm-started is the first marker after sendText().
The Swift events stream is single-consumer: iterate it from exactly one task, and consume session.states from a separate task if you need both.