Sessions

A session is one continuous voice conversation between a user and an agent. It begins with an HTTP POST, lives inside a LiveKit room, and ends when either side sends a close signal.

Session anatomy

Starting a session requires one HTTP call, carrying the session-config payload:

POST /api/v1/external/realtime/session/start
Authorization: Bearer <api-key or minted token>
Content-Type: application/json

{ "type": "session-config", "version": "1.0", "agent": { ... }, "session": { ... } }

The server responds with livekit_url, token, room_name, and session_id. Your SDK joins that room immediately — you never make this call by hand unless you're building a custom integration.

The control plane is a LiveKit reliable data channel inside the same room. JSON messages travel here — the ready handshake, transcripts, lifecycle events, tool call triples, errors. Audio travels over RTP tracks, not this channel. See Events for the full message map.

Session identity — every session has a server-assigned session_id, delivered on ready. Persist it if you want to resume the conversation after a disconnect (pass it back as session.experimental.resume_session_id on a fresh start).

The session-config payload

session-config splits into the two concerns of a session:

• agent — the persona: what the model is, independent of any one run. Either an inline definition or a reference to a workspace catalog agent by handle.
• session — per-run, transport-level options that vary run-to-run for the same agent.

Inline agent block (agent.type: "inline"): instructions, model (a model id, or a provider block carrying that provider's knobs), voice (name + speaking_style), audio (output, noise_cancellation), tools (client specs + server-tool opt-ins), greeting, interruption_sensitivity (default / high / low), and server hooks. Anything you omit gets a server default.

Catalog agent block (agent.type: "catalog"): name (the machine handle — workspace agents win on exact match; unmatched cosmo- names resolve from the built-in agent library, see resolution order) plus per-run ride-alongs only — inputs, client tools, and voice (speaking_style plus the one cosmetic name override). Stored-config fields such as instructions are structurally absent: sending one is a schema error, not a silent drop.

Session params:

The SDKs assemble this payload for you from the agent + start options:

import { RealtimeClient } from 'cosmo-ai';

const client = new RealtimeClient({ token: endUserJwt });

const agent = client.agent({
  instructions: 'You are Alex, a support agent at Acme.',
  voice: 'Puck',
  greeting: 'Hi, this is Alex — how can I help?',
  interruptionSensitivity: 'default',
});

const session = await agent.start();

from cosmo_ai import RealtimeClient

client = RealtimeClient(api_key="cosmo_...")

agent = client.agent(
    instructions="You are Alex, a support agent at Acme.",
    voice="Puck",
    greeting="Hi, this is Alex — how can I help?",
)

async with agent.start(store_recording=False) as session:
    async for event in session:
        ...

let client = RealtimeClient(apiKey: "cosmo_...")
let agent = try client.agent(
    instructions: "You are Alex, a support agent at Acme.",
    voice: VoiceConfig(name: "Puck"),
    greeting: "Hi, this is Alex — how can I help?"
)
let session = try await agent.start(storeRecording: false)

Server ready signal

Once the agent has joined the room and the model session is established, the server announces readiness two ways: a ready frame broadcast on the data channel, and the cosmo.ready participant attribute on the agent, which is room state a late joiner still reads. agent.start() waits for whichever lands first, so a started session is already ready. The frame:

{
  "type": "ready",
  "version": "1.0",
  "session_id": "0b9e9f9a-…",
  "agent": { "name": "support-triage", "tools": ["lookup_order", "end_call"] },
  "rejected_tools": [],
  "max_session_seconds": 3600
}

• session_id — the server-assigned id; persist it for resumption and support requests.
• agent — the resolved catalog-agent summary; null for inline sessions. Its tools list names each client tool, then each server tool's wire kind.
• rejected_tools — tools that were valid but are unavailable here, each with a reason: a server tool this deployment isn't configured for, or one gated off for this workspace. The session still starts without them.
• max_session_seconds — the effective server-enforced cap, so you can render a countdown; null = no cap.

Session end

Either side can end the session:

• Client — the SDK's end() sends the wire end frame; the server tears down the model session and closes. (close() tears down locally without the goodbye.)
• Server — sends session-ended with a stable reason slug (for example, max_session_duration) just before closing the room, usually preceded by session-ending-soon. A dead session can also end with an error frame with fatal: true.
• Network — the LiveKit room drops. LiveKit retries transient failures itself; if it gives up, the session is over.

Whatever the path, every SDK ends the event stream with a terminal session-ended item — see Lifecycle.

Pitfalls

• A session is single-attempt in every SDK: once it disconnects, it's terminal. Start a new session (optionally with resume_session_id) rather than reusing the old one.
• experimental.resume_session_id is an experimental knob and may change shape between releases.
• The greeting is voiced as soon as the model session opens, which can be before your client receives ready. A resumed session never re-greets.
• Tool problems fail the whole session start with a typed 422, whatever the kind — there is no soft rejection to fall back on. ready.rejected_tools reports the different case: a valid tool that isn't available on this deployment or workspace.