Sessions

A session is one continuous voice conversation between a user and an agent. It begins with an HTTP POST, lives inside a LiveKit room, and ends when either side sends a close signal.

What it is

Starting a session requires one HTTP call, carrying the session-config payload:

POST /api/v1/external/realtime/session/start
Authorization: Bearer <api-key or minted token>
Content-Type: application/json

{ "type": "session-config", "version": "1.0", "agent": { ... }, "session": { ... } }

The server responds with livekit_url, token, room_name, and session_id. Your SDK joins that room immediately — you never make this call by hand unless you are building a custom integration.

The control plane is a LiveKit reliable data channel inside the same room. JSON messages travel here — the ready handshake, transcripts, lifecycle events, tool call triples, errors. Audio travels over RTP tracks, not this channel. See Events for the full message map.

Session identity — every session has a server-assigned session_id, delivered on ready. Persist it if you want to resume the conversation after a disconnect (pass it back as session.experimental.resume_session_id on a fresh start).

The session-config payload

session-config splits into the two concerns of a session:

• agent — the persona: what the model is, independent of any one run. Either an inline definition or a reference to a workspace catalog agent by handle.
• session — per-run, transport-level options that vary run-to-run for the same agent.

Inline agent block (agent.type: "inline"): instructions, model, model_options (provider-scoped knobs), voice (name + speaking_style), audio (output, noise_cancellation, and ambience — present enables the bed), tools (client specs + server-tool opt-ins), greeting, interruption_sensitivity (default / high / low), and server hooks. Anything you omit gets a server default.

Catalog agent block (agent.type: "catalog"): name (the workspace-unique handle) plus per-run ride-alongs only — inputs, client tools, and voice (speaking_style plus the one cosmetic name override). Stored-config fields such as instructions are structurally absent: sending one is a schema error, not a silent drop.

Session params:

The SDKs assemble this payload for you from the agent + start options:

TypeScript

import { RealtimeClient } from 'cosmo-ai';

const client = new RealtimeClient({ token: endUserJwt });

const agent = client.agent({
  instructions: 'You are Alex, a support agent at Acme.',
  voice: 'Puck',
  greeting: 'Hi, this is Alex — how can I help?',
  interruptionSensitivity: 'default',
});

const session = await agent.start();

Python

from cosmo_ai import CosmoRealtime

client = CosmoRealtime(api_key="cosmo_...")

agent = client.agent(
    instructions="You are Alex, a support agent at Acme.",
    voice="Puck",
    greeting="Hi, this is Alex — how can I help?",
)

async with agent.start(store_recording=False) as session:
    async for event in session:
        ...

Swift

let session = try await RealtimeSession.start(
    .init(apiKey: "cosmo_...", baseURL: URL(string: "https://app.askcosmo.ai")!),
    config: SessionConfig(
        voice: .init(name: "Puck"),
        instructions: "You are Alex, a support agent at Acme.",
        greeting: "Hi, this is Alex — how can I help?"
    )
)

Server ready signal

Once the agent has joined the room and the model session is established, the server sends ready over the data channel:

{
  "type": "ready",
  "version": "1.0",
  "session_id": "0b9e9f9a-…",
  "agent": { "name": "support-triage", "tools": ["lookup_order", "cosmo.end_call"] },
  "rejected_tools": [],
  "max_session_seconds": 3600
}

• session_id — the server-assigned id; persist it for resumption and support requests.
• agent — the resolved catalog-agent summary (name + effective tool names); null for inline sessions.
• rejected_tools — client-tool specs the server refused (sanitization, schema caps, name collisions), each with a reason. The session still starts without them. Unknown server-tool names reject the whole start with a typed 422 instead.
• max_session_seconds — the effective server-enforced cap, so you can render a countdown; null = no cap.

Session end

Either side can end the session:

• Client — the SDK's end() sends the wire end frame; the server tears down the model session and closes. (close() tears down locally without the goodbye.)
• Server — sends session-ended with a stable reason slug (e.g. max_session_duration) just before closing the room, usually preceded by session-ending-soon. A dead session can also end with an error frame with fatal: true.
• Network — the LiveKit room drops. LiveKit retries transient failures itself; if it gives up, the session is over.

Whatever the path, every SDK ends the event stream with a terminal session-ended item — see Lifecycle.

Pitfalls

• A session is single-attempt in every SDK: once it disconnects, it is terminal. Start a new session (optionally with resume_session_id) rather than reusing the old one.
• experimental.resume_session_id may change shape without a protocol version bump — it lives under experimental for a reason.
• The greeting is voiced as soon as the model session opens, which can be before your client receives ready. A resumed session never re-greets.
• Tool policy differs by kind: unknown server-tool names fail the whole session start (422), while invalid client-tool specs are soft-rejected and echoed on ready.rejected_tools.

See also

• Transport — the REST + LiveKit architecture
• Agents — client / agent / session tiers, inline vs catalog
• Lifecycle — the event sequence inside a session
• Tools — declaring client tools and opting into server tools
• Errors — what happens when session start fails