Clients, agents, and sessions

All three SDKs are built around the same three nouns:

The split matters because the pieces vary at different rates: your credential is process-wide, your persona is designed once and reused, and a session is one conversation. If a value wouldn't differ between two runs of the same agent (voice, greeting, tool set), it belongs on the agent. If it varies run-to-run (resume id, whether to record), it's a session option.

const client = new RealtimeClient({ token: endUserJwt });
const agent = client.agent({
  instructions: 'You are Alex, a support agent at Acme.',
  voice: 'Puck',
  tools: [lookupOrder],
  greeting: 'Hi, this is Alex — how can I help?',
});
const session = await agent.start();

client = RealtimeClient(api_key=os.environ["COSMO_API_KEY"])

agent = client.agent(
    instructions="You are Alex, a support agent at Acme.",
    voice="Puck",
    tools=[lookup_order],
    greeting="Hi, this is Alex — how can I help?",
)

async with agent.start(store_recording=False) as session:
    async for event in session:
        ...

let client = RealtimeClient(token: endUserJwt)
let agent = try client.agent(
    instructions: "You are Alex, a support agent at Acme.",
    voice: VoiceConfig(name: "Puck"),
    tools: [lookupOrder],
    greeting: "Hi, this is Alex — how can I help?"
)
let session = try await agent.start(storeRecording: false)

Inline agents

client.agent(...) defines the persona in code. Everything is explicit: instructions, model (a model id, or a provider block carrying that provider's knobs), voice, tools, greeting, interruption_sensitivity, audio, skills, hooks. How the agent sounds rides under voice (name, speaking_style); the audio pipeline rides under audio (output, noise_cancellation). Anything you omit gets a server default (for example, the workspace's default model and voice) — audio.noise_cancellation among them, which is off. Pass denoise when several people share the microphone, or voice_focus when one speaker should be isolated from the rest of the room.

An agent is immutable once built and opens any number of sessions. To vary a persona field, build another agent — the factory call is cheap, and the two are independent.

Naturalness presets

Writing a good speaking_style from scratch — contractions, turn length, interruption manners — is prompt engineering most agents repeat. The TypeScript SDK ships three tuned presets as plain instruction text: naturalness('warm' | 'delivery' | 'human') resolves a rung to the verbatim speakingStyle string, so you can use one as-is or as a starting point to edit. All three set register, pacing, and turn length; the rungs differ in how explicitly the delivery direction is spelled out. They're ordinary instruction text, not a server feature — Python and Swift agents can use the same strings by pasting them into speaking_style. See the TypeScript types reference for the full export list.

Catalog agents

A catalog agent's configuration lives server-side, created and edited in the dashboard (Realtime agents). A REST CRUD exists at /api/v1/external/agents, but it authenticates as a workspace user rather than with a realtime-scoped key, so the dashboard is the path an SDK integration takes. Your code launches it by its machine handle:

const agent = client.catalogAgent('support-triage', {
  inputs: { customer_name: 'Dana' },
  voice: 'Puck',
});

agent = client.catalog_agent(
    "support-triage",
    inputs={"customer_name": "Dana"},   # fills {{customer_name}} in the stored prompt
    voice="Puck",                       # per-run cosmetic override
)

let agent = client.catalogAgent(
    "support-triage",
    inputs: ["customer_name": "Dana"],  // fills {{customer_name}} in the stored prompt
    voice: VoiceConfig(name: "Puck")    // per-run cosmetic override
)

Only per-run ride-alongs are accepted alongside a catalog handle — inputs, a voice override, and extra client tools (plus a local mcp block in Python and Swift; like client tools, MCP servers run in your process, so the server can't provide them). Server hooks are stored config and can't ride along; in-process client hooks live in your process and never reach the wire, so they work with either agent kind. Sending stored-config fields like instructions or model with a catalog agent is a schema error: the stored config is the source of truth, and the server rejects the ambiguity rather than guessing.

Catalog agents are how you iterate on a shipped product without an app release: the client names the agent; prompt, model, voice, and server-tool changes land from the dashboard.

Resolution order

A handle resolves against two sources at session start. Agents saved in your workspace win on exact match; a name starting with cosmo- that matches no workspace agent falls through to Cosmo's built-in agent library — prebuilt agents your workspace has included, which appear alongside your own in the dashboard. The cosmo- prefix is reserved for that library: you can't save a new workspace agent under it, so a library handle your code ships against can't be shadowed later by a same-named workspace agent. Any other unmatched name fails the session start rather than launching a default agent.

Language

There is no language or locale setting in any SDK — not on the agent, the voice, the audio block, or any model block. The native-audio models these sessions run on decide their working language from the audio itself, and no provider setting pins it; the provider-sanctioned control is system instructions. The language story therefore lives in instructions, like the rest of the persona.

Left unsteered, one misheard utterance can flip a session: a stretch of accented English perceived as another language is enough. Managed sessions therefore compose default language-stability guidance into every system prompt — hold the conversation's language, treat a single ambiguous or foreign-sounding utterance as an accent rather than a switch, and switch only when the user clearly asks for it or plainly speaks the new language across several consecutive turns. That guidance defers to instruction-level language rules: an agent that carries its own — a pin, a bilingual grant — keeps them. The local OSS cosmo-server composes nothing; the model receives your instructions verbatim.

To steer beyond the default, write the rule into instructions:

• Hard pin — the same wording the dashboard's Realtime agents editor offers as its Language pinning section: "The conversation is in English only. Do not respond in any other language, even if the user switches or asks you to. If the user speaks another language, reply in English and ask them to continue in English."
• Bilingual — grant the switch explicitly and say what governs it: "You speak English and Spanish. Mirror the user: when they clearly switch languages, follow; otherwise stay in the language of the conversation so far."

Both are steering, not an API guarantee: instructions make drift rare; no setting makes it impossible. Drift shows in the transcript first — user lines in a language nobody spoke — and in the worst case the agent follows: its replies, and even the text in its tool calls, come out in the wrong language mid-session. The agent's own replies are the reliable signal, though: on the OpenAI-family and Grok providers, user transcripts come from a separate speech-to-text model, so a wrong-language user line under correct-language replies is that model misidentifying the language — a transcription artifact instructions cannot reach, since they steer only the conversation model. When the replies flip, tighten the instruction wording — there is no configuration flag to find.

Session options

Per-run options ride on start(), not on the agent: