Cosmo Realtime SDK

Build realtime voice and multimodal AI into any application. Make it talk. Give it eyes and tools. Put it in your product. Ship it reliably.

The Cosmo Realtime SDK gives your app a live voice-and-vision agent in one call. Audio travels over LiveKit WebRTC tracks; typed JSON events (transcripts, tool calls, errors) travel over a reliable data channel in the same room. You never talk to LiveKit directly — the SDK handles the media plumbing.

Three official clients ship from the same wire protocol:

Every SDK follows the same three-tier model: a client holds your credential, an agent is a reusable persona (instructions, voice, tools), and a session is one live run. See Clients, agents, and sessions.

Make it talk

TypeScript

import { RealtimeClient } from 'cosmo-ai';

const client = new RealtimeClient({ token: endUserJwt }); // or { apiKey } server-side
const agent = client.agent({
  instructions: 'You are a terse voice assistant.',
  greeting: 'Hi — how can I help?',
});

const session = await agent.start();
session.on('transcript', (item) => {
  if (item.isFinal) console.log(`[${item.role}] ${item.text}`);
});

Python

import asyncio, os
from cosmo_ai import CosmoRealtime, RealtimeTranscriptDelta

async def main() -> None:
    client = CosmoRealtime(api_key=os.environ["COSMO_API_KEY"])
    agent = client.agent(instructions="You are a terse voice assistant.")

    async with agent.start() as session:
        await session.set_microphone_enabled(True)
        await session.set_speaker_enabled(True)
        async for event in session:
            match event:
                case RealtimeTranscriptDelta() if event.is_final:
                    print(f"[{event.role.value}] {event.text}")

asyncio.run(main())

Swift

import CosmoRealtime

var config = SessionConfig()
config.instructions = "You are a terse voice assistant."

let session = try await RealtimeSession.start(
    .init(apiKey: apiKey, baseURL: URL(string: "https://app.askcosmo.ai")!),
    config: config
)

for try await event in session.events {
    if case .transcript(let delta) = event, delta.isFinal {
        print("[\(delta.role == .user ? "user" : "assistant")] \(delta.text)")
    }
}

What gets handled for you

• Session negotiation — POST /api/v1/external/realtime/session/start returns a LiveKit room + token; the SDK joins automatically.
• Audio I/O — the SDK publishes your microphone and plays back the agent's audio. You never manage WebRTC tracks.
• Envelope reassembly — messages larger than the ~15 KiB data-channel limit are chunked and transparently reassembled.
• Reconnects — transient network drops recover in place; the server rotates upstream model sessions without ending yours.
• Forward compatibility — unknown event types surface as explicit unknown events, never as decode errors.

The developer journey

1. See what's possible — browse the examples.
2. Build one working conversation — pick a quickstart.
3. Understand the realtime model — sessions, events, turn-taking.
4. Add tools and multimodal input — tools, hooks, image input, screen share.
5. Put it on the phone — telephony.
6. Ship it reliably — end-user credentials, session limits, debugging.

Choose a path

Not sure which shape fits? Read Choose your architecture first.