Transport

The SDK separates session logic from delivery. RealtimeClient depends only on the RealtimeTransport interface; the default implementation uses LiveKit. You would swap the transport to test without a live server, to use a different SFU, or to inject a mock for integration testing.

RealtimeTransport interface

interface RealtimeTransport {
  connect(options: RealtimeConnectOptions): Promise<void>;
  disconnect(): Promise<void>;
  send(message: RealtimeClientMessage): Promise<void>;
  setMicMuted(muted: boolean): Promise<void>;
  addVideoStream?(stream: MediaStream, options?: VideoStreamOptions): Promise<VideoStreamHandle>;
  removeVideoStream?(streamId: VideoStreamHandle): Promise<void>;
  getInputStream(): MediaStream | null;
  getOutputAudioElement(): HTMLAudioElement | null;
  attachAudioElement(el: HTMLAudioElement | null): void;
  onMessage(cb: (msg: RealtimeServerMessage) => void): Unsubscribe;
  onClose(cb: (info?: RealtimeCloseInfo) => void): Unsubscribe;
}

Lifecycle contract

• connect resolves once media and data channels are open and the local mic is publishing. Throws on auth failure, mic denial, or transport failure; on any throw, all allocations are released.
• disconnect is idempotent and always resolves.
• onClose fires only on unsolicited disconnects. A close caused by disconnect() does not trigger this callback.

RealtimeConnectOptions

type RealtimeConnectOptions = {
  /** The external-protocol `session-config` body, built by the agent
   *  mapping. The transport POSTs it verbatim and never inspects it. */
  config: RealtimeSessionConfig;
  /** Absolute session-start URL, composed by the SDK from `baseUrl`
   *  (or the page origin), so the transport holds no URL layout itself. */
  sessionStartUrl: string;
  getAuthHeaders?: () => Record<string, string> | Promise<Record<string, string>>;
};

VideoStreamOptions

type VideoStreamOptions = {
  id?: string;
  fps?: number;
  kind?: 'camera' | 'screen';
};

kind: 'screen' tells the transport to publish as Track.Source.ScreenShare so the server can distinguish camera from screen.

RealtimeCloseInfo

type RealtimeCloseInfo = {
  reason?: string; // e.g. 'livekit:CLIENT_INITIATED'
  code?: string;
};

RealtimeClient strips the vendor prefix (e.g. livekit:) before surfacing reason in RealtimeError.message, so consumers never see transport-specific prefixes.

LiveKitTransport

The default transport. Connects using the session token returned by POST /api/v1/external/realtime/session/start, joins the LiveKit room, and routes JSON control messages over the reliable data channel. Audio and video ride separate RTP tracks.

LiveKitTransport is internal — the package does not export it, and there is no import that reaches it. RealtimeClient instantiates it for you unless transportFactory is overridden, so the default needs no import:

import { RealtimeClient } from 'cosmo-ai';

const client = new RealtimeClient({ getAuthHeaders: ... });

To run against something else, pass a transportFactory returning your own RealtimeTransport — see Swapping the transport.

What LiveKitTransport does

1. POSTs RealtimeSessionRequest to the session endpoint with the caller's auth headers.
2. Receives { livekit_url, token, room_name, session_id }.
3. Joins the LiveKit room at livekit_url with token.
4. Publishes the default mic as a Track.Source.Microphone track.
5. Routes outbound JSON via LocalParticipant.publishData(reliable: true).
6. Subscribes to dataReceived for inbound JSON messages.
7. Routes oversized messages through the envelope chunking layer (15 KiB limit per packet).

Swapping the transport

Test fake

import { RealtimeClient } from 'cosmo-ai';
import type { RealtimeTransport } from 'cosmo-ai/transport/types';

const fakeTransport: RealtimeTransport = {
  connect: vi.fn().mockResolvedValue(undefined),
  disconnect: vi.fn().mockResolvedValue(undefined),
  send: vi.fn().mockResolvedValue(undefined),
  setMicMuted: vi.fn().mockResolvedValue(undefined),
  getInputStream: () => null,
  getOutputAudioElement: () => null,
  attachAudioElement: () => {},
  onMessage: (cb) => {
    // simulate ready
    setTimeout(() => cb({ type: 'ready', session_id: 'test', version: '1.0' }), 10);
    return () => {};
  },
  onClose: () => () => {},
};

const client = new RealtimeClient({
  transportFactory: () => fakeTransport,
});

When to swap

• Unit tests — fake resolves immediately, no network.
• Custom auth layer — override getAuthHeaders instead; no transport swap needed.
• Alternative SFU — implement RealtimeTransport for a different WebRTC stack and pass it via transportFactory.

Envelope chunking

The LiveKit reliable data channel has a ~15 KiB per-packet cap. Both client and server transparently chunk oversized messages using the envelope protocol:

• Client → server: type: "envelope-chunk" with envelope_id, seq, total, base64-encoded fragment.
• Server → client: type: "server-envelope-chunk" with identical fields.

The transport and client handle reassembly automatically. Callers never need to deal with chunking directly.