React bindings

The React surface wraps RealtimeClient in a context provider and exposes read-only hooks plus four ready-made components. All hooks and components must be used inside <CosmoRealtimeProvider>.

import {
  CosmoRealtimeProvider,
  useRealtimeClient,
  useTransportState,
  useAgentState,
  useMediaState,
  useTranscript,
  useToolCalls,
  useRealtimeError,
  useMicLevel,
  useOutputLevel,
  RealtimeAudio,
  MicToggle,
  BarVisualizer,
  StartAudio,
} from 'cosmo-ai';

CosmoRealtimeProvider

CosmoRealtimeProvider subscribes to the RealtimeClient event stream and publishes normalized React state through context.

Two configurations:

• Bring your own client — pass the client prop. The provider reads from it but does not call disconnect() on unmount; lifecycle is the host's. Use this when the client needs a credential (token / apiKey) or screenInteraction — those are RealtimeClient constructor options, not provider props.
• Provider-owned client — pass baseUrl (and optionally getAuthHeaders / transportFactory). The provider constructs and owns the RealtimeClient; disconnect() runs on unmount or when any construction prop changes.

// Bring your own client
const client = new RealtimeClient({
  baseUrl: 'https://app.askcosmo.ai',
  token: mintedJwt,
});

<CosmoRealtimeProvider client={client}>
  <VoicePanel />
</CosmoRealtimeProvider>

// Provider-owned client
<CosmoRealtimeProvider
  baseUrl="https://app.askcosmo.ai"
  getAuthHeaders={getAuthHeaders} // stable identity — hoist or useCallback
>
  <VoicePanel />
</CosmoRealtimeProvider>

Props

type CosmoRealtimeProviderProps = {
  children: ReactNode;
  // --- bring-your-own-client mode ---
  client?: RealtimeClientLike;
  // --- provider-owned client mode (ignored when client is supplied) ---
  baseUrl?: string;
  getAuthHeaders?: () => Record<string,string> | Promise<Record<string,string>>;
  transportFactory?: () => RealtimeTransport;
  // --- shared ---
  maxTranscriptLength?: number; // default 12; pass Infinity for unbounded
};

Hooks

useRealtimeClient(): RealtimeClientLike

Returns the RealtimeClient (or compatible object) from context. Use this for imperative calls: building agents, starting sessions, sending text, toggling the mic.

const client = useRealtimeClient();

async function start() {
  const agent = client.agent({ instructions: 'You are a helpful guide.' });
  await agent.start();
  await client.waitUntilReady();
}

async function end() {
  await client.disconnect();
}

RealtimeClientLike exposes: agent, connectAsMonitor, disconnect, waitUntilReady, sendText, dial, registerRpcMethod, setMicMuted, acquireMic, publishHeldMic, releaseHeldMic, setOutputBlocked, resumeAudioPlayback, setListenMuted, startScreenShare, stopScreenShare, getVisionInputStatus, addVideoStream, removeVideoStream, attachAudioElement, isActive, getSnapshot, and on.

useTransportState(): TransportState

Current wire-connectivity state.

const state = useTransportState();
// 'disconnected' | 'requesting-permission' | 'connecting' | 'connected'
// | 'ready' | 'reconnecting' | 'disconnecting' | 'failed'

useAgentState(): AgentState

Current agent activity.

const agent = useAgentState();
// 'idle' | 'listening' | 'thinking' | 'speaking'

useMediaState(): MediaState

Mic, screen share, and output audio state.

const media = useMediaState();

media.mic    // MicState: 'unknown'|'requesting'|'granted'|'denied'|'muted'|'not-found'
media.output // OutputState: 'blocked'|'playing'|'silent'
media.screen // ScreenShareState: { kind: 'inactive'|'requesting'|'active'|'error' }

useTranscript(options?: UseTranscriptOptions): RealtimeTranscriptItem[]

Returns the in-memory transcript slice from the provider's React snapshot.

const transcript = useTranscript();
// or cap to last 5 items:
const recent = useTranscript({ limit: 5 });

type RealtimeTranscriptItem = {
  id: string;
  turnId: string;
  role: 'user' | 'assistant';
  text: string;
  isFinal: boolean;
};

The provider caps the array at maxTranscriptLength (default 12) to bound memory in long sessions. Pass maxTranscriptLength={Infinity} to the provider for unbounded history.

transcript.map((item) => (
  <div key={item.id} className={item.role === 'user' ? 'user' : 'bot'}>
    {item.text}
    {!item.isFinal && <span>…</span>}
  </div>
))

useToolCalls(): RealtimeToolCallItem[]

Returns in-flight and completed tool calls.

const tools = useToolCalls();

type RealtimeToolCallItem = {
  toolCallId: string;
  name: string;
  status: 'in_flight' | 'ok' | 'error';
  summary: string | null;
};

useRealtimeError(): RealtimeError | null

Most recent terminal error, or null when the session is healthy. Cleared to null when the next session start resets the snapshot.

const error = useRealtimeError();
if (error) {
  return <ErrorBanner code={error.code} message={error.message} />;
}

useMicLevel(): number

RMS level of the local mic input (0–1 normalized). Updates at animation-frame rate while any subscriber is mounted. Returns 0 when no session is active.

const level = useMicLevel(); // e.g. 0.23

useOutputLevel(): number

RMS level of the bot's audio output (0–1 normalized). Same cadence as useMicLevel.

const level = useOutputLevel();

useMonitorTranscript(monitor: MonitorSession | null): RealtimeTranscriptItem[]

Live folded transcript of one monitored session. Pass the handle returned by client.connectAsMonitor; null (not yet connected) yields an empty transcript. Unlike useTranscript, the returned bubbles are scoped to that session — a closed handle stops updating.

import { useMonitorTranscript } from 'cosmo-ai/react/hooks';

const transcript = useMonitorTranscript(monitor);

Components

<RealtimeAudio />

Mounts a hidden <audio autoPlay> element and wires it to the transport. Place it once inside the provider tree.

Handles autoplay detection: calls client.setOutputBlocked(true) when play() is rejected by the browser, and setOutputBlocked(false) when playback starts. This drives mediaState.output so <StartAudio /> knows when to render.

type RealtimeAudioProps = {
  onError?: (err: unknown) => void;
};

<CosmoRealtimeProvider baseUrl="https://app.askcosmo.ai">
  <RealtimeAudio onError={(err) => toast.error('Audio unavailable')} />
  <MyApp />
</CosmoRealtimeProvider>

<MicToggle />

A <button> bound to the SDK's mic state. Calls client.setMicMuted(!muted) on click. Disabled when transportState !== 'ready'.

No styling opinions — style via className or build your own button on top of useRealtimeClient() and useMediaState().

type MicToggleProps = {
  className?: string;
  label?: { muted: string; unmuted: string };
  onError?: (err: unknown) => void;
};

<MicToggle
  className="rounded-full p-2"
  label={{ muted: 'Unmute', unmuted: 'Mute' }}
  onError={(err) => console.error('Mic toggle failed', err)}
/>

Default labels are 'Mute' / 'Unmute'. The button carries aria-label="Mute microphone" / aria-label="Unmute microphone" and aria-pressed automatically. onError fires when setMicMuted rejects — the SDK has already rolled back its optimistic state by then.

<BarVisualizer />

Five animated bars that reflect the current AgentState. Pure CSS keyframes — no AudioContext required.

type BarVisualizerProps = {
  className?: string;
};

<BarVisualizer className="text-brand-600" />

The color inherits from currentColor. The bars animate differently per agent state:

The root element has data-agent-state set to the current state for CSS selector targeting.

For frequency-accurate waveforms, use useMicLevel() / useOutputLevel() to build your own visualizer.

<StartAudio />

Autoplay unlock. Browsers (Safari, mobile Chromium) block remote audio until a user gesture. With no children it renders a default button, shown only while playback is blocked; pass a render prop to supply your own affordance from the blocked boolean and start callback.

type StartAudioProps = {
  children?: (args: { blocked: boolean; start: () => Promise<void> }) => ReactNode;
  label?: string;      // default button text; ignored when children is supplied
  className?: string;
};

// Default: renders "Tap to enable voice" while blocked, nothing otherwise.
<StartAudio />

// Custom affordance.
<StartAudio>
  {({ blocked, start }) =>
    blocked ? (
      <button onClick={start}>Tap to enable voice</button>
    ) : null
  }
</StartAudio>

start plays the <audio> element mounted by <RealtimeAudio /> and calls client.resumeAudioPlayback() so every transport-owned track resumes too. Wire <RealtimeAudio /> before <StartAudio /> in the tree.

Exported types from the React layer