React bindings

The React surface wraps a RealtimeSession in a context provider and exposes read-only hooks plus four ready-made components. All hooks and components must be used inside <RealtimeProvider>, with one exception: useRealtimeSession sits above the provider — it opens the session the provider consumes.

import {
  RealtimeProvider,
  useRealtimeSession,
  useRealtimeSessionContext,
  useTransportState,
  useAgentState,
  useMediaState,
  useTranscript,
  useToolCalls,
  useRealtimeError,
  useMicLevel,
  useOutputLevel,
  useRealtimeSnapshot,
  RealtimeAudio,
  MicToggle,
  BarVisualizer,
  StartAudio,
} from 'cosmo-ai/react';

These names live at cosmo-ai/react only. The package root does not re-export them, which is what keeps react off a headless consumer's dependency graph.

RealtimeProvider

RealtimeProvider is the read side of one RealtimeSession: it subscribes to the session's event stream and publishes normalized React state through context. Pass it the session a run returned — useRealtimeSession's session, or your own agent.start() result. With no session (null, between runs) the snapshot reports the initial idle state. Session lifecycle stays the caller's: the provider never starts or ends anything.

const session = await client.agent({ instructions }).start();

<RealtimeProvider session={session}>
  <VoicePanel />
</RealtimeProvider>

Props

type RealtimeProviderProps = {
  children: ReactNode;
  session?: RealtimeSession | null; // the run to read from; null between runs
};

useRealtimeSession

useRealtimeSession owns the session lifecycle for the common one-session-at-a-time browser app: it constructs a fresh RealtimeClient per run, starts the session, funnels every exit path (End button, server hangup, network loss) into one teardown, and holds the Start affordance closed until the spent session has released the microphone. It is sugar over the imperative surface, the same way the provider is sugar for reads. Feed its session to the provider so the read hooks track the run.

function useRealtimeSession(options: UseRealtimeSessionOptions): UseRealtimeSessionResult;

type UseRealtimeSessionOptions = {
  // Build the agent for one run on the freshly constructed client — return
  // client.agent({...}) or client.catalogAgent(name). Called once per start().
  makeAgent: (client: RealtimeClient) => RealtimeAgent;
  // Constructor options for each run's client (credential, transport).
  clientOptions?: RealtimeClientOptions;
};

type UseRealtimeSessionResult = {
  start: (options?: SessionStartOptions) => Promise<RealtimeSessionStartResult>;
  end: () => Promise<void>;
  error: Error | null;                       // why the last start() failed
  rejectedTools: RejectedTool[];             // typed rejected specs from `ready`
  warning: string | null;                    // ready-made notice over rejectedTools
  lastEnd: RealtimeSessionEndSummary | null; // typed record of the last end
  endedReason: string | null;                // sugar over lastEnd; null for local ends
} & (
  | { phase: 'idle' | 'starting' | 'ending'; client: null; session: null }
  | { phase: 'live'; client: RealtimeClient; session: RealtimeSession }
);

type RealtimeSessionStartResult =
  | { ok: true; session: RealtimeSession }
  | { ok: false; reason: 'busy' | 'failed' | 'ended'; error: Error | null };

type RealtimeSessionEndSummary = {
  reason: DisconnectReason; // 'client_ended' | 'client_closed' | 'server_ended' | ...
  detail: string | null;    // server end slug or transport message, when one exists
};

The result is a discriminated union on phase: when phase === 'live', client and session are statically non-null — no null checks needed after narrowing (destructuring all three from the same call preserves this). Neither option needs to be memoized — the latest committed value is read at start() time, so inline object and function literals are fine.

function App() {
  const { phase, session, start, end, error, endedReason } = useRealtimeSession({
    makeAgent: (client) => client.agent({ instructions: 'You are a helpful guide.' }),
    clientOptions: { token: mintedJwt },
  });

  if (phase === 'live') {
    return (
      <RealtimeProvider session={session}>
        <RealtimeAudio />
        <LiveView onEnd={() => void end()} />
      </RealtimeProvider>
    );
  }

  return (
    <div>
      {error !== null && <p>{error.message}</p>}
      {endedReason !== null && <p>Call ended ({endedReason}).</p>}
      <button onClick={() => void start()} disabled={phase !== 'idle'}>
        {phase === 'starting' ? 'Connecting…' : 'Start'}
      </button>
    </div>
  );
}

Behavior:

• Phases. start() moves idle → starting, then live once the session is ready. Any exit moves through ending → idle; end() enters ending immediately, and the phase reaches idle only after the spent session's microphone release lands, so a Start button gated on phase === 'idle' can never open a session whose mic track is still held by the last one.
• start(options?) accepts the same per-run SessionStartOptions as agent.start(). On ok: true the session is ready and every method usable. Otherwise reason says why: busy (a run is already underway; nothing changed), failed (the start threw; the error is returned and also lands in error), or ended (the run was over before it went live — cancelled, or ended the instant it started).
• end() gracefully ends a live session; during starting it cancels the run instead — the in-flight session is ended the moment the start settles, and start() resolves { ok: false, reason: 'ended' }. A no-op when nothing is underway. Ends the app didn't request (server hangup, network loss) run through the same teardown and land in lastEnd; endedReason is the one-line version, null for locally requested ends.
• rejectedTools / warning — tool specs the server refused at ready, with reasons; the session runs without them. warning is a ready-made notice for apps that don't need their own wording.
• Unmount ends a live session — or, mid-start, ends it as soon as the start settles — releasing the microphone either way.
• Errors during a live session are the provider's domain — read them with useRealtimeError() below.

Hooks

Each hook reads one slice of session state and re-renders only on changes to that slice.

useRealtimeSessionContext(): RealtimeSession | null

Returns the provider's current session, or null between runs. Use this for imperative calls from components that don't own the run themselves: sending text, toggling the mic, screen share.

const session = useRealtimeSessionContext();

async function send(text: string) {
  await session?.sendText(text);
}

async function end() {
  await session?.end();
}

useTransportState(): TransportState

Current wire-connectivity state.

const state = useTransportState();
// 'disconnected' | 'requesting-permission' | 'connecting' | 'connected'
// | 'ready' | 'reconnecting' | 'disconnecting' | 'failed'

useAgentState(): AgentState

Current agent activity.

const agent = useAgentState();
// 'idle' | 'listening' | 'thinking' | 'speaking'

useMediaState(): MediaState

Mic, screen share, and output audio state.

const media = useMediaState();

media.mic    // MicState: 'unknown'|'requesting'|'granted'|'denied'|'muted'|'not-found'|'in-use'
media.output // OutputState: 'blocked'|'playing'|'silent'
media.screen // ScreenShareState: { kind: 'inactive'|'requesting'|'active'|'error' }

useTranscript(options?: UseTranscriptOptions): readonly TranscriptItem[]

Returns the session's coalesced conversation — one item per turn, folded by the session itself.

const transcript = useTranscript();
// or cap to last 5 items:
const recent = useTranscript({ limit: 5 });

type TranscriptItem = {
  id: string; // stable render key for the bubble
  role: 'user' | 'assistant';
  text: string;
  isFinal: boolean; // false while the turn is in progress; frozen once true
};

The full conversation is kept for the session's lifetime. Pass { limit } when a view only needs the tail.

transcript.map((item) => (
  <div key={item.id} className={item.role === 'user' ? 'user' : 'bot'}>
    {item.text}
    {!item.isFinal && <span>…</span>}
  </div>
))

useToolCalls(): RealtimeToolCallItem[]

Returns in-flight and completed tool calls.

const tools = useToolCalls();

type RealtimeToolCallItem = {
  toolCallId: string;
  name: string;
  status: 'in_flight' | 'ok' | 'error';
  summary: string | null;
};

useRealtimeError(): SessionStartError | AudioUnavailableError | ErrorEvent | null

Most recent terminal error, or null when the session is healthy. Cleared to null when the next session start resets the snapshot.

The value is the error itself — the instance start() rejected with, or the server's own error frame. All three carry code and message; see Errors for branching between them.

const error = useRealtimeError();
if (error) {
  return <ErrorBanner code={error.code} message={error.message} />;
}

useMicLevel(): number

RMS level of the local mic input (0–1 normalized). Updates at animation-frame rate while any subscriber is mounted. Returns 0 when no session is active.

const level = useMicLevel(); // e.g. 0.23

useOutputLevel(): number

RMS level of the bot's audio output (0–1 normalized). Same cadence as useMicLevel.

const level = useOutputLevel();

useRealtimeSnapshot(): RealtimeSnapshotState

The whole snapshot in one read, for a component that needs several slices at once. It re-renders on any change to the snapshot, so prefer the single-slice hooks above where they suffice.

const { transportState, agentState, transcript, toolCalls } = useRealtimeSnapshot();

Components

These components handle the browser plumbing that every voice app needs.

<RealtimeAudio />

Mounts a hidden <audio autoPlay> element and wires it to the transport. Place it once inside the provider tree.

Handles autoplay detection: calls session.setOutputBlocked(true) when play() is rejected by the browser, and setOutputBlocked(false) when playback starts. This drives mediaState.output so <StartAudio /> knows when to render.

type RealtimeAudioProps = {
  onError?: (err: unknown) => void;
};

<RealtimeProvider session={session}>
  <RealtimeAudio onError={(err) => toast.error('Audio unavailable')} />
  <MyApp />
</RealtimeProvider>

<MicToggle />

A <button> bound to the SDK's mic state. Calls session.setMuted(!muted) on click. Disabled when transportState !== 'ready' or no session is in context.

No styling opinions — style with className or build your own button on top of useRealtimeSessionContext() and useMediaState().

type MicToggleProps = {
  className?: string;
  label?: { muted: string; unmuted: string };
  onError?: (err: unknown) => void;
};

<MicToggle
  className="rounded-full p-2"
  label={{ muted: 'Unmute', unmuted: 'Mute' }}
  onError={(err) => console.error('Mic toggle failed', err)}
/>

Default labels are 'Mute' / 'Unmute'. The button carries aria-label="Mute microphone" / aria-label="Unmute microphone" and aria-pressed automatically. onError fires when setMuted rejects — the SDK has already rolled back its optimistic state by then.

<BarVisualizer />

Five animated bars that reflect the current AgentState. Pure CSS keyframes — no AudioContext required.

type BarVisualizerProps = {
  className?: string;
};

<BarVisualizer className="text-brand-600" />

The color inherits from currentColor. The bars animate differently per agent state:

The root element has data-agent-state set to the current state for CSS selector targeting.

For frequency-accurate waveforms, use useMicLevel() / useOutputLevel() to build your own visualizer.

<StartAudio />

Autoplay unlock. Browsers (Safari, mobile Chromium) block remote audio until a user gesture. With no children it renders a default button, shown only while playback is blocked; pass a render prop to supply your own affordance from the blocked boolean and start callback.

type StartAudioProps = {
  children?: (args: { blocked: boolean; start: () => Promise<void> }) => ReactNode;
  label?: string;      // default button text; ignored when children is supplied
  className?: string;
};

// Default: renders "Tap to enable voice" while blocked, nothing otherwise.
<StartAudio />

// Custom affordance.
<StartAudio>
  {({ blocked, start }) =>
    blocked ? (
      <button onClick={start}>Tap to enable voice</button>
    ) : null
  }
</StartAudio>

start plays the <audio> element mounted by <RealtimeAudio /> and calls session.resumeAudioPlayback() so every transport-owned track resumes too. Wire <RealtimeAudio /> before <StartAudio /> in the tree.

Exported types from the React layer

The following table lists the types the React layer exports.