TypeScript types

All types are available as named imports from cosmo-ai. This page is the type index; the classes (RealtimeClient, RealtimeAgent, RealtimeSession) are documented on RealtimeClient, the React surface on React.

Client options

`RealtimeClientOptions`

Field	Type	Description
`baseUrl?`	`string`	Origin of the Cosmo external API. Must be `https://` (`http://` only for localhost). Omit in a browser to use the page origin.
`apiKey?`	`string`	Workspace-scoped API key (server-side secret). Requires `baseUrl`. At most one of `apiKey` / `token`.
`token?`	`string`	Minted end-user JWT (from `mintToken`). Safe on devices; can open sessions but not mint.
`getAuthHeaders?`	`() => Record<string,string> \| Promise<...>`	Custom auth headers; a configured credential's `Authorization` wins.
`transportFactory?`	`() => RealtimeTransport`	Transport override, constructed per session start. Defaults to LiveKit.
`screenInteraction?`	`ScreenInteraction`	Opt into grounded screen control. See Screen interaction.

Agent configuration

Type	Kind	Description
`AgentConfig`	object	The inline persona: `instructions?`, `model?`, `modelOptions?`, `voice?`, `tools?`, `interruptionSensitivity?`, `greeting?`, `audio?`, `skills?`, `hooks?`.
`VoiceConfig`	object	How the agent sounds: `name?` (provider voice id), `speakingStyle?` (delivery guidance appended after the persona). Accepted anywhere a plain voice-id string is.
`AudioConfig`	object	The audio pipeline: `output?` (`false` runs text-only), `noiseCancellation?`, `ambience?` (`AmbienceConfig`: `track?`, `gainDb?` — presence enables the bed).
`CatalogAgentOptions`	object	Per-run ride-alongs for `client.catalogAgent(name, {...})`: `inputs?`, `tools?`, `voice?`, `hooks?` (client hooks only).
`SessionStartOptions`	object	Per-run options for `agent.start()`: `resumeSessionId?`, `storeRecording?`, `publishMicrophone?` (default `true`; `false` joins as a silent observer).
`ModelOptions`	union	`GeminiModelOptions \| OpenAIModelOptions \| UltravoxModelOptions \| PersonaplexModelOptions`, discriminated on `provider`.
`GeminiModelOptions`	object	`provider: 'gemini'`, `temperature?`, `maxOutputTokens?`, `thinkingLevel?`.
`OpenAIModelOptions`	object	`provider: 'openai'` — no knobs today.
`UltravoxModelOptions`	object	`provider: 'cosmo_voice_ultravox'`, `temperature?`, `turnEndpointDelaySeconds?`.
`PersonaplexModelOptions`	object	`provider: 'cosmo_voice_personaplex'` — no knobs today.

Tools

Type	Kind	Description
`RealtimeTool`	union	`ClientToolSpec \| BackgroundClientToolSpec \| ServerToolSpec`.
`ClientToolSpec`	object	`kind: 'client'`, `name`, `description`, `parameters` (JSON Schema), `handler?: ClientToolHandler`.
`BackgroundClientToolSpec`	object	`kind: 'client'`, `background: true` — long-running tool; the handler acks via `ClientToolJob` and delivers the result later.
`ServerToolSpec`	object	`kind: 'server'`, `name` — opt-in to a server tool by dot-namespaced name.
`ClientToolHandler`	function	`(args) => Promise<Record<string, unknown> \| null \| void>` — returned object is the tool result; throw to surface an error.
`BackgroundClientToolHandler`	function	`(args, job: ClientToolJob) => Promise<void>` — `job.ack(note)` then `job.complete(...)` / `job.fail(...)`.
`ClientToolJob`	class	Handle for a background tool call: `ack`, `complete`, `fail`.

Skills and hooks

Type	Kind	Description
`Skill`	object	`{ name, description, body }` — see Skills.
`parseSkillMd`	function	Parse a `SKILL.md` document into a `Skill`. Throws `SkillParseError`.
`Hook`	class	One declared client hook (seam + callback + optional matcher). Build via the factories below.
`sessionStart` / `preToolUse` / `postToolUse` / `sessionEnd`	factories	`(fn, opts?) => Hook`. Tool seams accept `{ matcher }` (glob grammar; malformed matchers throw at declaration).
`HookEventName`	union	`'SessionStart' \| 'PreToolUse' \| 'PostToolUse' \| 'SessionEnd'`.
`SessionStartContext` / `PreToolUseContext` / `PostToolUseContext` / `SessionEndContext`	objects	Per-seam contexts passed to hook callbacks.
`SessionStartResult`	object	`{ additionalContext? }` — injected into the instructions.
`PreToolUseResult`	object	`{ permission?: 'allow' \| 'deny', reason?, updatedArguments? }`.
`SessionStartHook` / `PreToolUseHook` / `PostToolUseHook` / `SessionEndHook`	functions	Callback signatures per seam.
`ToolOutcome`	union	`{ kind: 'ok', result } \| { kind: 'error', message } \| { kind: 'denied', reason }` — what `PostToolUse` observes.
`ServerHook`	alias	`SilenceTimeout` — declarative server-executed hook config.
`ServerHookAction`	union	The `say` / `end_call` action a fired server hook performed.
`SilenceTimeout` / `Say` / `EndCall`	wire types	Re-exported wire shapes for server hooks.

See Hooks for semantics.

State

Type	Values / shape
`TransportState`	`'disconnected' \| 'requesting-permission' \| 'connecting' \| 'connected' \| 'ready' \| 'reconnecting' \| 'disconnecting' \| 'failed'`
`SessionLifecycleState`	`{ kind: 'idle' \| 'connecting' \| 'connected' \| 'reconnecting' \| 'disconnected'; disconnectReason?; detail? }` — the formal cross-SDK lifecycle. See Lifecycle.
`DisconnectReason`	`'client_ended' \| 'client_closed' \| 'handshake_failed' \| 'server_ended' \| 'transport_error'`
`AgentState`	`'idle' \| 'listening' \| 'thinking' \| 'speaking'`
`MicState`	`'unknown' \| 'requesting' \| 'granted' \| 'denied' \| 'muted' \| 'not-found'`
`OutputState`	`'blocked' \| 'playing' \| 'silent'` — `'blocked'` means the browser is suppressing autoplay; use `<StartAudio />`.
`MediaState`	`{ mic: MicState; screen: ScreenShareState; output: OutputState }`
`ScreenShareState`	`{ kind: 'inactive' } \| { kind: 'requesting' } \| { kind: 'active'; startedAt } \| { kind: 'error'; error }`

Events

Payloads for client.on(...) / RealtimeEventMap:

Type	Shape
`TranscriptEvent`	`{ id, turnId, role: 'user' \| 'assistant', text, isFinal, append }` — coalesce by `(turnId, role)`; `append` says extend vs. new bubble.
`ModelTextEvent`	`{ text, isFinal }` — model text channel, not a transcription.
`TurnCompleteEvent`	`{ role: 'user' \| 'assistant' }`
`ToolCallEvent`	`{ toolCallId, name }`
`ToolDispatchStartedEvent`	`{ toolCallId, name }`
`ToolResultEvent`	`{ toolCallId, ok, summary }`
`ReadyEvent`	`{ sessionId, rejectedTools, maxSessionSeconds, agent: ResolvedAgentInfo \| null }`
`ResolvedAgentInfo`	`{ name, tools }` — resolved catalog agent, informational.
`ReconnectingEvent`	`{ secondsRemaining }`
`UserSpeechTimeoutEvent`	`{ sessionId, silenceMs, triggerCount, maxCount, action }`
`VolumeEvent`	`{ mic, output }` — 0–1 RMS, per animation frame while subscribed.
`PongEvent`	`{}`
`RealtimeEventMap`	Event-name → payload map: `transport_state`, `agent_state`, `media_state`, `lifecycle`, `transcript`, `model_text`, `tool_call`, `tool_dispatch_started`, `tool_result`, `session_state`, `volume`, `error`, `ready`, `session_started`, `reconnecting`, `session_ending_soon`, `session_ended`, `turn_complete`, `pong`, `user_speech_timeout`.
`RealtimeEventName`	`keyof RealtimeEventMap`
`Unsubscribe`	`() => void`

Session stream items

Yielded by for await (const event of session):

Type	Shape
`RealtimeSessionEvent`	Wire frames verbatim, plus the two below.
`SessionEndedEventItem`	`{ type: 'session-ended', reason }` — SDK-local terminal item, always last.
`UnknownEvent`	`{ type: 'unknown', rawType, payload }` — forward-compat item for unrecognized frames; never terminal.

MonitorSession (from client.connectAsMonitor) exposes sessionId, getTranscript(), and subscribe().

Errors

All SDK-thrown errors extend CosmoRealtimeError.

Type	Description
`CosmoRealtimeError`	Base class — `err instanceof CosmoRealtimeError` catches the family.
`RealtimeError`	`{ code: RealtimeErrorCode; message }` — the `error` event payload.
`RealtimeErrorCode`	`'mic_denied' \| 'screen_denied' \| 'screen_start_failed' \| 'session_start_failed' \| 'session_rejected' \| 'auth_error' \| 'not_ready' \| 'transport_connect_timeout' \| 'transport_disconnect' \| 'unsupported_browser' \| 'server_error'`
`RealtimeNotReadyError`	Thrown by imperative methods called before the session is ready (`code: 'not_ready'`).
`RealtimeCredentialError`	Unusable credential configuration (both `apiKey` and `token`, or `apiKey` without `baseUrl`).
`RealtimeMintTokenError`	`mintToken()` failed; `code: MintTokenErrorCode` (server slug or `no_api_key` / `transport_error` / `invalid_response`).
`RealtimeDialError`	`session.dial()` failed; `code: DialErrorCode` (server slugs like `phone_calls_disabled`, `minute_limit_exceeded`, `session_not_live`, or client-side `invalid_phone_number`, `transport_error`).
`SkillParseError`	`SKILL.md` parsing failed.

MintTokenErrorCode and DialErrorCode are open-ended string aliases — treat unknown codes defensively.

Auth and telephony

Type	Shape
`MintedToken`	`{ jwt: string; expiresAt: Date }` — from `client.mintToken(externalUserId)`. See End-user credentials.
`DialResult`	`{ dialId: string }` — from `session.dial(phoneNumber)`. See Telephony.

Screen interaction

ScreenInteraction, ScreenCapture, ScreenElement, ScreenButton, ScreenAction — the platform capability object behind grounded screen control. See Screen interaction.

Constants and presets

Export	Description
`PROTOCOL_VERSION`	`'1.0'` — the external wire-protocol version the SDK speaks.
`NaturalnessRung`	`'warm' \| 'delivery' \| 'human'`.
`naturalness(rung)`	Resolves a rung to the verbatim `speakingStyle` instruction text.
`NATURALNESS_RUNGS` / `NATURALNESS_INSTRUCTIONS` / `NATURALNESS_VERSION`	The rung list, prompt catalog, and its schema version.

React types

CosmoRealtimeProviderProps, RealtimeClientLike, RealtimeSnapshotState, RealtimeTranscriptItem, RealtimeToolCallItem — documented with the provider, hooks (useTransportState, useAgentState, useMediaState, useTranscript, useToolCalls, useRealtimeError, useMicLevel, useOutputLevel), and components (RealtimeAudio, MicToggle, BarVisualizer, StartAudio) on React.

TypeScript types

On this page