TypeScript types

All types are available as named imports from cosmo-ai. This page is the type index; the classes (RealtimeClient, RealtimeAgent, RealtimeSession) are documented on RealtimeClient, the React surface on React.

Client options

RealtimeClientOptions

Token sources

A TokenSource is the credential shape for distributed apps: it fetches a fresh minted token from your backend, caches it in memory, and re-fetches when the cached token has less than 60 seconds of life left, so new RealtimeClient({ token: TokenSource.endpoint(...) }) stays valid for the life of the process. Concurrent callers share one in-flight fetch, and a session start rejected with HTTP 401 drops the cache.

Entry points

The package publishes the root barrel plus the fixed set of subpaths in its build entry map — everything importable is listed here; anything else has no dist file.

Agent configuration

Tools

Skills and hooks

See Hooks for semantics.

State

Events

Payloads for session.on(...) / RealtimeEventMap:

Four payload types — UsageEvent, SessionStateWriteEvent, SessionEndingSoonEvent, and SessionEndedEvent — are not exported from the package root; import them from the cosmo-ai/core/events subpath. Every other type in this table is a named import from cosmo-ai.

Session stream items

Yielded by for await (const event of session):

Errors

Every SDK-thrown error extends RealtimeError and sets a distinct name, so err instanceof RealtimeError catches the whole family. Branch on the concrete class (err instanceof DialError) or on err.code for the specific failure.

DialErrorCode is an open-ended string alias — treat unknown codes defensively. SkillErrorCode, MintTokenErrorCode, TokenSourceErrorCode and AudioUnavailableErrorCode are closed unions: the SDK raises every one of them, so a switch over one is exhaustive. The open half of a mint or token-source failure is serverCode, which belongs to whichever backend answered.

Auth and telephony

Screen tools

The shapes screen-tool handlers receive and answer with, all barrel exports. See Screen tools for the task-level walkthrough.

Logging

Constants and presets

React types

RealtimeProviderProps, RealtimeSnapshotState, RealtimeToolCallItem — documented with the provider, hooks (useTransportState, useAgentState, useMediaState, useTranscript, useToolCalls, useRealtimeError, useMicLevel, useOutputLevel), and components (RealtimeAudio, MicToggle, BarVisualizer, StartAudio) on React.

Combined usage

The core types working together — options into a client, an AgentConfig into an agent, session options into start(), and a typed event payload out:

import {
  RealtimeClient,
  TokenSource,
  type AgentConfig,
  type TranscriptDeltaEvent,
} from 'cosmo-ai';

const config: AgentConfig = {
  instructions: 'You are a terse voice assistant.',
  voice: { name: 'Puck' },
  audio: { noiseCancellation: 'denoise' },
};

const client = new RealtimeClient({
  token: TokenSource.endpoint('/api/cosmo/token'),
});

const session = await client.agent(config).start({ storeRecording: false });

session.on('transcript', (event: TranscriptDeltaEvent) => {
  if (event.isFinal) console.log(event.role, event.text);
});

Gemini tool response policies

GeminiToolResponsePolicy has behavior: 'blocking' | 'non_blocking' and optional scheduling: 'when_idle' | 'silent' | 'interrupt'. Set GeminiModel.toolResponsePolicy for the default and toolResponseOverrides for a map from declared tool names to replacement policies. Omitted policies keep tools blocking on standard Gemini Live. Non-blocking results default to when_idle; silent absorbs the result without speaking and interrupt interrupts current speech.

import { GeminiModel } from 'cosmo-ai';

const model = GeminiModel({
  modelId: 'gemini-3.8-live',
  toolResponseOverrides: {
    lookup: { behavior: 'non_blocking', scheduling: 'when_idle' },
  },
});

gemini-3.8-live-extended-thinking defaults to non-blocking tools, accepts low, medium or high thinking, and rejects minimal, blocking policies and explicit scheduling. An utterance can finish while the model is still thinking. Select independent lookups for non-blocking behavior; keep dependent actions blocking on standard Gemini Live. Background job tools retain their acknowledgement and completion contract.