Cosmo Realtime SDK
Quickstart

Swift quickstart

Build a voice session on macOS or iOS with CosmoAI in five minutes.

Prerequisites

Set up

The recommended route is the CLI — it signs you in, stores the key, and equips your coding agent in one command:

curl -fsSL https://platform.askcosmo.ai/docs/install.sh | sh
cosmo init

Your browser opens, you pick a workspace, and the CLI stores a key at ~/.cosmo/credentials. The SDK reads the same file, so try RealtimeClient() resolves the credential and the backend it was issued for on its own — no key in your source and nothing to export. Run cosmo whoami any time to see which workspace you're pointed at, and see Set up with the CLI for what else it does.

Prefer to manage the key yourself? Create one in the dashboard (API keys) and export COSMO_API_KEY=cosmo_... instead — the zero-argument client checks that first, before the credentials file. Either way the key scopes your workspace server-side, so there is no project ID to pass.

An API key is a server-side secret — fine for a CLI or prototype on your own machine, but a distributed app must use a minted end-user token (RealtimeClient(token:)) instead. See End-user credentials.

Add the package

Create a package and give it a Package.swift that declares the SDK's platform floor — the SDK requires macOS 13 or iOS 16, and a manifest without a platforms: block fails to resolve against it:

// swift-tools-version: 5.9
import PackageDescription

let package = Package(
    name: "MyApp",
    platforms: [
        .macOS(.v13),
    ],
    dependencies: [
        .package(url: "https://github.com/socratic-ai/cosmo-swift-sdk", from: "0.8.1"),
    ],
    targets: [
        .executableTarget(
            name: "MyApp",
            dependencies: [
                .product(name: "CosmoRealtime", package: "cosmo-swift-sdk"),
            ]
        ),
    ]
)

The package: label is the repository name, cosmo-swift-sdk — that is the package's identity for a URL dependency, not the CosmoAI name inside its manifest.

Or use Xcode: File → Add Package Dependencies and paste the URL.

On 0.7.0 and earlier, the first build in Xcode fails with Validate plug-in "OpenAPIGenerator" … must be enabled before it can be used. Those versions generate their API client during the build, and Xcode will not run a package build plugin until you trust it: click Trust & Enable on the prompt, or choose the plugin under File → Packages → Trust & Enable Plugins. On a build machine with no one to answer the prompt — Xcode Cloud, or any headless runner — pass -skipPackagePluginValidation to xcodebuild instead.

Later versions ship the generated client already built, so there is no plugin and no prompt.

The from: range accepts every release below 1.0, and these docs describe the latest release — if an API on these pages is missing in your build, run swift package update first. Pre-1.0 minors may include breaking changes; the changelog lists them per release.

Start a session

Three objects, one per concern. RealtimeClient holds the credential and the timeouts, client.agent(...) describes the assistant — instructions, voice, greeting, tools — and agent.start() opens one run: it performs the REST session-start, joins the LiveKit room publishing your microphone during the join, and returns once the agent is ready, so every session method works immediately.

This program uses top-level await and readLine(), so it belongs in Sources/MyApp/main.swift — SwiftPM only allows top-level code in a file with that name.

import CosmoRealtime
import Foundation

print("Connecting…")
let client = try RealtimeClient()
let agent = try client.agent(
    instructions: "You are a terse voice assistant.",
    greeting: "Hi — how can I help?"
)
let session = try await agent.start()

Consume the event stream

Everything the server says arrives on one typed stream, session.events. No listeners to register up front — the stream buffers from session start, so you miss nothing. .sessionEnded is the terminal element and finishes the stream.

let events = Task {
    do {
        for try await event in session.events {
            switch event {
            case .ready(let ready):
                print("Session ready — id: \(ready.sessionId)")
                print("Speak into your microphone. Press Enter to end.")
            case .transcript(let delta) where delta.isFinal && !delta.text.isEmpty:
                // A console is append-only: print each completed turn once,
                // off the raw delta stream. A UI renders the session-owned
                // transcript wholesale instead.
                print("[\(delta.role)] \(delta.text)")
            case .error(let err):
                fputs("Server error (\(err.code.rawValue)): \(err.message)\n", stderr)
            case .sessionEnded(let ended):
                print("Session ended: \(ended.reason ?? "")")
            default:
                break
            }
        }
    } catch {
        fputs("event stream error: \(error)\n", stderr)
    }
}

session.transcript holds the same items for reading at any point — including after the session ends. Transcripts covers the item shape and the raw delta stream underneath.

End the session

_ = readLine()  // block until Enter

await session.end()
events.cancel()

Run it

swift run

Grant the microphone permission when prompted and say hello.

agent.start(...) publishes the microphone during the join unless you pass micMuted: true — pass it for a push-to-talk UX, or whenever your UI presents the session as muted, and unmute later with setMuted(false).

A bare swift run binary has no app bundle, so it has no NSMicrophoneUsageDescription of its own and runs under the host terminal's microphone grant. Fine for this quickstart; a shipped .app needs its own purpose string and embedded frameworks — see Packaging a macOS app.

What you should see

Connecting…
Session ready — id: 3f2b8c1e-9a4d-4e7f-8b21-06d5c9a1f4e2
Speak into your microphone. Press Enter to end.
[assistant] Hi — how can I help?
[user]… what's the tallest mountain
[user] what's the tallest mountain in the world
[assistant] Mount Everest, at 8,849 meters.
Session ended: client_ended

The complete runnable version of this program — including a client tool the agent can call — lives at examples/swift/HelloRealtime in the examples repo.

Call session methods

The following table lists the methods available on a live session.

MethodWhat it does
send(text:)Send a text turn the agent answers.
send(context:)Give the agent context without asking it anything — no turn, no speech.
setMuted(_:)Mute or unmute the microphone.
ping()Send a keepalive ping.
end()End the session gracefully.

Troubleshooting

The app terminates the moment it asks for the microphone. Cause: NSMicrophoneUsageDescription is missing from Info.plist. Solution: add it. See Swift platform behavior.

CredentialsError before the session starts. Cause: nothing resolved — no COSMO_API_KEY in the environment and no credentials file. Solution: run cosmo login. If you signed in under a named profile, set COSMO_PROFILE to match.

SessionStartError with an auth-related serverCode. Cause: the key is expired, revoked, or lacks the realtime:start scope. Solution: run cosmo whoami — it names the workspace and says whether the credential can start sessions. For a dashboard key, confirm its scopes on the API keys page.

The session connects but the agent can't hear you. Cause: the process has no microphone permission, or the mic never published. Solution: check the permission state before starting, and confirm a prewarm isn't running before the prompt is answered — MicPrewarmCoordinator no-ops when permission isn't yet granted.

Next steps

On this page