Cartographer

Run the Cartographer example — a SwiftUI macOS app that listens while you think out loud and draws a live mind map, one client tool call per idea.

Cartographer is a SwiftUI macOS app that listens while you think out loud and draws what you say as a live mind map. Ideas land on the canvas while the agent is still speaking, because each one arrives as a client tool call.

Where the HelloRealtime example shows a session in a terminal, this shows one wired to a real UI: streaming transcripts folded into an observable view model, client tools with Decodable arguments mutating on-screen state, and a hook enforcing an app-side limit. It also ships bundle.sh, a working answer to packaging a SwiftPM-built executable as a signed .app that can actually capture audio.

Full source: examples/swift/Cartographer in the examples repo.

Prerequisites

macOS 14+ with Xcode command-line tools
A workspace API key with the realtime:start scope (API keys)
A microphone (or use demo mode, which needs none)

Run it

Clone the examples repo:

git clone https://github.com/socratic-ai/cosmo-ai
cd cosmo-ai/examples/swift/Cartographer

Export your key and launch:
```
export COSMO_API_KEY=cosmo_...
./run.sh              # talk to it
./run.sh --demo       # no mic: connects and types a seed idea for you
```
run.sh builds the .app bundle via bundle.sh on first run, then launches it. --demo is the fastest way to see the pipeline end to end without granting microphone access.
Think out loud. Nodes appear on the canvas as you speak; the agent stays mostly silent while the map grows.

There is also a windowless smoke test: COSMO_API_KEY=cosmo_... NO_AUDIO=1 swift run Probe connects, declares one client tool, sends a text line, proves the tool round-trips, and hangs up.

How it works

One Conductor class owns the session and folds everything it emits into @Published state the UI renders. Tools are declared with clientTool, which decodes arguments straight into a Sendable struct:

struct AddIdea: Decodable, Sendable {
    let idea: String
    let parent: String?
}

let addIdea = try AgentTool.clientTool(
    name: "add_idea",
    description: "Put one idea on the visible mind map. Call this the moment the user says something worth keeping.",
    input: .object(
        properties: [
            "idea": .string(description: "The idea, 1-5 words."),
            "parent": .string(description: "Existing idea this hangs off. Omit for a new root."),
        ],
        required: ["idea"]
    )
) { (args: AddIdea) -> [String: JSONValue] in
    await placeOnCanvas(args.idea, under: args.parent)
    return ["placed": .bool(true)]
}

A preToolUse hook enforces a limit the model can't be trusted to remember — past ~40 nodes the map is unreadable, so the hook denies further add_idea calls and tells the model why, which makes it start pruning instead of retrying:

try preToolUse(matcher: "add_idea") { _ in
    guard await mapIsFull() else { return nil }
    return PreToolUseResult(
        permission: .deny,
        reason: "The map is full (40 ideas). Summarise or link existing ideas instead."
    )
}

Two details in Conductor are worth reading before you copy this shape into your own app:

Transcripts aren't all the same. A non-final transcript event carries only the new fragment for that turn, so you append it; the final event carries the whole turn, so you replace what you accumulated. Rendering both the same way duplicates every turn once it reaches a real UI.
The session is already live when start(...) returns. It resolves at the agent's ready handshake, so there is no gap to guard against. Conductor marks itself live on the first event of any kind, and the ready event it also receives carries the session id and any tools the server rejected.

Package it as a real app

swift build produces a bare executable, which isn't enough to capture audio on macOS. bundle.sh assembles a real .app because three things are required and none come for free outside Xcode: an Info.plist with NSMicrophoneUsageDescription (no bundle, no microphone prompt), the LiveKit binary frameworks copied into Contents/Frameworks/ with an rpath (SwiftPM links but doesn't embed them), and an ad-hoc signature with library validation disabled (the frameworks carry a different team identity). Packaging a macOS app walks the same ground in depth.

What you should see

The app opens with an empty canvas and the agent greets you with "Map's open. What are we thinking about?" As you talk through a topic, nodes appear within a second or two of each idea leaving your mouth — the tool feed down the side shows each add_idea call as it lands. When two ideas relate, a labelled edge appears (link_ideas), and once the subject is clear the map's title updates (title_map). Ask "what's on the map so far?" and the agent calls read_map before summarizing. In --demo mode the same flow runs from a typed seed idea instead of your voice.

Troubleshooting

Issue: the microphone prompt never appears and the agent hears nothing. Cause: the binary ran outside an .app bundle, so macOS has no NSMicrophoneUsageDescription to show. Solution: launch through ./run.sh (which builds the bundle) rather than swift run Cartographer.

Next steps

Build a Mac voice app — the guided walkthrough of the same stack
Packaging a macOS app — the bundling and signing details
Hooks — matchers, fold rules, and the other hook seams