Screen share

await session.startScreenShare();   // browser shows the picker
// … the agent now sees the shared surface …
await session.stopScreenShare();

Track the lifecycle for your UI:

const state = client.getScreenShareState();
// { kind: 'inactive' } | { kind: 'requesting' }
// | { kind: 'active'; startedAt: number }
// | { kind: 'error'; error: { code: 'screen_denied' | 'screen_start_failed'; message: string } }
switch (state.kind) {
  case 'active':
    showSharingSince(state.startedAt);
    break;
  case 'error':
    showShareError(state.error.message);
    break;
}

The state is a discriminated object, not a bare string — switch on state.kind, and the active and error arms carry their payloads (startedAt, the failing error).

To show the person what they are sharing, render the SDK's own capture — don't call getDisplayMedia a second time (a second capture prompts again, and anything captured outside the SDK never reaches the agent):

// Plain TypeScript: the stream is on the session while a share is active.
const stream = session.getScreenShareStream();  // MediaStream | null

// React: the hook pairs the lifecycle state with the stream.
function SharePreview() {
  const { state, stream } = useScreenShare();
  const videoRef = useRef<HTMLVideoElement | null>(null);
  useEffect(() => {
    if (videoRef.current) videoRef.current.srcObject = stream;
  }, [stream]);
  if (state.kind !== 'active') return null;
  return <video ref={videoRef} autoPlay playsInline muted />;
}

The SDK owns the stream's lifecycle — end it with session.stopScreenShare(), never by stopping its tracks.

A denied picker surfaces as a screen_denied screen-share state — treat it as a normal user choice, not a failure. getVisionInputStatus() tells you whether the model has a fresh frame, which is the honest signal for a "👁 agent can see your screen" indicator.

Python drives capture itself — the SDK publishes the track, you supply the frames.

There's no OS capture helper, since a server process rarely has a display. Feed frames from whatever surface you render — a headless browser, a rendered canvas, or a file. See Video for a worked example.

Swift splits capture from transport, because on Apple platforms you own the capture pipeline (ScreenCaptureKit on macOS, ReplayKit on iOS):

try await session.startScreenShare()          // create the track

// From your capture callback (any thread):
session.pushScreenShareFrame(sampleBuffer)    // nonisolated, safe from capture threads

await session.stopScreenShare()

The track publishes on the first frame you push, not at startScreenShare() — so a capture pipeline that never produces frames never publishes an empty track. Two hooks matter in production:

• setScreenShareFrameProcessor(_:) — transform frames before they leave the device (redaction, watermarking, cropping a region).
• onScreenShareFailed(_:) — observe SFU rejections or codec failures without tearing down the session.