> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rtvi.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Callbacks and events

RTVI clients listen for messages and events from the bot via the transport. This allows you to respond to changes in state, errors, and other events.

## Callbacks

RTVI clients exposes callback hooks that can be defined on the [constructor](./client-constructor).

```typescript
const rtviClient = new RTVIClient({
  // ...
  callbacks: {
    onBotReady: bot_ready_handler_func,
  },
});
```

### Messages and errors

<ParamField path="onGenericMessage" type="data:unknown">
  If the client receives an unknown message type from the transport (see
  [messages and events](./messages)), it can be handled here.
</ParamField>

<ParamField path="onMessageError" type="message:RTVIMessage">
  Response error when an action fails or an unknown message type is sent from
  the client.
</ParamField>

<ParamField path="onError" type="message:RTVIMessage">
  Error signalled by the bot. This could be due to a malformed config update or
  an unknown action dispatch or the ability to complete the requested action.
</ParamField>

### State and connectivity

<ParamField path="onTransportStateChanged" type="state:TransportState">
  Provides a `TransportState` string representing the connectivity state of the
  local client. See [transports](./transports) for state explanation.
</ParamField>

<ParamField path="onConnected">
  Local user successfully established a connection to the transport.
</ParamField>

<ParamField path="onDisconnected">
  Local user disconnected from the transport, either intentionally by calling
  `rtviClient.disconnect()` or due to an error.
</ParamField>

<ParamField path="onBotConnected">
  Bot connected to the transport and is configuring. Note: bot connectivity does
  not infer that its pipeline is yet ready to run. Please use `onBotReady`
  instead.
</ParamField>

<ParamField path="onParticipantJoined" type="participant: Participant">
  A non-bot participant joined the session.
</ParamField>

<ParamField path="onParticipantLeft" type="participant: Participant">
  A non-bot participant left the session. Note: excluded local participant.
</ParamField>

<ParamField path="onBotReady" type="botReadyData:BotReadyData">
  The bot has been instantiated, its pipeline is configured, and it is receiving
  user media and interactions. This method is passed a `BotReadyData` object,
  which contains a `config RTVIClientConfigOption[]` array and the RTVI
  `version` number. It is recommended to hydrate your application with the
  passed config in case the bot alters any configuration based on its
  implementation. Since the bot is remote and may be using a different version
  of RTVI than the client, you can use the passed `version` string to check for
  compatibility.
</ParamField>

<ParamField path="onBotDisconnected" type="participant: Participant">
  Bot disconnected from the transport. This may occur due to session expiry, a
  pipeline error or because the local participant left the session.
</ParamField>

### Configuration

<ParamField path="onConfigUpdated" type="config:RTVIClientConfigOption[]">
  Sent when the bots configuration changes. This is most likely in response to a
  user `updateConfig` action, but can occur within a bots pipeline in response
  to actions or messages.
</ParamField>

<ParamField path="onConfigDescribe" type="configDescription:unknown">
  A list of available configuration options for each service. Sent in response to a user `describeConfig()` call.

  ```js
  "config": [
    {
        "service": "llm",
        "options": [
          { "name": "model", "type": "string" },
          { "name": "messages", "type": "array" },
          ...
        ]
    },
    {
        "service": "tts",
        "options": [
          { "name": "voice", "type": "string" },
          ...
        ]
    },
    ...
  ]
  ```
</ParamField>

### Configuration

<ParamField path="onActionsAvailable" type="actions: unknown">
  Response to `describeActions()` call. Lists available actions for the bot and
  their associated arguments and types.
</ParamField>

### Media and devices

<ParamField path="onAvailableMicsUpdated" type="mics:MediaDeviceInfo[]">
  Lists available local media microphone devices. Triggered when a new device
  becomes available or in response to `rtviClient.initDevices()`.
</ParamField>

<ParamField path="onAvailableCamsUpdated" type="cams:MediaDeviceInfo[]">
  Lists available local media camera devices. Triggered when a new device
  becomes available or in response to `rtviClient.initDevices()`.
</ParamField>

<ParamField path="onMicUpdated" type="mic:MediaDeviceInfo">
  User selected a new microphone as their selected/active device.
</ParamField>

<ParamField path="onCamUpdated" type="cam:MediaDeviceInfo">
  User selected a new camera as their selected/active device.
</ParamField>

<ParamField path="onTrackStarted" type="track: MediaStreamTrack, participant:Participant">
  Media track from a local or remote participant was started and playable. Can
  be either an audio or video track.
</ParamField>

<ParamField path="onTrackStopped" type="track: MediaStreamTrack, participant:Participant">
  Media track from a local or remote participant was stopped and no longer
  playable.
</ParamField>

### Audio and Voice Activity

<ParamField path="onLocalAudioLevel" type="level:number">
  Local audio gain level (0 to 1).
</ParamField>

<ParamField path="onRemoteAudioLevel" type="level: number, participant: Participant">
  Remote audio gain level (0 to 1). Note: if more than one participant is
  connected to the transport, the `participant` property details the associated
  peer.
</ParamField>

<ParamField path="onBotStartedSpeaking">
  The bot started speaking/sending speech audio.
</ParamField>

<ParamField path="onBotStoppedSpeaking">
  The bot stopped speaking/sending speech audio.
</ParamField>

<ParamField path="onUserStartedSpeaking">
  The local user started speaking. This method is more reliable than using audio
  gain and is the result of the bot's VAD (voice activity detection) model. This
  provides a more accurate result in noisy environments.
</ParamField>

<ParamField path="onUserStoppedSpeaking">
  The local user stopped speaking, indicated by the VAD model.
</ParamField>

### Transcription

<ParamField path="onUserTranscript" type="transcript:TranscriptData">
  Transcribed local user input (both partial and final).

  Callback receives a `TranscriptData` object:

  ```typescript
  export type TranscriptData = {
    text: string;
    final: boolean;
    timestamp: string;
    user_id: string;
  };
  ```
</ParamField>

<ParamField path="onBotTranscript" type="text:BotLLMTextData">
  Finalized bot output text generated by the LLM. Sentence aggregated.
</ParamField>

<ParamField path="onBotLLmText" type="text:BotLLMTextData">
  Streamed LLM token response text generated by the LLM service.
  Please note that this event is not dispatched when in a connected state.
  This callback is intended to be used for offline, single-turn actions, where not TTS service is connected.

  When connected:

  * If you want to align words as they are spoken by the bot, use `onBotTtsText` instead.

  * If you want the full text of the LLM response, use `onBotTranscript`.
</ParamField>

<ParamField path="onBotLlmStarted">LLM service inference started.</ParamField>

<ParamField path="onBotLlmStopped">LLM service inference concluded.</ParamField>

<ParamField path="onBotTtsText" type="text: BotTTSTextData">
  If your TTS service supports streamed responses over sockets, the text
  parameter contains the words from TTS service as they are spoken If you are
  using a HTTP based TTS service, the text parameter will contain the full text
  of the TTS response.
</ParamField>

<ParamField path="onBotTtsStarted">TTS service started inference.</ParamField>

<ParamField path="onBotTtsStarted">TTS service inference concluded.</ParamField>

### Other

<ParamField path="onMetrics" default="data:PipecatMetricsData">
  Pipeline data provided by Pipecat. Please see [Pipecat
  documentation](https://docs.pipecat.ai) for more information.
</ParamField>

<ParamField path="onStorageItemStored" default="data: StorageItemStoredData">
  A message or messages item was saved to the storage / database. Assumed you
  have a storage context processor as part of your bot pipeline. Data references
  the saved message.
</ParamField>

## Events

Callbacks methods are also defined as events that map to messages/actions received from the bot. The following events are available:

```javascript
MessageError = "messageError";
Error = "error";

Connected = "connected";
Disconnected = "disconnected";
TransportStateChanged = "transportStateChanged";

Config = "config";
ConfigDescribe = "configDescribe";
ActionsAvailable = "actionsAvailable";

ParticipantConnected = "participantConnected";
ParticipantLeft = "participantLeft";
TrackStarted = "trackStarted";
TrackedStopped = "trackStopped";

AvailableCamsUpdated = "availableCamsUpdated";
AvailableMicsUpdated = "availableMicsUpdated";
CamUpdated = "camUpdated";
MicUpdated = "micUpdated";

BotConnected = "botConnected";
BotReady = "botReady";
BotDisconnected = "botDisconnected";
BotStartedSpeaking = "botStartedSpeaking";
BotStoppedSpeaking = "botStoppedSpeaking";
RemoteAudioLevel = "remoteAudioLevel";

UserStartedSpeaking = "userStartedSpeaking";
UserStoppedSpeaking = "userStoppedSpeaking";
LocalAudioLevel = "localAudioLevel";

UserTranscript = "userTranscript";
UserText = "userText";
BotTranscript = "botTranscript";

BotText = "botText";
BotLlmStarted = "botLlmStarted";
BotLlmStopped = "botLlmStopped";

BotTtsText = "botTtsText";
BotTtsStarted = "botTtsStarted";
BotTtsStopped = "botTtsStopped";

LLMFunctionCall = "llmFunctionCall";
LLMFunctionCallStart = "llmFunctionCallStart";
LLMJsonCompletion = "llmJsonCompletion";

Metrics = "metrics";
StorageItemStored = "storageItemStored";
```

Handlers for these can be bound on the client like so:

```typescript
import { RTVIEvent } from "realtime-ai";

function handleBotReady() {
  console.log("Bot is ready!");
}

// Bind an event handler
rtviClient.on(RTVIEvent.BotReady, handleBotReady);
// or
rtviClient.on("botReady", handleBotReady);

// Unbind an event handler
rtviClient.off(RTVIEvent.BotReady, handleBotReady);
```