Actions

Actions are service-specific messages that are dispatched to the bot in order to trigger certain pipeline behaviour.

Some examples of actions include:

• tts:say - Speak a message using text-to-speech.
• tts:interrupt - Interrupt the current text-to-speech message.
• llm:append_to_messages - Append a context to the current LLM messages.

Under the hood, actions are blobs of data that are sent via the transport layer to the bot. The bot listens for and then processes the action to perform the necessary operations.

RTVI-enabled bots will have defined actions available for the user to invoke in their client.

Actions do not directly trigger events or callbacks in the client, but instead return a Promise that resolves once the bot has processed the action. Actions are useful ways of extending the functionality of an RTVI bot without needing to modify the client.

Actions differ from messages in that:

• They are service specific
• They do not trigger callbacks or events
• They return a Promise that resolves once the bot has processed the action.

Obtaining available actions

To obtain a list of available actions, you can use the describeActions method on the RTVI client.

const actions = await rtviClient.describeActions();

This will return an array of action objects that you can use to determine which actions are available.

{
  "label": "rtvi-ai",
  "type": "actions-available",
  "id": "UNIQUE_ID_FROM_REQUEST",
  "data": {
    "actions": [
      {
	      "service": "tts",
	      "action": "say",
	      "arguments": [
		      { "name": "text", "type": "string" },
		      { "name": "interrupt", "type": "bool" },
		      ...
	      ]
      },
      ...
    ]
  }
}

Anatomy of an action

An action object has the following properties:

• service - The service that the action belongs to.
• action - The name of the action, as defined by the bot.
• arguments - An array of argument objects that the action accepts.

When the client dispatches an action, it will pass the action name and arguments to the bot, alongside a unique ID that is referenced on response to resolve the Promise.

RTVI clients maintain an internal queue of actions that are dispatched to the bot. The action is sent as JSON data via the transport layer to the bot, which processes the action.

Once the action has been processed the bot will send a response message with the same unique ID that the client uses to reference the action queue to resolve the Promise.

Dispatching an action

const someAction = await rtviClient.action({
  service: "tts",
  action: "say",
  arguments: [
    { name: "text", value: "Hello, world!" },
    { name: "interrupt", value: false },
  ],
});

// > Promise<RTVIActionResponse>

An action is resolved or rejected with the following:

{
  "label": "rtvi-ai",
  "type": "action-response",
  "id": "UNIQUE_ID_FROM_REQUEST",
  "data": {
    "result": BOOL | NUMBER | STRING | ARRAY | OBJECT
  }
}

If an action is unable to be processed, the bot will return a error-response typed message with the same unique ID assigned by the client, triggering a rejection.

You can handle error responses in multiple ways:

• onMessageError callback
• MessageError event
• try / catching the Promise

try {
	const someAction = await rtviClient.action({...});
} catch(e) {
	console.error(e);
}

Action response data

Some actions resolve with data. This data is specific to the action and is defined by the bot.

A successful action will resolve with RTVIActionResponse:

{
  "label": "rtvi-ai",
  "type": "action-response",
  "id": "UNIQUE_ID_FROM_REQUEST",
  "data": {
	"result": "Hello, world!"
  }
}

Awaiting an action will return the RTVIActionResponse object, which contains a result property with any associated data.

try {
	const someAction = await rtviClient.action({...});
	console.log(someAction.data.result);
} catch(e) {
	console.error(e);
}