Introduction

Helpers provide additional ways to interact with your bot pipeline. They offer a common interface for working with transports, events, and messages within the context of a specific service.

Since RTVI does not make any assumptions about which services a developer will leverage in their bots, helpers reduce unnecessary functionality in your voice clients and facilitate framework extensibility.

Some examples of where helpers can be useful include:

• LLM message context modificiation
• Direct text-to-speech output
• Multi-lingual workflows
• Custom convenience functions

It’s easiest to think of helpers as wrappers around existing voice client functionality that target a specific service, making your RTVI codebase less verbose.

Using a helper

Helpers must be imported, defined and registered to you voice client.

Here is an example using the included LLM helper:

import {
  LLMHelper,
  LLMHelperEvent
  VoiceMessage,
} from "realtime-ai";

const voiceClient = new VoiceClient({
    services: {
        llm: "openai"
    },
    config: [ 
        // ...
    ]
});

const llmHelper = new LLMHelper({
    callbacks: {
        onLLMContextUpdate: (messages) => {
            // ...
        }
    }
});

voiceClient.registerHelper("llm", llmHelper);

// Events
voiceClient.on(LLMHelperEvent.LLMContextUpdate, (messages) => console.log(messages));

await llmHelper.appendContext({
    role: "user",
    content: "Tell me a joke!",
 });

Registering a helper

Registering a helper provides it with a reference to your voice client. Helpers must be registered in order to use event and action dispatch / resolution.

voiceClient.registerHelper(service, helper);

service · string · required

Service targeted by the helper as part of your voice client constructor services map. An exception is thrown if no matching service can be found.

Note: targeting services is required for scenarios where a pipeline contains multiple declarations of the same service type. For example, your pipeline may have two or more LLM services registered.

helper · VoiceClientHelper · required

A helper subclass instance of VoiceClientHelper.

Retrieving a helper

The voice client holds a reference to all registered helpers. This can be useful if you do not want to hold / pass a reference across your codebase:

const llmHelper = voiceClient.getHelper("llm") as LLMHelper;

Unregistering a helper

A service in your pipeline can currently have reference to a single helper to avoid conflicting action dispatch.

You can unregister a helper from a service like so:

voiceClient.unregisterHelper("llm")

Unregistering helpers will not delete any references in your own code. Any subsequent attempts to use an unregistered helper against the service name will raise an exception.

If you register a new helper with the same service name and use a previous helper instance, you may encounter errors.

How are helper messages handled?

The VoiceClientHelper base class requires that a helper define a handleMessage method.

Any RTVI messages received from the transport that does not match native voice client message types are passed through to your client’s registered helpers:

public handleMessage(ev: VoiceMessage): void {
    switch(ev.type) {
        case MyHelperMessageType.SOME_MESSAGE:
            this._options.callbacks.onSomeMessage(ev.data);
            break;
            // etc
    }
}

Be mindful of defining a ‘default’ switch handler if you have multiple registered helpers listening for the same message type. Catching them will prevent lower order helpers from receiving the message.

---

Writing helpers

Creating your own helper classes can be useful when you want to create reusable code that targets a specific service (or series of services.)

The abstract VoiceClientHelper class is intentionally lightweight to provide as much flexibility to the developer as needed.

A typical helper definition will likely include:

• A constructor
• Callback methods
• Event and message types

A basic implementation of a helper would look like this:

myhelper.ts

import { 
  VoiceMessage,
  VoiceClientHelper,
  VoiceClientHelperOptions
} from "realtime-ai";

// --- Events
// E.g. voiceClient.on(MyHelperEvents.SomeEvent, (message) => { ... });
export enum MyHelperEvents {
  SomeEvent = "some-event",
}

// --- Message types
// E.g. await voiceClient.action({ service: "myService", action: MyHelperMessageType.SOME_MESSAGE, arguments: [...]});
export enum MyHelperMessageType {
  SOME_MESSAGE = "some-message",
}

// --- Callbacks
export type MyHelperCallbacks = Partial<{
  onSomeMessage: (message) => void;
}>;

// --- Interface and class
export interface MyHelperOptions extends VoiceClientHelperOptions {
  callbacks?: MyHelperCallbacks;
}

export class LLMHelper extends VoiceClientHelper {
  protected declare _options: MyHelperOptions;

  constructor(options: MyHelperOptions) {
    super(options);
  }

  public handleMessage(ev: VoiceMessage): void {
    switch (ev.type) {
      case MyHelperMessageType.SOME_MESSAGE:
        // dispatch action response etc
        break;
    }
  }
  public getMessageTypes(): string[] {
    return Object.values(MyHelperMessageType) as string[];
  }
}