LLM Helper

RTVI Client Web includes a LLM helper for typical LLM tasks and workflows.

Using the LLM helper

import { RTVIClient, LLMHelper } from "realtime-ai";

const rtviClient = new RTVIClient({
  // ...
});
const llmHelper = new LLMHelper({
  callbacks: {
    // ...
  },
});
rtviClient.registerHelper("llm", llmHelper);

Actions

All of the below are abstracted actions.

As with all actions, they can be awaited or chained with .then().

If the bot is unable to process the action, it will trigger the onMessageError callback and MessageError event.

getContext()

llm:get_context

Retrieve LLM context from bot. Returns Promise<LLMContext>

const llmHelper = rtviClient.getHelper("llm") as LLMHelper;
const ctx = await llmHelper.getContext();

// >  { messages?: LLMContextMessage[]; tools?: []; }

setContext()

llm:set_context

Replaces the current LLM context with the provided one. Returns Promise<boolean>.

context · LLMContext · required

LLMContext option to set.

interrupt · boolean · default: "false"

Interrupt the current conversation and apply the new context immediately.

const llmHelper = rtviClient.getHelper("llm") as LLMHelper;
await llmHelper.setContext(
  {
    messages: [
      {
        role: "system",
        content: "Your are a helpful assistant",
      },
      {
        role: "user",
        content: "Tell me a joke",
      },
    ],
  },
  false | true
);

// >  true | false (if error)

appendToMessages()

llm:append_to_messages

Append a new message to the existing context. Returns Promise<boolean>.

context · LLMContextMessage · required

New message to apply to the context.

runImmediately · boolean · default: "false"

Apply the new message immediately, or wait until the current turn concludes.

const llmHelper = rtviClient.getHelper("llm") as LLMHelper;
await llmHelper.appendToMessages(
  {
    role: "user",
    content: "Tell me a joke",
  },
  false | true
);

// >  true | false (if error)

Callbacks and events

onLLMJsonCompletion: (jsonString: string) => void;
onLLMFunctionCall: (func: LLMFunctionCallData) => void;
onLLMFunctionCallStart: (functionName: string) => void;
onLLMMessage: (message: LLMContextMessage) => void;