Live Captions Translation

Use Live Captions Translation to translate real-time captions into different languages for participants in a Vonage Video session.

This page includes the following sections:

Overview
Session-level translation
Subscriber-level translation
Supported languages
Notes

Overview

Live Captions Translation extends the Live Captions feature by adding real-time translation of captions. With Live Captions, audio is transcribed in the spoken language. With Live Captions Translation, those captions can be automatically translated into a different language for participants.

Note: Live Captions Translation is available starting with version 2.36.0 of the Client SDKs.

Before using Live Captions Translation, make sure you are familiar with the Live Captions feature, including how to start captions using the REST API and how to subscribe to captions in the client SDKs.

Translation can be enabled at two levels:

Session-level translation — Set a translation language for all participants when starting captions via the server-side API.
Subscriber-level translation — Individual subscribers choose their own translation language using the client SDKs, independently of the session-level setting.

These two levels can be used together or independently.

Session-level translation

When calling the Start Captions REST API (POST /v2/project/<applicationId>/captions), you can include an optional targetLanguageCode parameter in the request body:

{
  "sessionId": "<session-id>",
  "token": "A valid token with the role set to moderator",
  "languageCode": "en-US",
  "targetLanguageCode": "es-ES",
  "maxDuration": 1800,
  "partialCaptions": true
}

targetLanguageCode (String) — (Optional) The BCP-47 language code for the target translation language (see the supported languages list). When set, all participants in the session will receive translated captions in the specified language instead of the original transcribed language.

Behavior:

If targetLanguageCode is not set (or omitted), captions work as normal — participants receive captions in the original spoken language defined by languageCode.
If targetLanguageCode is set, all participants in the session receive captions translated into the specified language by default.
A subscriber who sets their own translation language (see Subscriber-level translation) will receive captions in their chosen language, overriding the session-level targetLanguageCode.

You can also set the targetLanguageCode using the Vonage Video server SDKs when calling the start captions method.

Subscriber-level translation

Subscribers can choose their own translation language using the client SDKs, and can change it at any time during a session. This works independently of the session-level targetLanguageCode.

You can configure subscriber-level translation using any of the Vonage Video API client SDKs, including Web, iOS, Android, Windows, macOS, Linux, and React Native. Below are reference examples for each platform.

Setting the translation language (Web)

The default captions language is determined by the languageCode (or targetLanguageCode) set in the Start Captions API. To change the translation language for a specific subscriber, call setCaptionsTranslationLanguage with a valid BCP-47 language code. Pass null to revert to the default language.

// Example BCP-47 language code for captions translation
const captionsLanguageCode = 'es-ES'

// Set the subscriber's captions translation language
try {
    await subscriber.setCaptionsTranslationLanguage(captionsLanguageCode)
} catch (err) {
    console.error(`Failed to set captions translation language: ${err.message}`)
}

// Pass null to revert to the default language
try {
    await subscriber.setCaptionsTranslationLanguage(null)
} catch (err) {
    console.error(`Failed to revert captions translation language: ${err.message}`)
}

Getting the current translation language (Web)

Use getCaptionsTranslationLanguage to check the subscriber's currently selected translation language. It returns the selected BCP-47 language code, or null if no translation language has been set.

const captionsLanguage = subscriber.getCaptionsTranslationLanguage()
if (captionsLanguage) {
    console.log(`Captions translation language is set to: ${captionsLanguage}`)
} else {
    console.log('No translation language set; using the session default.')
}

Setting the translation language (Android)

Use the setCaptionsTranslationLanguage() method of the SubscriberKit object to set the translation language. Pass null to revert to the default language.

// Set the subscriber's captions translation language
subscriber.setCaptionsTranslationLanguage("es-ES");

// Revert to the default language
subscriber.setCaptionsTranslationLanguage(null);

Getting the current translation language (Android)

Use the getCaptionsTranslationLanguage() method to check the subscriber's currently selected translation language. It returns the selected BCP-47 language code, or null if no translation language has been set.

String captionsLanguage = subscriber.getCaptionsTranslationLanguage();
if (captionsLanguage != null) {
    Log.d(TAG, "Captions translation language is set to: " + captionsLanguage);
} else {
    Log.d(TAG, "No translation language set; using the session default.");
}

Setting the translation language (iOS)

Set the captionsTranslationLanguage property of the OTSubscriberKit object to the desired BCP-47 language code. Set it to nil to revert to the default language.

// Set the subscriber's captions translation language
subscriber.captionsTranslationLanguage = @"es-ES";

// Revert to the default language
subscriber.captionsTranslationLanguage = nil;

Getting the current translation language (iOS)

Read the captionsTranslationLanguage property to check the subscriber's currently selected translation language. It returns the selected BCP-47 language code, or nil if no translation language has been set.

NSString *captionsLanguage = subscriber.captionsTranslationLanguage;
if (captionsLanguage) {
    NSLog(@"Captions translation language is set to: %@", captionsLanguage);
} else {
    NSLog(@"No translation language set; using the session default.");
}

Setting the translation language (Windows)

Set the CaptionsTranslationLanguage property of the Subscriber object to the desired BCP-47 language code. Set it to null to revert to the default language.

// Set the subscriber's captions translation language
subscriber.CaptionsTranslationLanguage = "es-ES";

// Revert to the default language
subscriber.CaptionsTranslationLanguage = null;

Getting the current translation language (Windows)

Read the CaptionsTranslationLanguage property to check the subscriber's currently selected translation language. It returns the selected BCP-47 language code, or null if no translation language has been set.

string captionsLanguage = subscriber.CaptionsTranslationLanguage;
if (captionsLanguage != null)
{
    Console.WriteLine($"Captions translation language is set to: {captionsLanguage}");
}
else
{
    Console.WriteLine("No translation language set; using the session default.");
}

Supported languages

Live Captions Translation uses the same BCP-47 language codes as Live Captions. See the supported languages list for the full list of available languages.

Notes

Live Captions Translation requires Live Captions to be active for the session. You must start captions using the Start Captions API before translation is available.
Setting a subscriber-level translation language overrides the session-level targetLanguageCode for that subscriber only. Passing null reverts the subscriber to the default behavior (session-level targetLanguageCode if set, otherwise the original languageCode).
For the base Live Captions feature documentation — including architecture, publishing captions, subscribing to captions, status webhooks, and known issues — see the Live Captions guide.