ElevenLabs Brings Live Translation to Phone Calls Across Multiple Languages

ElevenLabs has announced Eleven Realtime Translation, an AI service that translates spoken language during a conversation and reads the translation aloud. It is launching as an alpha for ElevenAPI business customers. Telecoms and customer service teams can integrate the technology into their own systems; consumers aren’t getting a new calling app.

According to ElevenLabs’ announcement, each participant speaks their own language and hears the other person in their preferred language. The company is targeting scenarios such as customer calls where the caller and employee don’t speak the same language.

A customer service team could use the translation service to bridge the gap between callers and employees. ElevenLabs says this could spare customers from waiting until someone with the right language skills is available. It also points to conferences, where attendees could follow a speaker in their own language.

For businesses, this could change how they provide international support. With translation assistance, an employee who knows the product could help more customers. Whether that actually saves time depends on how well the system performs in real conversations: a misunderstood complaint can mean more work, not less.

Business access is key. Organizations must first connect the translation feature to their phone system or other software. The announcement does not mean the service will automatically work with existing phone calls.

ElevenLabs uses Eleven v4 Turbo by default for spoken output. The model supports more than 90 languages. AI Wereld previously covered how Eleven v4 and Turbo generate multilingual speech. The new translation feature applies that speech technology to a different challenge: understanding each other while a conversation is still underway.

The ElevenLabs product information makes an important distinction: the system does not automatically preserve the speaker’s original voice exactly. Providers can select an output voice from the library or use a cloned voice to approximate a particular speaker.

Two-way translation also requires configuration by the provider. ElevenLabs gives the example of setting up a separate connection for each participant. The calling service must decide which audio goes to which translation—and what each person hears.

Developers can choose how early the translation starts. The technical documentation describes three settings: low, medium and high. The fastest setting starts translating earlier, while someone is still speaking. The more measured setting waits for more of the sentence, with the aim of improving translation quality, according to ElevenLabs.

It’s a practical trade-off. Speed is useful for a short answer. For a complicated explanation, waiting to hear how the sentence ends could help. The announcement offers no independent real-world test showing how these settings perform across different languages.

Along with audio, the connection provides text of both the original speech and the translation, allowing developers to add features such as subtitles. For sensitive conversations, a setting can keep transcripts, translations and spoken text out of application logs and session history. According to the documentation, this setting is off by default.

According to ElevenLabs, Deutsche Telekom is already working on new calling features for Magenta AI Call Assistant. The partnership didn’t start this week: in March, AI Wereld reported on the Magenta assistant for phone calls.

In its MWC documentation, Deutsche Telekom had already described network-based translation that requires no extra apps. The latest announcement shows ElevenLabs continuing to develop the underlying technology. It does not confirm that the new translation feature is generally available to all Telekom customers.

Ultimately, a calling service is only useful if people understand each other correctly. Businesses will need to assess more than whether the voice sounds natural: amounts, names, interruptions and how the system recovers from misunderstandings all need to be tested in real-world conditions.