YouTube adding AI dub technology to translate videos for creators

Joel Loynds
ai generated image of a robot in a recording booth with the youtube logo behind itFirefly AI

A new feature for YouTube will allow creators to dub their videos into other languages with an upcoming AI integration.

At VidCon 2023, YouTube announced that it would begin letting creators dub their videos into other languages via the use of AI. The software, developed by Aloud and part of Google’s Area 120 incubator, will open up content to a much wider audience once implemented.

Speaking with The Verge, YouTube’s Amjad Hanif said that the tool is already being tested with hundreds of creators and that the AI program supports just a few languages right now.

Article continues after ad

Right now, you’ll find Spanish, Portuguese, and English available for translation purposes.

Currently, YouTube allows viewers to use automatic captions if the original uploader hasn’t provided subtitles. However, this can be very limited depending on the accents, and translating subtitles between languages is spotty or leads to worse accusations.

You can already try out the function yourself, as a science channel, Amoeba Sisters, seems to already have a video using it. To do so, just click the little gear, and audio tracks should be an option.

Sign up to Dexerto for free and receive:
Fewer Ads|Dark Mode|Deals in Gaming, TV and Movies, and Tech
Article continues after ad

The results are very clearly AI-generated, with a robotic undertone as the Spanish voice tries to keep up with the English one.

YouTube introduces new AI dubbing tool for creators

Hanif mentions that the audio should improve in 2024, as Aloud continues to develop it. This will begin to mimic inflections, expressions, and lip sync.

While it’s unclear how YouTube would implement lip-synced video, we’ve already seen Nvidia alter the human body on the fly to keep eyes looking directly at the camera.

Article continues after ad

However, we suspect that it’ll function similarly to how YouTube is implementing the dubbing process. Aloud takes the audio, similar to the automatic captions. Once it has been processed and transcribed into text, you can edit and make adjustments. Finally, the AI voice is overlayed and integrated into the content.

Related Topics

About The Author

E-Commerce Editor. You can get in touch with him over email: joel.loynds@dexerto.com. He's written extensively about video games and tech for over a decade for various sites. Previously seen on Scan, WePC, PCGuide, Eurogamer, Digital Foundry and Metro.co.uk. A deep love for old tech, bad games and even jankier MTG decks.