Technology, Smartphones, Games

Microsoft's New Real Time AI Video Translation Feature for Edge


This week, Microsoft introduced a new AI-powered video translation feature for its Edge browser. This innovative tool will automatically create foreign language subtitles and voiceovers for videos across several popular websites, including YouTube, LinkedIn (which is owned by Microsoft), Bloomberg, Reuters, and Coursera.

To showcase how this feature works, Microsoft posted a short demonstration video. In the video, an Edge user watches a LinkedIn video of a Microsoft Ignite keynote by CEO Satya Nadella. The user clicks on a translation icon at the top of the video, which brings up a menu with options for translating the audio or adding subtitles from English to various other languages.

The user selects English as the input language and Spanish as the output language from the drop-down menus. After clicking "Translate," the video briefly pauses. When it resumes, a Spanish voiceover has replaced the original English audio.

When this feature launches, Edge will support translations from Spanish to English and from English into Spanish, Italian, Russian, and Hindi. Microsoft plans to add more languages in the future. Additionally, they are planning to offer real-time interpretation for news websites like Reuters, CNBC, and Bloomberg.

While some of these translations on YouTube use AI tools, they often combine AI with human oversight to ensure accuracy. However, Microsoft's solution seems to be fully automatic. This could raise concerns about the reliability of the translations. AI-generated translations, like those from Google Translate, can vary significantly in accuracy depending on the language pair.

This new translation tool in Edge aims to offer a browser-wide solution similar to the features YouTube has been testing for years. YouTube has allowed content creators to upload videos with multiple audio tracks since last year. Independent YouTubers and companies have been providing human-translated and dubbed videos on the site for some time. A small industry of startups has emerged to help popular YouTube creators dub their videos into multiple languages.

In summary, Microsoft's new feature for the Edge browser will provide automatic translations and dubs for videos on many popular sites, enhancing accessibility for non-English speakers. Initially, it will support a limited number of languages, with plans to expand and include real-time news interpretation. While promising, the feature's fully automated nature may lead to concerns about translation accuracy and reliability, similar to issues faced by other AI translation tools like Google Translate.