Microsoft has filed a patent for an artificial intelligence-based tool that can edit live video broadcasts in real time. The system can replace or delete the speaker's words while simultaneously correcting facial expressions and lip movements in the frame.
Microsoft has filed a patent for an artificial intelligence-based tool that can edit live video broadcasts in real time. The system can replace or delete the speaker's words while simultaneously correcting facial expressions and lip movements in the frame.
As Futurism notes, citing patent tracking service Patentlyze, the inventors describe the technology as “enhancing videos or presentations in augmented reality (AR) format during live broadcasts using artificial intelligence in near real-time.”
According to the documentation, special AI agents will identify unwanted fragments and adjust the image of the speaker's face and mouth. The model will edit the audio and video immediately before and after the changed section to create the smoothest possible transition without visible glue.
Among the basic application scenarios are simultaneous translation into other languages, cleaning speech from parasitic words, and correcting stuttering and speech defects.
At the same time, the functionality involves much deeper intervention in the content. The system will be able to cut out colloquial expressions, dialectisms, tactless, offensive or incorrectly pronounced words from the live broadcast. The AI will be able to search for options to replace expressions directly on the Internet.
The authors of the development have foreseen the risks of misinformation and distortion of facts: the patent states that the original recording of the event will be stored to ensure "objectivity." It is expected that the technology can be used during online lectures, corporate presentations, sports broadcasts and TV shows.
