Microsoft’s new AI model neural network can make pictures ‘talk’
Over the past couple of years, generative neural networks have overcome an important stage of development, becoming more powerful and capable of creating not only images, but also videos based on text descriptions. Microsoft’s new VASA-1 algorithm will probably surprise many because it requires no description at all to work. It is enough to provide one image of a person and an audio track, on the basis of which the neural network will generate a video of a speaking person with a wide range of emotions and natural facial expressions.
The result of VASA-1 looks very natural and believable. From just one photo of a face and a voice recording, the algorithm creates a realistic video in which the person captured in the photo literally “comes to life”, and his facial expressions, lip and head movements look completely natural. Since videos created using VASA-1 are difficult to immediately distinguish from real ones, there are already concerns that the algorithm could be used by attackers to create fakes.
As for the neural network itself, its main difference from other similar algorithms is the presence of a holistic model for generating facial expressions and head movements. Microsoft conducted extensive research, including evaluating a number of new metrics. As a result, they found that the new algorithm is significantly superior to previously presented analogues in many respects.
Our method not only generates high-quality video with realistic facial expressions and head movements, but also supports online video generation of 512×512 pixels at 40 frames per second with low initial latency. This paves the way for real-time interactions with realistic avatars that mimic human conversational behavior,” Microsoft said in a statement.
In other words, the neural network can create high-quality fake videos based on just one image. So it’s not surprising that Microsoft calls VASA-1 a “research demonstration” and has no plans to bring it to the commercial market, at least not anytime soon.