flâneur

Generating audio for video - Google DeepMind

deepmind.google · 936 words · saved by 1 readers

Video generation models are advancing at an incredible pace, but many current systems can only generate silent output. One of the next major steps toward bringing generated movies to life is creating soundtracks for these silent videos. Today, we're sharing progress on our video-to-audio (V2A) technology, which makes synchronized audiovisual generation possible. V2A combines video pixels with natural language text prompts to generate rich soundscapes for the on-screen action. Our V2A technology is pairable with video generation models like Veo to create shots with a dramatic score, realistic sound effects or dialogue that matches the characters and tone of a video. It can also generate soundtracks for a range of traditional footage, including archival material, silent films and more — opening a wider range of creative opportunities. Watch Prompt for audio: Cinematic, thriller, horror film, music, tension, ambience, footsteps on concrete Watch Prompt for audio: Cute baby dinosaur chirps

June 17, 2024 Research Generative Media team Video-to-audio research uses video pixels and text prompts to generate rich soundtracks Video generation models are advancing at an incredible pace, but many current systems can only generate silent output. One of the next major steps toward bringing generated movies to life is creating soundtracks for these silent videos. Today, we're sharing progress on our video-to-audio (V2A) technology, which makes synchronized audiovisual generation possible. V2A combines video pixels with natural language text prompts to generate rich soundscapes for…

saved by

related reading