好网站,从官网开始
企业认领/联系与合作
官网导航 提交官网
D

Draw an Audio官网

创新的视频生成音频系统,它通过先进的AI技术,能够自动分析视频内容并生成与之匹配的声音效果。

官网资料

网站名称
Draw an Audio
所属分类
AI音频音乐
身份核实
尚未完成企业认领核实
最近更新
链接检测
尚未检测

关于 Draw an Audio

Draw an Audio --> --> --> Draw an Audio: Leveraging Multi-Instruction for Video-to-Audio Synthesis Qi Yang 1 , Binjie Mao 2 , Zili Wang 1 , Xing Nie 1 , Pengfei Gao 2 , Ying Guo 2 , Cheng Zhen 2 , Pengfei Yan 2 , Shiming Xiang 1 1 Institute of Automation, Chinese Academy of Sciences, 2 Meituan Paper arXiv Code (coming soon) Abstract Foley is a term commonly used in filmmaking, referring to the addition of daily sound effects to silent films or videos to enhance the auditory experience. Video-to-Audio (V2A), as a particular type of automatic foley task, presents inherent challenges related to audio-visual synchronization. These challenges encompass maintaining the content consistency between the input video and the generated audio, as well as the alignment of temporal and loudness properties within the video. To address these issues, we construct a controllable video-to-audio synthesis model, termed Draw an Audio, which supports multiple input instructions through drawn masks and loudness signals. To ensure content consistency between the synthesized audio and target video, we introduce the Mask-Attention Module (MAM), which employs masked video instruction to enable the model to focus on regions of interest. Additionally, we implement the Time-Loudness Module (TLM), which uses an auxiliary loudness signal to ensure the synthesis of sound that aligns with the video in both loudness and temporal dimensions. Furthermore, we have extended a large-scale V2A dataset, named VGGSound-Caption, by annotating caption prompts. Extensive experiments on challenging benchmarks across two large-scale V2A datasets verify Draw an Audio achieves the state-of-the-art. --> Visualization Vanilla Foley v.s. Draw an Audio. The traditional methods produce the entire audio only from video inputs, demonstrating limitations in controllability and flexibility. Draw an Audio, in contrast, offers a more appealing alternative that employs multiple instructions to produce high-quality synchronized audio and can produce mixed audio in multi-stages, thereby exhibiting greater practical application. Architecture The architecture of Draw an Audio, which incorporates a Latent Diffusion Model (LDM) as the foundational model, a Text Condition Model for text instruction, a Masked-Attention Module (MAM) for video instruction, a Time-Loudness Module (TLM) for signal instruction. Demos It is recommended to use earphones to hear the demos videos, raise the volume and zoom in the videos. unconditional

访问提示

本站仅提供官网信息与访问导航,不代表对该网站的产品、服务或交易作出担保。访问外部网站时,请核对网址,注意账号和财产安全。

同类型网站 318

查看分类

AISong.Fun 是一个使用 AI 生成歌曲的平台,支持歌词和纯音乐创作,免费提供两次生成机会。

AssemblyAI是一个为开发者提供高精度语音转文本API及情感分析、内容摘要等高级语音智能功能的平台,帮助企业从音频和视频中提取文本洞察

让更多人找到你的官网

提交网站信息,建立清晰、完整的官网资料。

免费提交官网

咨询提交、推广、认领

客服微信二维码,长按识别或保存后使用微信扫一扫

扫一扫添加微信,沟通官网导航提交、认领、推广需求。

可长按识别,或保存图片后在微信扫一扫中识别。

添加时可备注“官网导航”,便于沟通您的企业需求。

保存二维码 打开原图