好网站,从官网开始
企业认领/联系与合作
官网导航 提交官网
B

BuboGPT官网

BuboGPT是一个创新的大型多模态语言模型,它通过结合视觉、音频和文本输入,提供了一种全新的交互体验。

官网资料

网站名称
BuboGPT
身份核实
尚未完成企业认领核实
最近更新
链接检测
尚未检测

关于 BuboGPT

BuboGPT BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs Yang Zhao * , Zhijie Lin * , Daquan Zhou , Zilong Huang , Jiashi Feng , Bingyi Kang + , &#x25B6 Bytedance Inc. &nbsp&nbsp * Equal Contribution &nbsp&nbsp + Project Lead arXiv Code Demo Dataset Model Video --> BuboGPT is an advanced Large Language Model (LLM) that incorporates multi-modal inputs including text, image and audio, with a unique ability to ground its responses to visual objects. It demonstrates remarkable chat abilities for arbitrary image-audio data understanding, whether aligned or unaligned. &#x25B6 Bubo owls are well known for having strong vision and hearing abilities that help them thrive. --> --> --> --> --> Abstract LLMs have demonstrated remarkable abilities at interacting with humans through language, especially with the usage of instruction-following data. Recent advancements in LLMs, such as MiniGPT-4, LLaVA, and X-LLM, further enlarge their abilities by incorporating multi-modal inputs, including image, video, and speech. Despite their effectiveness at generating precise and detailed language understanding of the given modality signal, these LLMs give up the ability to ground specific parts of inputs, thus only constructing a coarse-grained mapping. However, explicit and informative correspondence between text and other modalities will not only improve the user experience but also help to expand the application scenario of multi-modal LLMs. BuboGPT Architecture . We build a multi-modal LLM, BuboGPT for multi-modal understanding including image, audio and text by learning a common semantic space and further explore the fine-grained relation between different visual objects and different modalities. Multimodal Instruct Data . We construct a high-quality multi-modal instruction-tuning dataset including fine-grained audio descriptions and cross-modal sound localization, and introduce both positive and negative image-audio pairs for semantic matching to facilitate the cross-modal understanding.. BuboGPT Architecture --> As the figure shown, we perform joint multi-modal understanding and chatting for text, vision and audio, which is achieved by learning a shared representation space that aligns well with pre-trained Vicuna . We also build an off-the-shelf visual grounding pipeline to explore the fine-grained relation between different visual objects and modalities. The framework of BuboGPT. [HuggingFace Dataset] .--> --> --> --> --> Data file name --> File Size --> Sample Siz

访问提示

本站仅提供官网信息与访问导航,不代表对该网站的产品、服务或交易作出担保。访问外部网站时,请核对网址,注意账号和财产安全。

同类型网站 366

查看分类

AutoGPT 是面向自主智能体的开源平台,支持用自然语言或可视化流程创建、运行和管理能够持续执行任务的智能体。

支持多 AI Agent 框架的可视化管理面板,支持 OpenClaw 和 Hermes Agent 双引擎,内置智能助手一键安装诊断

让更多人找到你的官网

提交网站信息,建立清晰、完整的官网资料。

免费提交官网

咨询提交、推广、认领

客服微信二维码,长按识别或保存后使用微信扫一扫

扫一扫添加微信,沟通官网导航提交、认领、推广需求。

可长按识别,或保存图片后在微信扫一扫中识别。

添加时可备注“官网导航”,便于沟通您的企业需求。

保存二维码 打开原图