AutoGPT 是面向自主智能体的开源平台,支持用自然语言或可视化流程创建、运行和管理能够持续执行任务的智能体。
关于 VideoWorld
--> VideoWorld: Exploring Knowledge Learning from Unlabeled Videos --> 图片居中 More Research HyperNeRF Nerfies LatentFusion PhotoShape --> VideoWorld: Exploring Knowledge Learning from Unlabeled Videos Zhongwei Ren 1,2 , Yunchao Wei 1 † , Xun Guo 2 , Yao Zhao 1 , Bingyi Kang 2 , Jiashi Feng 2 , Xiaojie Jin 2 † ‡ 1 Beijing Jiaotong University, 3 ByteDance Seed ( † Correspondence, ‡ Project Lead) Paper arXiv Video Code Data --> Figure 1: VideoWorld explores learning knowledge from raw videos, ranging from task-specific rules to high-level reasoning and planning capabilities.Compared to other learning methods: reinforcement learning (RL), supervised learning (SL) and text-based learning, it offers three advantages: 1. better generalization with unified visual representation for various tasks and interfaces, 2. lower mannual annotation burden, and 3. richer real-world information than text description. Figure 2: VideoWorld plays Go by generating next board state. Figure 3: VideoWorld controls robotic arms across different environments. Abstract This work explores whether a deep generative model can learn complex knowledge solely from visual input, in contrast to the prevalent focus on text-based models like large language models (LLMs). We develop VideoWorld, an autoregressive video generation model trained on raw video data, and test its knowledge acquisition abilities in video-based Go and robotic control tasks. Our experiments reveal two key findings: (1) video-only training provides sufficient information for learning knowledge, including rules, reasoning and planning capabilities, and (2) increasing the compactness of visual representations significantly enhances learning efficiency. To improve both the efficiency and efficacy of knowledge learning, we introduce the Latent Dynamics Model (LDM) as a key component of VideoWorld. Remarkably, VideoWorld reaches a 5-dan professional level in the Video-GoBench with just a 300-million-parameter model, without relying on search algorithms or reward mechanisms typical in reinforcement learning. In robotic tasks, VideoWorld effectively learns diverse control operations and generalizes across environments, approaching the performance of oracle models in CALVIN and RLBench. This study opens new avenues for knowledge acquisition from visual data, with all code, data, and models to be open-sourced for further research. --> 🔥 Highlights 1. We explore for the first time, whether video generation models can learn sophisticat
访问提示
本站仅提供官网信息与访问导航,不代表对该网站的产品、服务或交易作出担保。访问外部网站时,请核对网址,注意账号和财产安全。
同类型网站
查看分类百川智能发布的金融大模型,金融场景能力领先GPT-4o近20%
Build a Large Language Model (From Scratch) 是一款AI 大模型方向的 AI 工具,适合主流大语言模型与开源模型。
支持多 AI Agent 框架的可视化管理面板,支持 OpenClaw 和 Hermes Agent 双引擎,内置智能助手一键安装诊断