Abstract
How to enable robots to predict the outcomes of their own motion intentions in three-dimensional space has been a fundamental problem in embodied intelligence. To explore more general spatial imagination capabilities, here we present AirScape, the first world model designed for six-degree-of-freedom aerial agents. AirScape predicts future observation sequences based on current visual inputs and motion intentions. Specifically, we construct an dataset for aerial world model training and testing, which consists of 11k video-intention pairs. This dataset includes first-person-view videos capturing diverse drone actions across a wide range of scenarios, with over 1,000 hours spent annotating the corresponding motion intentions. Then we develop a two-phase training schedule to train a foundation model -- initially devoid of embodied spatial knowledge -- into a world model that is controllable by motion intentions and adheres to physical spatio-temporal constraints.
Abstract (translated)
如何使机器人能够在三维空间中预测自己运动意图的结果,一直是具身智能中的一个基本问题。为了探索更广泛的时空想象能力,我们在此介绍AirScape,这是首个为具备六自由度的空中代理设计的世界模型。AirScape基于当前视觉输入和运动意图来预测未来的观察序列。具体而言,我们构建了一个用于空中国际象棋世界模型训练与测试的数据集,其中包括了11,000对视频-意图组合。该数据集包含了第一人称视角的视频,捕捉到了无人机在各种场景中的多样动作,并且花费超过1,000小时标注对应的运动意图。然后,我们开发了一种两阶段的训练计划,将一个初始不包含具身时空知识的基础模型训练成可以根据运动意图控制、并遵守物理时空约束的世界模型。
URL
https://arxiv.org/abs/2507.08885