Abstract
We present FlightDiffusion, a diffusion-model-based framework for training autonomous drones from first-person view (FPV) video. Our model generates realistic video sequences from a single frame, enriched with corresponding action spaces to enable reasoning-driven navigation in dynamic environments. Beyond direct policy learning, FlightDiffusion leverages its generative capabilities to synthesize diverse FPV trajectories and state-action pairs, facilitating the creation of large-scale training datasets without the high cost of real-world data collection. Our evaluation demonstrates that the generated trajectories are physically plausible and executable, with a mean position error of 0.25 m (RMSE 0.28 m) and a mean orientation error of 0.19 rad (RMSE 0.24 rad). This approach enables improved policy learning and dataset scalability, leading to superior performance in downstream navigation tasks. Results in simulated environments highlight enhanced robustness, smoother trajectory planning, and adaptability to unseen conditions. An ANOVA revealed no statistically significant difference between performance in simulation and reality (F(1, 16) = 0.394, p = 0.541), with success rates of M = 0.628 (SD = 0.162) and M = 0.617 (SD = 0.177), respectively, indicating strong sim-to-real transfer. The generated datasets provide a valuable resource for future UAV research. This work introduces diffusion-based reasoning as a promising paradigm for unifying navigation, action generation, and data synthesis in aerial robotics.
Abstract (translated)
我们介绍了FlightDiffusion,这是一个基于扩散模型的框架,用于从第一人称视角(FPV)视频训练自主无人机。我们的模型可以从单帧生成逼真的视频序列,并添加相应的行为空间以在动态环境中实现推理驱动导航。除了直接的学习策略之外,FlightDiffusion 还利用其生成能力来合成多样化的 FPV 轨迹和状态-动作对,从而无需高昂的实际数据收集成本即可创建大规模训练数据集。 我们的评估表明,生成的轨迹具有物理上的合理性及可执行性,平均位置误差为 0.25 米(均方根误差 RMSE:0.28 米),平均方向误差为 0.19 弧度(RMSE:0.24 弧度)。这种方法能够提升策略学习和数据集规模的扩展能力,在下游导航任务中表现出色。模拟环境中的结果表明了增强的鲁棒性、更加平滑的轨迹规划以及对未见条件的适应能力。 方差分析揭示出,仿真与现实性能之间没有显著统计差异(F(1, 16) = 0.394,p = 0.541),成功率分别为 M = 0.628 (SD = 0.162) 和 M = 0.617 (SD = 0.177),表明了强大的仿真到现实的转换能力。生成的数据集为未来的无人机研究提供了宝贵的资源。 这项工作引入基于扩散模型的推理作为统一空中机器人导航、动作产生和数据合成的有前景范式。
URL
https://arxiv.org/abs/2509.14082