Paper Reading AI Learner

FlightDiffusion: Revolutionising Autonomous Drone Training with Diffusion Models Generating FPV Video

2025-09-17 15:28:09
Valerii Serpiva, Artem Lykov, Faryal Batool, Vladislav Kozlovskiy, Miguel Altamirano Cabrera, Dzmitry Tsetserukou

Abstract

We present FlightDiffusion, a diffusion-model-based framework for training autonomous drones from first-person view (FPV) video. Our model generates realistic video sequences from a single frame, enriched with corresponding action spaces to enable reasoning-driven navigation in dynamic environments. Beyond direct policy learning, FlightDiffusion leverages its generative capabilities to synthesize diverse FPV trajectories and state-action pairs, facilitating the creation of large-scale training datasets without the high cost of real-world data collection. Our evaluation demonstrates that the generated trajectories are physically plausible and executable, with a mean position error of 0.25 m (RMSE 0.28 m) and a mean orientation error of 0.19 rad (RMSE 0.24 rad). This approach enables improved policy learning and dataset scalability, leading to superior performance in downstream navigation tasks. Results in simulated environments highlight enhanced robustness, smoother trajectory planning, and adaptability to unseen conditions. An ANOVA revealed no statistically significant difference between performance in simulation and reality (F(1, 16) = 0.394, p = 0.541), with success rates of M = 0.628 (SD = 0.162) and M = 0.617 (SD = 0.177), respectively, indicating strong sim-to-real transfer. The generated datasets provide a valuable resource for future UAV research. This work introduces diffusion-based reasoning as a promising paradigm for unifying navigation, action generation, and data synthesis in aerial robotics.

Abstract (translated)

我们介绍了FlightDiffusion,这是一个基于扩散模型的框架,用于从第一人称视角(FPV)视频训练自主无人机。我们的模型可以从单帧生成逼真的视频序列,并添加相应的行为空间以在动态环境中实现推理驱动导航。除了直接的学习策略之外,FlightDiffusion 还利用其生成能力来合成多样化的 FPV 轨迹和状态-动作对,从而无需高昂的实际数据收集成本即可创建大规模训练数据集。 我们的评估表明,生成的轨迹具有物理上的合理性及可执行性,平均位置误差为 0.25 米(均方根误差 RMSE:0.28 米),平均方向误差为 0.19 弧度(RMSE:0.24 弧度)。这种方法能够提升策略学习和数据集规模的扩展能力,在下游导航任务中表现出色。模拟环境中的结果表明了增强的鲁棒性、更加平滑的轨迹规划以及对未见条件的适应能力。 方差分析揭示出,仿真与现实性能之间没有显著统计差异(F(1, 16) = 0.394,p = 0.541),成功率分别为 M = 0.628 (SD = 0.162) 和 M = 0.617 (SD = 0.177),表明了强大的仿真到现实的转换能力。生成的数据集为未来的无人机研究提供了宝贵的资源。 这项工作引入基于扩散模型的推理作为统一空中机器人导航、动作产生和数据合成的有前景范式。

URL

https://arxiv.org/abs/2509.14082

PDF

https://arxiv.org/pdf/2509.14082.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot