Paper Reading AI Learner

Label-Free Long-Horizon 3D UAV Trajectory Prediction via Motion-Aligned RGB and Event Cues

2025-07-04 07:58:04
Hanfang Liang, Shenghai Yuan, Fen Liu, Yizhuo Yang, Bing Wang, Zhuyu Huang, Chenyang Shi, Jing Jin

Abstract

The widespread use of consumer drones has introduced serious challenges for airspace security and public safety. Their high agility and unpredictable motion make drones difficult to track and intercept. While existing methods focus on detecting current positions, many counter-drone strategies rely on forecasting future trajectories and thus require more than reactive detection to be effective. To address this critical gap, we propose an unsupervised vision-based method for predicting the three-dimensional trajectories of drones. Our approach first uses an unsupervised technique to extract drone trajectories from raw LiDAR point clouds, then aligns these trajectories with camera images through motion consistency to generate reliable pseudo-labels. We then combine kinematic estimation with a visual Mamba neural network in a self-supervised manner to predict future drone trajectories. We evaluate our method on the challenging MMAUD dataset, including the V2 sequences that feature wide-field-of-view multimodal sensors and dynamic UAV motion in urban scenes. Extensive experiments show that our framework outperforms supervised image-only and audio-visual baselines in long-horizon trajectory prediction, reducing 5-second 3D error by around 40 percent without using any manual 3D labels. The proposed system offers a cost-effective, scalable alternative for real-time counter-drone deployment. All code will be released upon acceptance to support reproducible research in the robotics community.

Abstract (translated)

消费者无人机的广泛使用为航空安全和公共安全带来了严重的挑战。由于其高机动性和不可预测的动作,无人机难以被追踪和拦截。尽管现有方法主要集中在检测当前位置上,许多反无人机策略却依赖于对未来轨迹的预测,并因此需要超越被动反应式的检测技术才能有效实施。为了弥补这一关键缺口,我们提出了一种基于无监督视觉的方法来预测无人机的三维轨迹。我们的方法首先使用一种无监督的技术从原始LiDAR点云中提取无人机轨迹,然后通过运动一致性将这些轨迹与相机图像对齐以生成可靠的伪标签。接着,我们将动力学估计与视觉Mamba神经网络结合,在自我监督的方式下预测未来的无人机轨迹。 我们在具有挑战性的MMAUD数据集上评估了我们的方法,该数据集包括V2序列,这些序列包含广角多模态传感器和城市环境中动态的UAV运动。广泛的实验表明,我们的框架在长时域轨迹预测方面优于仅基于图像的监督式基础模型以及音频-视觉基础模型,在不使用任何手动3D标签的情况下将5秒内的三维误差减少了大约40%。 所提出的系统提供了一种成本效益高、可扩展的选择,适用于实时反无人机部署。所有代码将在接受后发布,以支持机器人领域的可重复研究。

URL

https://arxiv.org/abs/2507.03365

PDF

https://arxiv.org/pdf/2507.03365.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot