Paper Reading AI Learner

Cross-modal Offset-guided Dynamic Alignment and Fusion for Weakly Aligned UAV Object Detection

2025-06-20 04:11:39
Liu Zongzhen, Luo Hui, Wang Zhixing, Wei Yuxing, Zuo Haorui, Zhang Jianlin

Abstract

Unmanned aerial vehicle (UAV) object detection plays a vital role in applications such as environmental monitoring and urban security. To improve robustness, recent studies have explored multimodal detection by fusing visible (RGB) and infrared (IR) imagery. However, due to UAV platform motion and asynchronous imaging, spatial misalignment frequently occurs between modalities, leading to weak alignment. This introduces two major challenges: semantic inconsistency at corresponding spatial locations and modality conflict during feature fusion. Existing methods often address these issues in isolation, limiting their effectiveness. In this paper, we propose Cross-modal Offset-guided Dynamic Alignment and Fusion (CoDAF), a unified framework that jointly tackles both challenges in weakly aligned UAV-based object detection. CoDAF comprises two novel modules: the Offset-guided Semantic Alignment (OSA), which estimates attention-based spatial offsets and uses deformable convolution guided by a shared semantic space to align features more precisely; and the Dynamic Attention-guided Fusion Module (DAFM), which adaptively balances modality contributions through gating and refines fused features via spatial-channel dual attention. By integrating alignment and fusion in a unified design, CoDAF enables robust UAV object detection. Experiments on standard benchmarks validate the effectiveness of our approach, with CoDAF achieving a mAP of 78.6% on the DroneVehicle dataset.

Abstract (translated)

无人飞行器(UAV)对象检测在环境监测和城市安全等领域扮演着至关重要的角色。为了增强其鲁棒性,最近的研究探索了通过融合可见光(RGB)和红外(IR)图像来实现多模态检测的方法。然而,由于无人机平台的运动以及异步成像的原因,不同模式之间经常会出现空间错位现象,导致对齐度较弱。这引出了两个主要挑战:对应空间位置上的语义不一致性以及在特征融合过程中产生的模态冲突。现有的方法通常单独解决这些问题,从而限制了它们的有效性。 本文提出了一种统一框架——跨模态偏移引导动态对齐与融合(CoDAF),旨在同时应对弱对齐的无人机对象检测中的两大挑战。CoDAF包含两个创新模块:偏移引导语义对齐(OSA)和动态注意力引导融合模块(DAFM)。其中,OSA通过基于共享语义空间估算注意力驱动的空间偏移,并使用可变形卷积来实现更精确的特征对齐;而DAFM则通过门控机制自适应地平衡模态贡献,并通过空间-通道双重注意机制优化融合后的特征。通过将对齐与融合整合到统一的设计中,CoDAF实现了鲁棒的无人机对象检测。 在标准基准测试上的实验验证了我们方法的有效性,其中CoDAF在DroneVehicle数据集上达到了78.6%的平均精度(mAP)。

URL

https://arxiv.org/abs/2506.16737

PDF

https://arxiv.org/pdf/2506.16737.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot