Paper Reading AI Learner

DOD-SA: Infrared-Visible Decoupled Object Detection with Single-Modality Annotations

2025-08-14 08:33:51
Hang Jin, Chenqiang Gao, Junjie Guo, Fangcen Liu, Kanghui Tian, Qinyao Chang

Abstract

Infrared-visible object detection has shown great potential in real-world applications, enabling robust all-day perception by leveraging the complementary information of infrared and visible images. However, existing methods typically require dual-modality annotations to output detection results for both modalities during prediction, which incurs high annotation costs. To address this challenge, we propose a novel infrared-visible Decoupled Object Detection framework with Single-modality Annotations, called DOD-SA. The architecture of DOD-SA is built upon a Single- and Dual-Modality Collaborative Teacher-Student Network (CoSD-TSNet), which consists of a single-modality branch (SM-Branch) and a dual-modality decoupled branch (DMD-Branch). The teacher model generates pseudo-labels for the unlabeled modality, simultaneously supporting the training of the student model. The collaborative design enables cross-modality knowledge transfer from the labeled modality to the unlabeled modality, and facilitates effective SM-to-DMD branch supervision. To further improve the decoupling ability of the model and the pseudo-label quality, we introduce a Progressive and Self-Tuning Training Strategy (PaST) that trains the model in three stages: (1) pretraining SM-Branch, (2) guiding the learning of DMD-Branch by SM-Branch, and (3) refining DMD-Branch. In addition, we design a Pseudo Label Assigner (PLA) to align and pair labels across modalities, explicitly addressing modality misalignment during training. Extensive experiments on the DroneVehicle dataset demonstrate that our method outperforms state-of-the-art (SOTA).

Abstract (translated)

红外-可见光物体检测技术在实际应用中展现出巨大潜力,通过利用红外图像和可见光图像的互补信息,实现了全天候感知。然而,现有方法通常需要双模态注释来输出两种模式的检测结果,这导致了高昂的标注成本。为解决这一挑战,我们提出了一种新颖的单模态标签红外-可见光解耦物体检测框架(DOD-SA)。该框架基于单一和双重模态协作教师-学生网络(CoSD-TSNet),包含一个单模态分支(SM-Branch)和一个双模态解耦分支(DMD-Branch)。教师模型生成未标注模式的伪标签,同时支持学生模型的训练。这种协同设计使得知识可以从已标注的模态传递到未标注的模态,并促进有效的SM到DMD分支监督。 为了进一步提高模型的解耦能力和伪标签质量,我们引入了一种渐进式和自适应训练策略(PaST),该策略将模型训练分为三个阶段:(1) 预训练单模态分支;(2) 使用单模态分支指导双模态解耦分支的学习;以及 (3) 优化和完善双模态解耦分支。此外,我们设计了一种伪标签分配器(PLA),用于跨模式对齐和配对标签,明确解决了训练过程中模态不匹配的问题。 在DroneVehicle数据集上的广泛实验表明,我们的方法优于当前最先进的技术(SOTA)。

URL

https://arxiv.org/abs/2508.10445

PDF

https://arxiv.org/pdf/2508.10445.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot