Abstract
Infrared-visible object detection has shown great potential in real-world applications, enabling robust all-day perception by leveraging the complementary information of infrared and visible images. However, existing methods typically require dual-modality annotations to output detection results for both modalities during prediction, which incurs high annotation costs. To address this challenge, we propose a novel infrared-visible Decoupled Object Detection framework with Single-modality Annotations, called DOD-SA. The architecture of DOD-SA is built upon a Single- and Dual-Modality Collaborative Teacher-Student Network (CoSD-TSNet), which consists of a single-modality branch (SM-Branch) and a dual-modality decoupled branch (DMD-Branch). The teacher model generates pseudo-labels for the unlabeled modality, simultaneously supporting the training of the student model. The collaborative design enables cross-modality knowledge transfer from the labeled modality to the unlabeled modality, and facilitates effective SM-to-DMD branch supervision. To further improve the decoupling ability of the model and the pseudo-label quality, we introduce a Progressive and Self-Tuning Training Strategy (PaST) that trains the model in three stages: (1) pretraining SM-Branch, (2) guiding the learning of DMD-Branch by SM-Branch, and (3) refining DMD-Branch. In addition, we design a Pseudo Label Assigner (PLA) to align and pair labels across modalities, explicitly addressing modality misalignment during training. Extensive experiments on the DroneVehicle dataset demonstrate that our method outperforms state-of-the-art (SOTA).
Abstract (translated)
红外-可见光物体检测技术在实际应用中展现出巨大潜力,通过利用红外图像和可见光图像的互补信息,实现了全天候感知。然而,现有方法通常需要双模态注释来输出两种模式的检测结果,这导致了高昂的标注成本。为解决这一挑战,我们提出了一种新颖的单模态标签红外-可见光解耦物体检测框架(DOD-SA)。该框架基于单一和双重模态协作教师-学生网络(CoSD-TSNet),包含一个单模态分支(SM-Branch)和一个双模态解耦分支(DMD-Branch)。教师模型生成未标注模式的伪标签,同时支持学生模型的训练。这种协同设计使得知识可以从已标注的模态传递到未标注的模态,并促进有效的SM到DMD分支监督。 为了进一步提高模型的解耦能力和伪标签质量,我们引入了一种渐进式和自适应训练策略(PaST),该策略将模型训练分为三个阶段:(1) 预训练单模态分支;(2) 使用单模态分支指导双模态解耦分支的学习;以及 (3) 优化和完善双模态解耦分支。此外,我们设计了一种伪标签分配器(PLA),用于跨模式对齐和配对标签,明确解决了训练过程中模态不匹配的问题。 在DroneVehicle数据集上的广泛实验表明,我们的方法优于当前最先进的技术(SOTA)。
URL
https://arxiv.org/abs/2508.10445