Abstract
Existing multi-object tracking algorithms typically fail to adequately address the issues in low-quality videos, resulting in a significant decline in tracking performance when image quality deteriorates in real-world scenarios. This performance degradation is primarily due to the algorithms' inability to effectively tackle the problems caused by information loss in low-quality images. To address the challenges of low-quality video scenarios, inspired by vision-language models, we propose a multi-object tracking framework guided by visual semantic distillation (VSD-MOT). Specifically, we introduce the CLIP Image Encoder to extract global visual semantic information from images to compensate for the loss of information in low-quality images. However, direct integration can substantially impact the efficiency of the multi-object tracking algorithm. Therefore, this paper proposes to extract visual semantic information from images through knowledge distillation. This method adopts a teacher-student learning framework, with the CLIP Image Encoder serving as the teacher model. To enable the student model to acquire the capability of extracting visual semantic information suitable for multi-object tracking tasks from the teacher model, we have designed the Dual-Constraint Semantic Distillation method (DCSD). Furthermore, to address the dynamic variation of frame quality in low-quality videos, we propose the Dynamic Semantic Weight Regulation (DSWR) module, which adaptively allocates fusion weights based on real-time frame quality assessment. Extensive experiments demonstrate the effectiveness and superiority of the proposed method in low-quality video scenarios in the real world. Meanwhile, our method can maintain good performance in conventional scenarios.
Abstract (translated)
现有多目标跟踪算法通常难以充分解决低质量视频中的问题，导致在真实场景中图像质量下降时跟踪性能显著降低。这种性能退化主要源于算法无法有效应对低质量图像中信息丢失所引发的问题。为应对低质量视频场景的挑战，本文受视觉语言模型启发，提出一种由视觉语义蒸馏引导的多目标跟踪框架（VSD-MOT）。具体而言，我们引入CLIP图像编码器从图像中提取全局视觉语义信息，以补偿低质量图像的信息损失。然而，直接整合会显著影响多目标跟踪算法的效率。因此，本文提出通过知识蒸馏从图像中提取视觉语义信息。该方法采用师生学习框架，以CLIP图像编码器作为教师模型。为使学生模型能够从教师模型中获取适用于多目标跟踪任务的视觉语义信息提取能力，我们设计了双约束语义蒸馏方法（DCSD）。此外，针对低质量视频中帧质量的动态变化，我们提出了动态语义权重调节模块（DSWR），该模块能基于实时帧质量评估自适应分配融合权重。大量实验表明，所提方法在真实世界低质量视频场景中具有有效性和优越性。同时，我们的方法在常规场景中也能保持良好性能。
URL
https://arxiv.org/abs/2603.20731
