Object_Tracking
-
VSD-MOT: End-to-End Multi-Object Tracking in Low-Quality Video Scenes Guided by Visual Semantic Distillation
2026-03-21 09:33:41Jun DuAbstract
Existing multi-object tracking algorithms typically fail to adequately address the issues in low-quality videos, resulting in a significant decline in tracking performance when image quality deteriorates in real-world scenarios. This performance degradation is primarily due to the algorithms' inability to effectively tackle the problems caused by information loss in low-quality images. To address the challenges of low-quality video scenarios, inspired by vision-language models, we propose a multi-object tracking framework guided by visual semantic distillation (VSD-MOT). Specifically, we introduce the CLIP Image Encoder to extract global visual semantic information from images to compensate for the loss of information in low-quality images. However, direct integration can substantially impact the efficiency of the multi-object tracking algorithm. Therefore, this paper proposes to extract visual semantic information from images through knowledge distillation. This method adopts a teacher-student learning framework, with the CLIP Image Encoder serving as the teacher model. To enable the student model to acquire the capability of extracting visual semantic information suitable for multi-object tracking tasks from the teacher model, we have designed the Dual-Constraint Semantic Distillation method (DCSD). Furthermore, to address the dynamic variation of frame quality in low-quality videos, we propose the Dynamic Semantic Weight Regulation (DSWR) module, which adaptively allocates fusion weights based on real-time frame quality assessment. Extensive experiments demonstrate the effectiveness and superiority of the proposed method in low-quality video scenarios in the real world. Meanwhile, our method can maintain good performance in conventional scenarios.
Abstract (translated)
现有多目标跟踪算法通常难以充分解决低质量视频中的问题，导致在真实场景中图像质量下降时跟踪性能显著降低。这种性能退化主要源于算法无法有效应对低质量图像中信息丢失所引发的问题。为应对低质量视频场景的挑战，本文受视觉语言模型启发，提出一种由视觉语义蒸馏引导的多目标跟踪框架（VSD-MOT）。具体而言，我们引入CLIP图像编码器从图像中提取全局视觉语义信息，以补偿低质量图像的信息损失。然而，直接整合会显著影响多目标跟踪算法的效率。因此，本文提出通过知识蒸馏从图像中提取视觉语义信息。该方法采用师生学习框架，以CLIP图像编码器作为教师模型。为使学生模型能够从教师模型中获取适用于多目标跟踪任务的视觉语义信息提取能力，我们设计了双约束语义蒸馏方法（DCSD）。此外，针对低质量视频中帧质量的动态变化，我们提出了动态语义权重调节模块（DSWR），该模块能基于实时帧质量评估自适应分配融合权重。大量实验表明，所提方法在真实世界低质量视频场景中具有有效性和优越性。同时，我们的方法在常规场景中也能保持良好性能。
URL
https://arxiv.org/abs/2603.20731
PDF
-
Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision
2026-03-18 21:36:41Nikhil Gosala, B. Ravi Kiran, Senthil Yogamani, Abhinav ValadaAbstract
Monocular 3D object tracking aims to estimate temporally consistent 3D object poses across video frames, enabling autonomous agents to reason about scene dynamics. However, existing state-of-the-art approaches are fully supervised and rely on dense 3D annotations over long video sequences, which are expensive to obtain and difficult to scale. In this work, we address this fundamental limitation by proposing the first sparsely supervised framework for monocular 3D object tracking. Our approach decomposes the task into two sequential sub-problems: 2D query matching and 3D geometry estimation. Both components leverage the spatio-temporal consistency of image sequences to augment a sparse set of labeled samples and learn rich 2D and 3D representations of the scene. Leveraging these learned cues, our model automatically generates high-quality 3D pseudolabels across entire videos, effectively transforming sparse supervision into dense 3D track annotations. This enables existing fully-supervised trackers to effectively operate under extreme label sparsity. Extensive experiments on the KITTI and nuScenes datasets demonstrate that our method significantly improves tracking performance, achieving an improvement of up to 15.50 p.p. while using at most four ground truth annotations per track.
Abstract (translated)
URL
https://arxiv.org/abs/2603.18298
PDF
-
ModTrack: Sensor-Agnostic Multi-View Tracking via Identity-Informed PHD Filtering with Covariance Propagation
2026-03-16 18:43:10Aditya Iyer, Jack Roberts, Nora AyanianAbstract
Multi-View Multi-Object Tracking (MV-MOT) aims to localize and maintain consistent identities of objects observed by multiple sensors. This task is challenging, as viewpoint changes and occlusion disrupt identity consistency across views and time. Recent end-to-end approaches address this by jointly learning 2D Bird's Eye View (BEV) representations and identity associations, achieving high tracking accuracy. However, these methods offer no principled uncertainty accounting and remain tightly coupled to their training configuration, limiting generalization across sensor layouts, modalities, or datasets without retraining. We propose ModTrack, a modular MV-MOT system that matches end-to-end performance while providing cross-modal, sensor-agnostic generalization and traceable uncertainty. ModTrack confines learning methods to just the \textit{Detection and Feature Extraction} stage of the MV-MOT pipeline, performing all fusion, association, and tracking with closed-form analytical methods. Our design reduces each sensor's output to calibrated position-covariance pairs $(\mathbf{z}, R)$; cross-view clustering and precision-weighted fusion then yield unified estimates $(\hat{\mathbf{z}}, \hat{R})$ for identity assignment and temporal tracking. A feedback-coupled, identity-informed Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter with HMM motion modes uses these fused estimates to maintain identities under missed detections and heavy occlusion. ModTrack achieves 95.5 IDF1 and 91.4 MOTA on \textit{WildTrack}, surpassing all prior modular methods by over 21 points and rivaling the state-of-the-art end-to-end methods while providing deployment flexibility they cannot. Specifically, the same tracker core transfers unchanged to \textit{MultiviewX} and \textit{RadarScenes}, with only perception-module replacement required to extend to new domains and sensor modalities.
Abstract (translated)
URL
https://arxiv.org/abs/2603.15812
PDF
-
Video Detector: A Dual-Phase Vision-Based System for Real-Time Traffic Intersection Control and Intelligent Transportation Analysis
2026-03-16 06:02:43Mustafa Fatih \c{S}en, Hal\^uk G\"um\"u\c{s}kaya, \c{S}enol PazarAbstract
Urban traffic management increasingly requires intelligent sensing systems capable of adapting to dynamic traffic conditions without costly infrastructure modifications. Vision-based vehicle detection has therefore become a key technology for modern intelligent transportation systems. This study presents Video Detector (VD), a dual-phase vision-based traffic intersection management system designed as a flexible and cost-effective alternative to traditional inductive loop detectors. The framework integrates a real-time module (VD-RT) for intersection control with an offline analytical module (VD-Offline) for detailed traffic behavior analysis. Three system configurations were implemented using SSD Inception v2, Faster R-CNN Inception v2, and CenterNet ResNet-50 V1 FPN, trained on datasets totaling 108,000 annotated images across 6-10 vehicle classes. Experimental results show detection performance of up to 90% test accuracy and 29.5 mAP@0.5, while maintaining real-time throughput of 37 FPS on HD video streams. Field deployments conducted in collaboration with Istanbul IT and Smart City Technologies Inc. (ISBAK) demonstrate stable operation under diverse environmental conditions. The system supports virtual loop detection, vehicle counting, multi-object tracking, queue estimation, speed analysis, and multiclass vehicle classification, enabling comprehensive intersection monitoring without the need for embedded road sensors. The annotated dataset and training pipeline are publicly released to support reproducibility. These results indicate that the proposed framework provides a scalable and deployable vision-based solution for intelligent transportation systems and smart-city traffic management.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14861
PDF
-
QTrack: Query-Driven Reasoning for Multi-modal MOT
2026-03-14 05:11:36Tajamul Ashraf, Tavaheed Tariq, Sonia Yadav, Abrar Ul Riyaz, Wasif Tak, Moloud Abdar, Janibul BashirAbstract
Multi-object tracking (MOT) has traditionally focused on estimating trajectories of all objects in a video, without selectively reasoning about user-specified targets under semantic instructions. In this work, we introduce a query-driven tracking paradigm that formulates tracking as a spatiotemporal reasoning problem conditioned on natural language queries. Given a reference frame, a video sequence, and a textual query, the goal is to localize and track only the target(s) specified in the query while maintaining temporal coherence and identity consistency. To support this setting, we construct RMOT26, a large-scale benchmark with grounded queries and sequence-level splits to prevent identity leakage and enable robust evaluation of generalization. We further present QTrack, an end-to-end vision-language model that integrates multimodal reasoning with tracking-oriented localization. Additionally, we introduce a Temporal Perception-Aware Policy Optimization strategy with structured rewards to encourage motion-aware reasoning. Extensive experiments demonstrate the effectiveness of our approach for reasoning-centric, language-guided tracking. Code and data are available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2603.13759
PDF
-
TSDCRF: Balancing Privacy and Multi-Object Tracking via Time-Series CRF and Normalized Control Penalty
2026-03-14 00:27:12Bo Ma, Jinsong Wu, Weiqi YanAbstract
Multi-object tracking in video often requires appearance or location cues that can reveal sensitive identity information, while adding privacy-preserving noise typically disrupts cross-frame association and causes ID switches or target loss. We propose TSDCRF, a plug-in refinement framework that balances privacy and tracking by combining three components: (i) $(\varepsilon,\delta)$-differential privacy via calibrated Gaussian noise on sensitive regions under a configurable privacy budget; (ii) a Normalized Control Penalty (NCP) that down-weights unstable or conflicting class predictions before noise injection to stabilize association; and (iii) a time-series dynamic conditional random field (DCRF) that enforces temporal consistency and corrects trajectory deviation after noise, mitigating ID switches and resilience to trajectory hijacking. The pipeline is agnostic to the choice of detector and tracker (e.g., YOLOv4 and DeepSORT). We evaluate on MOT16, MOT17, Cityscapes, and KITTI. Results show that TSDCRF achieves a better privacy--utility trade-off than white noise and prior methods (NTPD, PPDTSA): lower KL-divergence shift, lower tracking RMSE, and improved robustness under trajectory hijacking while preserving privacy. Source code in this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2603.13667
PDF
-
FC-Track: Overlap-Aware Post-Association Correction for Online Multi-Object Tracking
2026-03-13 08:00:26Cheng Ju, Zejing Zhao, Akio NamikiAbstract
Reliable multi-object tracking (MOT) is essential for robotic systems operating in complex and dynamic environments. Despite recent advances in detection and association, online MOT methods remain vulnerable to identity switches caused by frequent occlusions and object overlap, where incorrect associations can propagate over time and degrade tracking reliability. We present a lightweight post-association correction framework (FC-Track) for online MOT that explicitly targets overlap-induced mismatches during inference. The proposed method suppresses unreliable appearance updates under high-overlap conditions using an Intersection over Area (IoA)-based filtering strategy, and locally corrects detection-to-tracklet mismatches through appearance similarity comparison within overlapped tracklet pairs. By preventing short-term mismatches from propagating, our framework effectively mitigates long-term identity switches without resorting to global optimization or re-identification. The framework operates online without global optimization or re-identification, making it suitable for real-time robotic applications. We achieve 81.73 MOTA, 82.81 IDF1, and 66.95 HOTA on the MOT17 test set with a running speed of 5.7 FPS, and 77.52 MOTA, 80.90 IDF1, and 65.67 HOTA on the MOT20 test set with a running speed of 0.6 FPS. Specifically, our framework FC-Track produces only 29.55% long-term identity switches, which is substantially lower than existing online trackers. Meanwhile, our framework maintains state-of-the-art performance on the MOT20 benchmark.
Abstract (translated)
URL
https://arxiv.org/abs/2603.12758
PDF
-
Distributed Kalman--Consensus Filtering with Adaptive Uncertainty Weighting for Multi-Object Tracking in Mobile Robot Networks
2026-03-11 21:41:10Niusha Khosravi, Rodrigo Ventura, Meysam BasiriAbstract
This paper presents an implementation and evaluation of a Distributed Kalman--Consensus Filter (DKCF) for Multi-Object Tracking (MOT) in mobile robot networks operating under partial observability and heterogeneous localization uncertainty. A key challenge in such systems is the fusion of information from agents with differing localization quality, where frame misalignment can lead to inconsistent estimates, track duplication, and ghost tracks. To address this issue, we build upon the MOTLEE framework and retain its frame-alignment methodology, which uses consistently tracked dynamic objects as transient landmarks to improve relative pose estimates between robots. On top of this framework, we propose an uncertainty-aware adaptive consensus weighting mechanism that dynamically adjusts the influence of neighbor information based on the covariance of the transmitted estimates, thereby reducing the impact of unreliable data during distributed fusion. Local tracking is performed using a Kalman Filter (KF) with a Constant Velocity Model (CVM) and Global Nearest Neighbor (GNN) data association. simulation results demonstrate that adaptive weighting effectively protects local estimates from inconsistent data, yielding a MOTA improvement of 0.09 for agents suffering from localization drift, although system performance remains constrained by communication latency.
Abstract (translated)
本文提出了一种分布式卡尔曼-共识滤波器（DKCF）的实现和评估，用于在部分可观察性和异构定位不确定性条件下运行的移动机器人网络中的多目标跟踪（MOT）。此类系统的一个关键挑战是从具有不同定位质量的代理中融合信息，这可能导致帧对齐错误、估计不一致、重复跟踪以及虚警。为了解决这一问题，我们基于MOTLEE框架构建，并保留其帧对齐方法，该方法利用持续跟踪的动力对象作为瞬时地标以改进机器人之间的相对姿态估计。在此基础上，我们提出了一种感知不确定性的自适应共识加权机制，通过根据传输估算的协方差动态调整邻居信息的影响来减少分布式融合期间不可靠数据的影响。本地跟踪使用恒定速度模型（CVM）和全局最近邻（GNN）数据关联的卡尔曼滤波器（KF）进行。 仿真结果表明，自适应加权有效保护了局部估计免受不一致数据的影响，在遭受定位漂移的代理中MOTA指标提高了0.09，尽管系统性能仍受到通信延迟的限制。
URL
https://arxiv.org/abs/2603.11328
PDF
-
Exploring Modality-Aware Fusion and Decoupled Temporal Propagation for Multi-Modal Object Tracking
2026-03-10 07:10:05Shilei Wang, Pujian Lai, Dong Gao, Jifeng Ning, Gong ChengAbstract
Most existing multimodal trackers adopt uniform fusion strategies, overlooking the inherent differences between modalities. Moreover, they propagate temporal information through mixed tokens, leading to entangled and less discriminative temporal representations. To address these limitations, we propose MDTrack, a novel framework for modality aware fusion and decoupled temporal propagation in multimodal object tracking. Specifically, for modality aware fusion, we allocate dedicated experts to each modality, including infrared, event, depth, and RGB, to process their respective representations. The gating mechanism within the Mixture of Experts dynamically selects the optimal experts based on the input features, enabling adaptive and modality specific fusion. For decoupled temporal propagation, we introduce two separate State Space Model structures to independently store and update the hidden states of the RGB and X modal streams, effectively capturing their distinct temporal information. To ensure synergy between the two temporal representations, we incorporate a set of cross attention modules between the input features of the two SSMs, facilitating implicit information exchange. The resulting temporally enriched features are then integrated into the backbone through another set of cross attention modules, enhancing MDTrack's ability to leverage temporal information. Extensive experiments demonstrate the effectiveness of our proposed method. Both MDTrack S and MDTrack U achieve state of the art performance across five multimodal tracking benchmarks.
Abstract (translated)
大多数现有的多模态跟踪器采用统一的融合策略，忽视了不同模式之间的固有差异。此外，它们通过混合令牌传播时间信息，导致纠缠且不那么具区分性的时序表示。为了解决这些限制，我们提出了MDTrack，这是一个新颖的框架，用于模态感知融合和解耦的时间传播，在多模态目标跟踪中具有创新性。 具体而言，对于模态感知融合，我们在每个模式（包括红外、事件、深度和RGB）上分配了专门的专家来处理它们各自的表示。Mixture of Experts中的门控机制根据输入特征动态选择最佳的专家，从而实现适应性和特定于模态的融合。 对于解耦的时间传播，我们引入了两个独立的状态空间模型结构（SSMs），分别存储和更新RGB和X模式流的隐藏状态，有效捕捉它们各自独特的时序信息。为了确保这两种时间表示之间的协同作用，我们在两个SSM输入特征之间集成了交叉注意力模块组，促进隐式的信息交换。 然后，通过另一组交叉注意力模块将这些经过时间增强的特征整合到骨干网络中，从而增强了MDTrack利用时间信息的能力。广泛的实验展示了我们所提出方法的有效性。无论是MDTrack S还是MDTrack U，在五个多模态跟踪基准测试上都取得了最先进的性能。
URL
https://arxiv.org/abs/2603.09287
PDF
-
Can Vision-Language Models Solve the Shell Game?
2026-03-09 14:33:25Tiedong Liu, Wee Sun LeeAbstract
Visual entity tracking is an innate cognitive ability in humans, yet it remains a critical bottleneck for Vision-Language Models (VLMs). This deficit is often obscured in existing video benchmarks by visual shortcuts. We introduce VET-Bench, a synthetic diagnostic testbed featuring visually identical objects that necessitate tracking exclusively through spatiotemporal continuity. Our experiments reveal that current state-of-the-art VLMs perform at or near chance level on VET-Bench, exposing a fundamental limitation: an over-reliance on static frame-level features and a failure to maintain entity representations over time. We provide a theoretical analysis drawing connections to the state-tracking problem, proving that fixed-depth transformer-based VLMs are fundamentally limited in tracking indistinguishable objects without intermediate supervision due to expressivity constraints. To address this, we propose Spatiotemporal Grounded Chain-of-Thought (SGCoT): generating object trajectories as explicit intermediate states. Leveraging Molmo2's object tracking ability, we elicit SGCoT reasoning by fine-tuning on synthesized text-only data for alignment. Our method achieves state-of-the-art accuracy exceeding 90% on VET-Bench, demonstrating that VLMs can reliably solve the video shell-game task end-to-end without external tools. Our code and data are available at this https URL .
Abstract (translated)
视觉实体跟踪是人类的一种内在认知能力，然而对于视觉-语言模型（VLMs）来说，这仍然是一个关键的技术瓶颈。现有视频基准测试中所存在的视觉捷径常常掩盖了这种不足。我们引入了一个名为VET-Bench的合成诊断平台，该平台通过时空连续性来追踪视觉上完全相同的对象，从而展示了这一能力的重要性。实验表明，目前最先进的VLMs在VET-Bench上的表现接近随机水平，这揭示了一种基本限制：过度依赖静态帧级别的特征，并且无法随着时间维持实体表示。 我们进行理论分析并将其与状态跟踪问题联系起来，证明了基于固定深度变换器的VLMs由于表达能力的局限性，在没有中间监督的情况下追踪不可区分的对象时存在根本性的限制。为解决这一难题，我们提出了时空接地链式思维（Spatiotemporal Grounded Chain-of-Thought, SGCoT）：通过生成物体轨迹作为显式的中间状态来解决问题。 利用Molmo2的物体跟踪能力，并在合成的文字数据上进行微调以实现对齐，从而激发SGCoT推理。我们的方法在VET-Bench上的准确率达到了90%以上，证明了VLMs可以在没有外部工具的情况下可靠地完成视频壳游戏任务。代码和数据可通过此链接获取。 该研究揭示了视觉-语言模型在处理时间连续性方面的能力局限，并提出了一种新颖的方法来解决这一问题，展示出潜在的改进路径和技术前景。
URL
https://arxiv.org/abs/2603.08436
PDF
-
Fusion-Poly: A Polyhedral Framework Based on Spatial-Temporal Fusion for 3D Multi-Object Tracking
2026-03-09 10:26:44Xian Wu, Yitao Wu, Xiaoyu Li, Zijia Li, Lijun Zhao, Lining SunAbstract
LiDAR-camera 3D multi-object tracking (MOT) combines rich visual semantics with accurate depth cues to improve trajectory consistency and tracking reliability. In practice, however, LiDAR and cameras operate at different sampling rates. To maintain temporal alignment, existing data pipelines usually synchronize heterogeneous sensor streams and annotate them at a reduced shared frequency, forcing most prior methods to perform spatial fusion only at synchronized timestamps through projection-based or learnable cross-sensor association. As a result, abundant asynchronous observations remain underexploited, despite their potential to support more frequent association and more robust trajectory estimation over short temporal intervals. To address this limitation, we propose Fusion-Poly, a spatial-temporal fusion framework for 3D MOT that integrates asynchronous LiDAR and camera data. Fusion-Poly associates trajectories with multi-modal observations at synchronized timestamps and with single-modal observations at asynchronous timestamps, enabling higher-frequency updates of motion and existence states. The framework contains three key components: a frequency-aware cascade matching module that adapts to synchronized and asynchronous frames according to available detection modalities; a frequency-aware trajectory estimation module that maintains trajectories through high-frequency motion prediction, differential updates, and confidence-calibrated lifecycle management; and a full-state observation alignment module that improves cross-modal consistency at synchronized timestamps by optimizing image-projection errors. On the nuScenes test set, Fusion-Poly achieves 76.5% AMOTA, establishing a new state of the art among tracking-by-detection 3D MOT methods. Extensive ablation studies further validate the effectiveness of each component. Code will be released.
Abstract (translated)
LiDAR-相机3D多目标跟踪（MOT）结合了丰富的视觉语义和准确的深度线索，以提高轨迹的一致性和跟踪的可靠性。然而，在实践中，由于LiDAR与摄像头的工作频率不同，现有数据管道通常会同步异构传感器流，并在较低的共享频率下进行标注，这迫使大多数先前的方法仅在同步时间戳通过基于投影或可学习跨传感器关联来进行空间融合。因此，尽管异步观测具有支持更频繁关联和短时区间内更强健轨迹估计的潜力，但大量未充分利用。 为了解决这一限制，我们提出了Fusion-Poly，这是一种用于3D MOT的空间-时间融合框架，能够整合异步LiDAR和相机数据。Fusion-Poly在同步时间戳处将轨迹与多模态观察相关联，并在异步时间戳处仅使用单模态观察进行关联，从而实现了运动和存在状态的更高频率更新。 该框架包含三个关键组件： 1. **频率感知级联匹配模块**：根据可用检测模式自适应地处理同步帧和异步帧。 2. **频率感知轨迹估计模块**：通过高频运动预测、差分更新以及基于置信度的生命周期管理来维持轨迹。 3. **全状态观测对齐模块**：在同步时间戳处优化图像投影误差，提高跨模式一致性。 在nuScenes测试集中，Fusion-Poly达到了76.5%的AMOTA（平均多目标跟踪准确率），确立了其在基于检测的3D MOT方法中的最新技术水平。广泛的消融研究进一步验证了每个组件的有效性。代码将公开发布。
URL
https://arxiv.org/abs/2603.08199
PDF
-
Training-free Temporal Object Tracking in Surgical Videos
2026-03-08 23:09:16Subhadeep Koley, Abdolrahim Kadkhodamohammadi, Santiago Barbarisi, Danail Stoyanov, Imanol LuengoAbstract
Purpose: In this paper, we present a novel approach for online object tracking in laparoscopic cholecystectomy (LC) surgical videos, targeting localisation and tracking of critical anatomical structures and instruments. Our method addresses the challenges of costly pixel-level annotations and label inconsistencies inherent in existing datasets. Methods: Leveraging the inherent object localisation capabilities of pre-trained text-to-image diffusion models, we extract representative features from surgical frames without any training or fine-tuning. Our tracking framework uses these features, along with cross-frame interactions via an affinity matrix inspired by query-key-value attention, to ensure temporal continuity in the tracking process. Results: Through a pilot study, we first demonstrate that diffusion features exhibit superior object localisation and consistent semantics across different decoder levels and temporal frames. Later, we perform extensive experiments to validate the effectiveness of our approach, showcasing its superiority over competitors for the task of temporal object tracking. Specifically, we achieve a per-pixel classification accuracy of 79.19%, mean Jaccard Score of 56.20%, and mean F-Score of 79.48% on the publicly available CholeSeg8K dataset. Conclusion: Our work not only introduces a novel application of text-to-image diffusion models but also contributes to advancing the field of surgical video analysis, offering a promising avenue for accurate and cost-effective temporal object tracking in minimally invasive surgery videos.
Abstract (translated)
目的：本文提出了一种新颖的方法，用于腹腔镜胆囊切除术（LC）手术视频中的在线物体跟踪，旨在定位和追踪关键解剖结构和仪器。我们的方法解决了现有数据集中昂贵的像素级标注以及标签不一致的问题。 方法：通过利用预训练文本到图像扩散模型固有的对象定位能力，我们在没有任何训练或微调的情况下从手术帧中提取代表性特征。我们的跟踪框架使用这些特征，并结合跨帧互动（基于查询-键-值注意机制启发的亲和矩阵），以确保跟踪过程中的时间连续性。 结果：通过一项试点研究，我们首先证明了扩散特征在不同解码器层级和时间帧之间表现出色的对象定位能力和一致的语义。随后，我们进行了广泛的实验来验证我们的方法的有效性，并展示了其在时间对象跟踪任务中优于竞争对手的优势。具体而言，在公开可用的CholeSeg8K数据集上，我们实现了79.19%的每像素分类准确率、56.20%的平均Jaccard分数以及79.48%的平均F-Score。 结论：我们的工作不仅引入了文本到图像扩散模型的新应用方式，而且还为手术视频分析领域做出了贡献。它为微创手术视频中精确且成本效益高的时间对象跟踪提供了有前景的道路。
URL
https://arxiv.org/abs/2603.07839
PDF
-
SiamGM: Siamese Geometry-Aware and Motion-Guided Network for Real-Time Satellite Video Object Tracking
2026-03-08 10:00:24Zixiao Wen, Zhen Yang, Jiawei Li, Xiantai Xiang, Guangyao Zhou, Yuxin Hu, Yuhan LiuAbstract
Single object tracking in satellite videos is inherently challenged by small target, blurred background, large aspect ratio changes, and frequent visual occlusions. These constraints often cause appearance-based trackers to accumulate errors and lose targets irreversibly. To systematically mitigate both spatial ambiguities and temporal information loss, we propose SiamGM, a novel geometry-aware and motion-guided Siamese network. From a spatial perspective, we introduce an Inter-Frame Graph Attention (IFGA) module, closely integrated with an Aspect Ratio-Constrained Label Assignment (LA) method, establishing fine-grained topological correspondences and explicitly preventing surrounding background noise. From a temporal perspective, we introduce the Motion Vector-Guided Online Tracking Optimization method. By adopting the Normalized Peak-to-Sidelobe Ratio (nPSR) as a dynamic confidence indicator, we propose an Online Motion Model Refinement (OMMR) strategy to utilize historical trajectory information. Evaluations on two challenging SatSOT and SV248S benchmarks confirm that SiamGM outperforms most state-of-the-art trackers in both precision and success metrics. Notably, the proposed components of SiamGM introduce virtually no computational overhead, enabling real-time tracking at 130 frames per second (FPS). Codes and tracking results are available at this https URL.
Abstract (translated)
卫星视频中的单个目标跟踪由于小尺寸目标、模糊背景、显著纵横比变化以及频繁的视觉遮挡，面临巨大挑战。这些因素常常导致基于外观的追踪器积累错误，并且不可逆转地失去目标。为了系统性地缓解空间模糊和时间信息损失的问题，我们提出了SiamGM（几何感知与运动引导的暹罗网络），这是一种新颖的方法。从空间角度来看，我们引入了帧间图注意力模块(IFGA)，并将其紧密集成到纵横比约束标签分配(LA)方法中，以建立细粒度拓扑对应关系，并明确防止周围背景噪声的影响。从时间角度来看，我们提出了运动向量引导的在线追踪优化方法。 通过采用归一化峰值侧瓣比（nPSR）作为动态置信度指标，我们提出了一种在线运动模型精炼(OMMR)策略来利用历史轨迹信息。在两个具有挑战性的SatSOT和SV248S基准测试上进行的评估表明，SiamGM在精度和成功指标方面均优于大多数最先进的追踪器。值得注意的是，SiamGM所提出的组件几乎不引入计算开销，从而实现实时跟踪速度高达130帧每秒(FPS)。 代码及跟踪结果可在以下链接获取：[此链接](https://this-url.com)（原文中提供的链接未显示，请根据实际情况访问相关资源）。
URL
https://arxiv.org/abs/2603.07564
PDF
-
NOVA: Next-step Open-Vocabulary Autoregression for 3D Multi-Object Tracking in Autonomous Driving
2026-03-06 13:12:28Kai Luo, Xu Wang, Rui Fan, Kailun YangAbstract
Generalizing across unknown targets is critical for open-world perception, yet existing 3D Multi-Object Tracking (3D MOT) pipelines remain limited by closed-set assumptions and ``semantic-blind'' heuristics. To address this, we propose Next-step Open-Vocabulary Autoregression (NOVA), an innovative paradigm that shifts 3D tracking from traditional fragmented distance-based matching toward generative spatio-temporal semantic modeling. NOVA reformulates 3D trajectories as structured spatio-temporal semantic sequences, enabling the simultaneous encoding of physical motion continuity and deep linguistic priors. By leveraging the autoregressive capabilities of Large Language Models (LLMs), we transform the tracking task into a principled process of next-step sequence completion. This mechanism allows the model to explicitly utilize the hierarchical structure of language space to resolve fine-grained semantic ambiguities and maintain identity consistency across complex long-range sequences through high-level commonsense reasoning. Extensive experiments on nuScenes, V2X-Seq-SPD, and KITTI demonstrate the superior performance of NOVA. Notably, on the nuScenes dataset, NOVA achieves an AMOTA of 22.41% for Novel categories, yielding a significant 20.21% absolute improvement over the baseline. These gains are realized through a compact 0.5B autoregressive model. Code will be available at this https URL.
Abstract (translated)
在开放世界的感知环境中，跨越未知目标进行泛化是至关重要的，然而现有的三维多对象跟踪（3D MOT）管道仍然受限于封闭集合假设和“无语义”的启发式方法。为了解决这个问题，我们提出了Next-step Open-Vocabulary Autoregression (NOVA)，这是一种创新的范例，将3D跟踪从传统的基于距离匹配的方法转变为生成性的时空语义建模。 NOVA重新定义了三维轨迹作为结构化的时空间语义序列，从而同时编码物理运动连续性和深层语言先验。通过利用大型语言模型（LLMs）的自回归能力，我们将追踪任务转化为一种原则上的下一步序列完成过程。这种方法使模型能够明确地使用语言空间中的层次结构来解决细粒度的语义模糊性，并通过高层次的常识推理在复杂的长距离序列中保持身份一致性。 在nuScenes、V2X-Seq-SPD和KITTI数据集上进行的大量实验表明，NOVA表现出色。特别值得注意的是，在nuScenes数据集中，对于新类别（Novel categories），NOVA实现了22.41%的AMOTA，比基线有了显著的20.21%绝对改进。这些改进是通过一个紧凑的0.5B自回归模型实现的。 代码将在这个链接提供：[此URL]
URL
https://arxiv.org/abs/2603.06254
PDF
-
Occlusion-Aware SORT: Observing Occlusion for Robust Multi-Object Tracking
2026-03-06 08:40:07Chunjiang Li, Jianbo Ma, Li Shen, Yanru Chen, Liangyin ChenAbstract
Multi-object tracking (MOT) involves analyzing object trajectories and counting the number of objects in video sequences. However, 2D MOT faces challenges due to positional cost confusion arising from partial occlusion. To address this issue, we present the novel Occlusion-Aware SORT (OA-SORT) framework, a plug-and-play and training-free framework that includes the Occlusion-Aware Module (OAM), the Occlusion-Aware Offset (OAO), and the Bias-Aware Momentum (BAM). Specifically, OAM analyzes the occlusion status of objects, where a Gaussian Map (GM) is introduced to reduce background influence. In contrast, OAO and BAM leverage the OAM-described occlusion status to mitigate cost confusion and suppress estimation instability. Comprehensive evaluations on the DanceTrack, SportsMOT, and MOT17 datasets demonstrate the importance of occlusion handling in MOT. On the DanceTrack test set, OA-SORT achieves 63.1% and 64.2% in HOTA and IDF1, respectively. Furthermore, integrating the Occlusion-Aware framework into the four additional trackers improves HOTA and IDF1 by an average of 2.08% and 3.05%, demonstrating the reusability of the occlusion awareness.
Abstract (translated)
多目标跟踪（MOT）涉及分析对象轨迹并统计视频序列中的对象数量。然而，二维 MOT 面临着由于部分遮挡导致的位置成本混淆的挑战。为了解决这一问题，我们提出了一个新的 Occlusion-Aware SORT (OA-SORT) 框架，这是一个即插即用且无需训练的框架，包括了 Occlusion-Aware Module（OAM）、Occlusion-Aware Offset（OAO）和 Bias-Aware Momentum（BAM）。具体来说，OAM 分析对象的遮挡状态，在这里引入了一个高斯图（GM），以减少背景的影响。相对地，OAO 和 BAM 利用 OAM 描述的遮挡状态来缓解成本混淆并抑制估计不稳定。 在 DanceTrack、SportsMOT 以及 MOT17 数据集上进行的全面评估显示了处理遮挡在 MOT 中的重要性。在 DanceTrack 测试集中，OA-SORT 在 HOTA（Hit Rate-Orientation Accuracy）和 IDF1（Inverse Detection Fraction Overlap @ 1）两个指标中分别取得了63.1% 和 64.2% 的成绩。此外，将 Occlusion-Aware 框架整合到四个额外的跟踪器中后，HOTA 和 IDF1 平均提高了 2.08% 和 3.05%，这证明了对遮挡感知重用性的认可。
URL
https://arxiv.org/abs/2603.06034
PDF
-
Breaking Smooth-Motion Assumptions: A UAV Benchmark for Multi-Object Tracking in Complex and Adverse Conditions
2026-03-06 07:09:16Jingtao Ye, Kexin Zhang, Xunchi Ma, Yuehan Li, Guangming Zhu, Peiyi Shen, Linhua Jiang, Xiangdong Zhang, Liang ZhangAbstract
The rapid movements and agile maneuvers of unmanned aerial vehicles (UAVs) induce significant observational challenges for multi-object tracking (MOT). However, existing UAV-perspective MOT benchmarks often lack these complexities, featuring predominantly predictable camera dynamics and linear motion patterns. To address this gap, we introduce DynUAV, a new benchmark for dynamic UAV-perspective MOT, characterized by intense ego-motion and the resulting complex apparent trajectories. The benchmark comprises 42 video sequences with over 1.7 million bounding box annotations, covering vehicles, pedestrians, and specialized industrial categories such as excavators, bulldozers and cranes. Compared to existing benchmarks, DynUAV introduces substantial challenges arising from ego-motion, including drastic scale changes and viewpoint changes, as well as motion blur. Comprehensive evaluations of state-of-the-art trackers on DynUAV reveal their limitations, particularly in managing the intertwined challenges of detection and association under such dynamic conditions, thereby establishing DynUAV as a rigorous benchmark. We anticipate that DynUAV will serve as a demanding testbed to spur progress in real-world UAV-perspective MOT, and we will make all resources available at link.
Abstract (translated)
无人驾驶飞行器（无人机）的快速移动和敏捷操作给多目标跟踪（MOT）带来了显著的观测挑战。然而，现有的以无人机视角为主的MOT基准测试往往缺乏这些复杂性，主要特征是可预测的摄像机动态和线性运动模式。为了填补这一空白，我们引入了DynUAV，这是一个新的动态无人机视角下的多目标跟踪（MOT）基准测试，它具有强烈的自我移动特性，并因此产生了复杂的视感轨迹。该基准测试包括42个视频序列，包含超过170万条边界框标注，涵盖了车辆、行人以及挖掘机、推土机和起重机等专门的工业类别。 与现有的基准测试相比，DynUAV引入了由自我的运动引起的显著挑战，包括剧烈的尺度变化和视角变化，以及运动模糊。对最先进的跟踪器在DynUAV上的全面评估揭示了它们在处理这种动态条件下的检测和关联交织挑战时的局限性，从而确立了DynUAV作为严格的基准测试的地位。 我们预计DynUAV将成为促进现实世界无人机视角MOT进步的严格测试平台，并将提供所有资源链接。
URL
https://arxiv.org/abs/2603.05970
PDF
-
Safe-SAGE: Social-Semantic Adaptive Guidance for Safe Engagement through Laplace-Modulated Poisson Safety Functions
2026-03-05 18:59:02Lizhi Yang, Ryan M. Bena, Meg Wilkinson, Gilbert Bahati, Andy Navarro Brenes, Ryan K. Cosner, Aaron D. AmesAbstract
Traditional safety-critical control methods, such as control barrier functions, suffer from semantic blindness, exhibiting the same behavior around obstacles regardless of contextual significance. This limitation leads to the uniform treatment of all obstacles, despite their differing semantic meanings. We present Safe-SAGE (Social-Semantic Adaptive Guidance for Safe Engagement), a unified framework that bridges the gap between high-level semantic understanding and low-level safety-critical control through a Poisson safety function (PSF) modulated using a Laplace guidance field. Our approach perceives the environment by fusing multi-sensor point clouds with vision-based instance segmentation and persistent object tracking to maintain up-to-date semantics beyond the camera's field of view. A multi-layer safety filter is then used to modulate system inputs to achieve safe navigation using this semantic understanding of the environment. This safety filter consists of both a model predictive control layer and a control barrier function layer. Both layers utilize the PSF and flux modulation of the guidance field to introduce varying levels of conservatism and multi-agent passing norms for different obstacles in the environment. Our framework enables legged robots to navigate semantically rich, dynamic environments with context-dependent safety margins while maintaining rigorous safety guarantees.
Abstract (translated)
传统的安全关键控制系统，例如控制屏障函数，在处理障碍物时存在语义盲点问题，无论环境上下文如何变化，它们对障碍物的反应都是相同的。这种限制导致所有障碍物被统一对待，忽视了它们在不同情境下的意义差异。为此，我们提出了Safe-SAGE（社会-语义自适应引导安全互动）框架，该框架通过使用泊松安全性函数（PSF）调节拉普拉斯指导场来弥合高层次语义理解和低层次安全关键控制之间的差距。 我们的方法通过融合多传感器点云和基于视觉的实例分割以及持续的对象跟踪技术来感知环境，保持超出摄像机视野范围内的最新环境语义信息。接着使用一个多层安全过滤器来调节系统的输入，以实现利用这种环境语义理解的安全导航。该安全过滤器包括模型预测控制层和控制屏障函数层。两层都利用PSF和指导场的通量调制，在处理不同障碍物时引入不同程度的保守性和多代理通过规范。 我们的框架使腿部机器人能够在具有上下文相关安全边距的语义丰富且动态变化的环境中导航，同时保持严格的安全保证。
URL
https://arxiv.org/abs/2603.05497
PDF
-
EdgeDAM: Real-time Object Tracking for Mobile Devices
2026-03-05 18:35:25Syed Muhammad Raza, Syed Murtaza Hussain Abidi, Khawar Islam, Muhammad Ibrahim, Ajmal Saeed MianAbstract
Single-object tracking (SOT) on edge devices is a critical computer vision task, requiring accurate and continuous target localization across video frames under occlusion, distractor interference, and fast motion. However, recent state-of-the-art distractor-aware memory mechanisms are largely built on segmentation-based trackers and rely on mask prediction and attention-driven memory updates, which introduce substantial computational overhead and limit real-time deployment on resource-constrained hardware; meanwhile, lightweight trackers sustain high throughput but are prone to drift when visually similar distractors appear. To address these challenges, we propose EdgeDAM, a lightweight detection-guided tracking framework that reformulates distractor-aware memory for bounding-box tracking under strict edge constraints. EdgeDAM introduces two key strategies: (1) Dual-Buffer Distractor-Aware Memory (DAM), which integrates a Recent-Aware Memory to preserve temporally consistent target hypotheses and a Distractor-Resolving Memory to explicitly store hard negative candidates and penalize their re-selection during recovery; and (2) Confidence-Driven Switching with Held-Box Stabilization, where tracker reliability and temporal consistency criteria adaptively activate detection and memory-guided re-identification during occlusion, while a held-box mechanism temporarily freezes and expands the estimate to suppress distractor contamination. Extensive experiments on five benchmarks, including the distractor-focused DiDi dataset, demonstrate improved robustness under occlusion and fast motion while maintaining real-time performance on mobile devices, achieving 88.2% accuracy on DiDi and 25 FPS on an iPhone 15. Code will be released.
Abstract (translated)
边缘设备上的单目标跟踪（SOT）是一项关键的计算机视觉任务，要求在遮挡、干扰物和快速运动的情况下，在视频帧中进行准确且连续的目标定位。然而，最近最先进的干扰感知内存机制大多基于分割追踪器，并依赖于掩码预测和注意力驱动的记忆更新，这带来了大量的计算开销，限制了其在资源受限硬件上的实时部署；同时，轻量级的跟踪器虽然能够保持高吞吐量，但在视觉相似的干扰物出现时容易发生漂移。为了解决这些挑战，我们提出了EdgeDAM，这是一种轻量级的检测引导追踪框架，它将干扰感知内存重新构架以适应边界框追踪，并在严格的边缘约束条件下进行设计。EdgeDAM引入了两种关键策略：（1）双缓冲器干扰感知内存(Dual-Buffer Distractor-Aware Memory, DAM)，该策略集成了一个最近感知记忆模块来保留时间上一致的目标假设，并且使用一个干扰物解析内存模块明确存储难例负样本并在恢复期间惩罚其重新选择；（2）信心驱动切换和保框稳定化机制，这种机制根据追踪器可靠性和时间一致性标准自适应地激活检测和记忆引导的重识别，在遮挡时同时引入了一个保框机制暂时冻结并扩展估计以抑制干扰物污染。在包括专注于干扰物的DiDi数据集在内的五个基准测试上进行了广泛的实验，结果显示EdgeDAM在保持移动设备实时性能的同时，提高了遮挡和快速运动下的鲁棒性，并且在DiDi数据集中达到了88.2%的准确率，在iPhone 15上的帧率为25FPS。代码将在稍后发布。
URL
https://arxiv.org/abs/2603.05463
PDF
-
Video-based Locomotion Analysis for Fish Health Monitoring
2026-03-05 17:32:46Timon Palm, Clemens Seibold, Anna Hilsmann, Peter EisertAbstract
Monitoring the health conditions of fish is essential, as it enables the early detection of disease, safeguards animal welfare, and contributes to sustainable aquaculture practices. Physiological and pathological conditions of cultivated fish can be inferred by analyzing locomotion activities. In this paper, we present a system that estimates the locomotion activities from videos using multi object tracking. The core of our approach is a YOLOv11 detector embedded in a tracking-by-detection framework. We investigate various configurations of the YOLOv11-architecture as well as extensions that incorporate multiple frames to improve detection accuracy. Our system is evaluated on a manually annotated dataset of Sulawesi ricefish recorded in a home-aquarium-like setup, demonstrating its ability to reliably measure swimming direction and speed for fish health monitoring. The dataset will be made publicly available upon publication.
Abstract (translated)
监测鱼类的健康状况至关重要，因为它能够实现疾病的早期检测、保障动物福利，并促进可持续的水产养殖实践。通过分析鱼的行为活动可以推断出养殖鱼类的生理和病理状态。本文介绍了一种利用多目标跟踪技术从视频中估算行为活动的系统。我们的方法核心是一个嵌入到基于检测的追踪框架中的YOLOv11探测器。我们研究了不同配置的YOLOv11架构，并且通过整合多个帧来提高检测精度，探讨了其扩展性。 该系统在手动标注的数据集上进行了评估，数据集中包含在类似家庭水族箱环境中录制的苏拉威西稻鱼的行为记录。结果显示，本系统能够可靠地测量鱼类游泳的方向和速度，从而用于监测鱼类健康状况。在论文发表后，此数据集将公开发布供研究使用。
URL
https://arxiv.org/abs/2603.05407
PDF
-
ORMOT: A Dataset and Framework for Omnidirectional Referring Multi-Object Tracking
2026-03-05 17:15:01Sijia Chen, Zihan Zhou, Yanqiu Yu, En Yu, Wenbing TaoAbstract
Multi-Object Tracking (MOT) is a fundamental task in computer vision, aiming to track targets across video frames. Existing MOT methods perform well in general visual scenes, but face significant challenges and limitations when extended to visual-language settings. To bridge this gap, the task of Referring Multi-Object Tracking (RMOT) has recently been proposed, which aims to track objects that correspond to language descriptions. However, current RMOT methods are primarily developed on datasets captured by conventional cameras, which suffer from limited field of view. This constraint often causes targets to move out of the frame, leading to fragmented tracking and loss of contextual information. In this work, we propose a novel task, called Omnidirectional Referring Multi-Object Tracking (ORMOT), which extends RMOT to omnidirectional imagery, aiming to overcome the field-of-view (FoV) limitation of conventional datasets and improve the model's ability to understand long-horizon language descriptions. To advance the ORMOT task, we construct ORSet, an Omnidirectional Referring Multi-Object Tracking dataset, which contains 27 diverse omnidirectional scenes, 848 language descriptions, and 3,401 annotated objects, providing rich visual, temporal, and language information. Furthermore, we propose ORTrack, a Large Vision-Language Model (LVLM)-driven framework tailored for Omnidirectional Referring Multi-Object Tracking. Extensive experiments on the ORSet dataset demonstrate the effectiveness of our ORTrack framework. The dataset and code will be open-sourced at this https URL.
Abstract (translated)
多目标跟踪（MOT）是计算机视觉中的一个基本任务，旨在追踪视频帧中目标的位置。现有的MOT方法在一般的视觉场景下表现良好，但在扩展到涉及视觉和语言的设置时遇到了显著的挑战和限制。为了解决这一差距，最近提出了引用多对象跟踪（RMOT）的任务，其目的是根据语言描述来追踪物体。然而，目前的RMOT方法主要是在传统摄像机捕获的数据集上开发的，这些数据集视野有限。这种局限性常常导致目标移出画面范围，造成碎片化的追踪和上下文信息丢失。 为克服这一问题，在这项工作中，我们提出了一项新的任务——全向引用多对象跟踪（ORMOT），该任务扩展了RMOT至全向图像领域，旨在超越传统数据集的视野限制，并提升模型理解长跨度语言描述的能力。为了推动ORMOT的研究进展，我们构建了ORSet这一全向引用多目标追踪数据集，包含了27种多样化的全向场景、848条语言描述和3,401个标注对象，提供了丰富的视觉、时间以及语言信息。 此外，我们提出了ORTrack框架，这是一个针对全向引用多对象跟踪任务量身定制的大规模视觉-语言模型（LVLM）驱动的解决方案。在ORSet数据集上的广泛实验验证了我们的ORTrack框架的有效性。该数据集和代码将通过以下链接开放源码：[此URL]。 请注意，原文中提到的具体链接（"this https URL"）应替换为实际可访问的数据集和代码发布页面地址。
URL
https://arxiv.org/abs/2603.05384
PDF
