Object_Tracking
-
Architecture and evaluation protocol for transformer-based visual object tracking in UAV applications
2026-03-04 10:12:12Augustin Borne (ISL, Hochschule Karlsruhe -- Technik und Wirtschaft Karlsruhe University of Applied Sciences, IRIMAS), Pierre Notin (ISL), Christophe Hennequin (ISL), Sebastien Changey (ISL), Stephane Bazeille (IRIMAS), Christophe Cudel (IRIMAS), Franz QuintAbstract
Object tracking from Unmanned Aerial Vehicles (UAVs) is challenged by platform dynamics, camera motion, and limited onboard resources. Existing visual trackers either lack robustness in complex scenarios or are too computationally demanding for real-time embedded use. We propose an Modular Asynchronous Tracking Architecture (MATA) that combines a transformer-based tracker with an Extended Kalman Filter, integrating ego-motion compensation from sparse optical flow and an object trajectory model. We further introduce a hardware-independent, embedded oriented evaluation protocol and a new metric called Normalized time to Failure (NT2F) to quantify how long a tracker can sustain a tracking sequence without external help. Experiments on UAV benchmarks, including an augmented UAV123 dataset with synthetic occlusions, show consistent improvements in Success and NT2F metrics across multiple tracking processing frequency. A ROS 2 implementation on a Nvidia Jetson AGX Orin confirms that the evaluation protocol more closely matches real-time performance on embedded systems.
Abstract (translated)
来自无人飞行器（UAV）的对象跟踪面临平台动态、相机运动和有限的机载资源等挑战。现有的视觉追踪方法要么在复杂场景中不够鲁棒，要么对实时嵌入式应用来说计算需求过高。我们提出了一种模块化异步跟踪架构（MATA），该架构结合了基于变压器的追踪器和扩展卡尔曼滤波器，并集成了稀疏光流带来的自我运动补偿以及对象轨迹模型。此外，我们还引入了一种硬件无关、面向嵌入式设备的评估协议，以及一种新的度量标准——归一化失效时间（NT2F），用以量化追踪器在没有外部帮助的情况下能够维持跟踪序列的时间长度。实验结果表明，在包括增强版UAV123数据集（该数据集中合成了一些遮挡情况）在内的多个无人机基准测试中，MATA方法在成功率和NT2F度量标准上都有显著的改进，并且这些改进在不同的处理频率下是一致的。基于Nvidia Jetson AGX Orin设备上的ROS 2实现表明，这种评估协议更贴近嵌入式系统中的实时性能。 这段文字描述了一种新的对象跟踪架构以及相关的实验结果和评价方法，适用于无人飞行器（UAV）等场景下的视觉追踪任务。
URL
https://arxiv.org/abs/2603.03904
PDF
-
UETrack: A Unified and Efficient Framework for Single Object Tracking
2026-03-02 03:32:30Ben Kang, Jie Zhao, Xin Chen, Wanting Geng, Bin Zhang, Lu Zhang, Dong Wang, Huchuan LuAbstract
With growing real-world demands, efficient tracking has received increasing attention. However, most existing methods are limited to RGB inputs and struggle in multi-modal scenarios. Moreover, current multi-modal tracking approaches typically use complex designs, making them too heavy and slow for resource-constrained deployment. To tackle these limitations, we propose UETrack, an efficient framework for single object tracking. UETrack demonstrates high practicality and versatility, efficiently handling multiple modalities including RGB, Depth, Thermal, Event, and Language, and addresses the gap in efficient multi-modal tracking. It introduces two key components: a Token-Pooling-based Mixture-of-Experts mechanism that enhances modeling capacity through feature aggregation and expert specialization, and a Target-aware Adaptive Distillation strategy that selectively performs distillation based on sample characteristics, reducing redundant supervision and improving performance. Extensive experiments on 12 benchmarks across 3 hardware platforms show that UETrack achieves a superior speed-accuracy trade-off compared to previous methods. For instance, UETrack-B achieves 69.2% AUC on LaSOT and runs at 163/56/60 FPS on GPU/CPU/AGX, demonstrating strong practicality and versatility. Code is available at this https URL.
Abstract (translated)
随着现实世界需求的增长，高效的追踪技术受到了越来越多的关注。然而，大多数现有的方法仅限于RGB输入，并且在多模态场景中表现不佳。此外，当前的多模态跟踪方法通常采用复杂的架构设计，这使得它们过于庞大和缓慢，难以应用于资源受限的环境中。为了解决这些问题，我们提出了UETrack，这是一个用于单对象追踪的有效框架。UETrack展示了极高的实用性和灵活性，能够高效地处理包括RGB、深度、热感、事件及语言在内的多种模态，并解决了有效多模态跟踪领域的空白。 UETrack引入了两个关键组件：基于Token-Pooling的专家混合机制，通过特征聚合和专家专业化来增强建模能力；以及目标感知自适应蒸馏策略，根据样本特性进行选择性蒸馏以减少冗余监督并提高性能。在三个硬件平台上的12个基准测试中进行了广泛的实验表明，UETrack相较于先前的方法实现了更优的速度-精度权衡。例如，在LaSOT数据集上，UETrack-B达到了69.2%的AUC，并且分别在GPU、CPU和AGX平台上以每秒163帧、56帧和60帧的速度运行，展示了其强大的实用性和灵活性。代码可在提供的链接处获得。 该研究通过提供一个轻量级但高效的解决方案来应对现有技术的问题，这使得UETrack特别适合于资源受限的部署场景，并且在多模态追踪任务上表现出色。
URL
https://arxiv.org/abs/2603.01412
PDF
-
TP-Spikformer: Token Pruned Spiking Transformer
2026-02-28 07:53:26Wenjie Wei, Xiaolong Zhou, Malu Zhang, Ammar Belatreche, Qian Sun, Yimeng Shan, Dehao Zhang, Zijian Zhou, Zeyu Ma, Yang Yang, Haizhou LiAbstract
Spiking neural networks (SNNs) offer an energy-efficient alternative to traditional neural networks due to their event-driven computing paradigm. However, recent advancements in spiking transformers have focused on improving accuracy with large-scale architectures, which require significant computational resources and limit deployment on resource-constrained devices. In this paper, we propose a simple yet effective token pruning method for spiking transformers, termed TP-Spikformer, that reduces storage and computational overhead while maintaining competitive performance. Specifically, we first introduce a heuristic spatiotemporal information-retaining criterion that comprehensively evaluates tokens' importance, assigning higher scores to informative tokens for retention and lower scores to uninformative ones for pruning. Based on this criterion, we propose an information-retaining token pruning framework that employs a block-level early stopping strategy for uninformative tokens, instead of removing them outright. This also helps preserve more information during token pruning. We demonstrate the effectiveness, efficiency and scalability of TP-Spikformer through extensive experiments across diverse architectures, including Spikformer, QKFormer and Spike-driven Transformer V1 and V3, and a range of tasks such as image classification, object detection, semantic segmentation and event-based object tracking. Particularly, TP-Spikformer performs well in a training-free manner. These results reveal its potential as an efficient and practical solution for deploying SNNs in real-world applications with limited computational resources.
Abstract (translated)
脉冲神经网络（SNN）由于其事件驱动的计算范式，为传统的神经网络提供了一种节能的替代方案。然而，最近在脉冲变压器方面的进展主要集中在通过大规模架构来提高准确性上，这需要大量的计算资源，并限制了这些模型在资源受限设备上的部署。在这篇论文中，我们提出了一种简单而有效的脉冲变压器标记剪枝方法，名为TP-Spikformer，该方法减少了存储和计算开销，同时保持了竞争力的性能。 具体来说，首先我们引入了一个基于时空信息保留的标准来评估标记的重要性，给重要的标记分配较高的分数以保留，对不太重要的标记则分配较低的分数进行裁减。在此标准基础上，我们提出了一种信息保留标记剪枝框架，采用块级早期停止策略，而不是直接去除那些不太重要的标记，这也有助于在标记裁剪过程中保存更多信息。 通过广泛的实验验证了TP-Spikformer的有效性、效率和可扩展性，这些实验涵盖了多种架构（如Spikformer、QKFormer以及Spike驱动的Transformer V1 和V3），以及包括图像分类、目标检测、语义分割及基于事件的目标跟踪等多种任务。特别值得注意的是，TP-Spikformer在无需训练的情况下也能表现出色。 这些结果揭示了其作为部署资源受限的真实世界应用中的SNN的一个高效且实用解决方案的巨大潜力。
URL
https://arxiv.org/abs/2603.00527
PDF
-
FocusTrack: One-Stage Focus-and-Suppress Framework for 3D Point Cloud Object Tracking
2026-02-27 16:09:05Sifan Zhou, Jiahao Nie, Ziyu Zhao, Yichao Cao, Xiaobo LuAbstract
In 3D point cloud object tracking, the motion-centric methods have emerged as a promising avenue due to its superior performance in modeling inter-frame motion. However, existing two-stage motion-based approaches suffer from fundamental limitations: (1) error accumulation due to decoupled optimization caused by explicit foreground segmentation prior to motion estimation, and (2) computational bottlenecks from sequential processing. To address these challenges, we propose FocusTrack, a novel one-stage paradigms tracking framework that unifies motion-semantics co-modeling through two core innovations: Inter-frame Motion Modeling (IMM) and Focus-and-Suppress Attention. The IMM module employs a temp-oral-difference siamese encoder to capture global motion patterns between adjacent frames. The Focus-and-Suppress attention that enhance the foreground semantics via motion-salient feature gating and suppress the background noise based on the temporal-aware motion context from IMM without explicit segmentation. Based on above two designs, FocusTrack enables end-to-end training with compact one-stage pipeline. Extensive experiments on prominent 3D tracking benchmarks, such as KITTI, nuScenes, and Waymo, demonstrate that the FocusTrack achieves new SOTA performance while running at a high speed with 105 FPS.
Abstract (translated)
在三维点云物体跟踪领域，以运动为中心的方法由于其在建模帧间运动方面的优越性能而成为一条有前途的途径。然而，现有的两阶段基于运动的方法存在根本性的局限性：(1) 因为在运动估计之前进行显式的前景分割而导致解耦优化所造成的误差积累；(2) 由顺序处理导致的计算瓶颈。为了应对这些挑战，我们提出了FocusTrack——一种新颖的一阶段跟踪框架，通过两个核心创新将运动-语义共模建整合在一起：帧间运动建模（IMM）和关注与抑制注意力机制。 IMM模块采用时序差异暹罗编码器来捕捉相邻帧之间的全局运动模式。关注与抑制注意机制则通过对来自IMM的时序感知运动上下文进行基于运动显著性特征门控的方法，增强前景语义并压制背景噪声，而无需显式分割。 根据以上两种设计，FocusTrack能够实现端到端训练，并且具有紧凑的一阶段流水线。在诸如KITTI、nuScenes和Waymo等突出的3D跟踪基准测试中进行广泛的实验表明，FocusTrack能够在以每秒105帧的速度运行的同时达到新的SOTA（State-of-the-Art）性能水平。
URL
https://arxiv.org/abs/2602.24133
PDF
-
SpikeTrack: A Spike-driven Framework for Efficient Visual Tracking
2026-02-27 12:16:51Qiuyang Zhang, Jiujun Cheng, Qichao Mao, Cong Liu, Yu Fang, Yuhong Li, Mengying Ge, Shangce GaoAbstract
Spiking Neural Networks (SNNs) promise energy-efficient vision, but applying them to RGB visual tracking remains difficult: Existing SNN tracking frameworks either do not fully align with spike-driven computation or do not fully leverage neurons' spatiotemporal dynamics, leading to a trade-off between efficiency and accuracy. To address this, we introduce SpikeTrack, a spike-driven framework for energy-efficient RGB object tracking. SpikeTrack employs a novel asymmetric design that uses asymmetric timestep expansion and unidirectional information flow, harnessing spatiotemporal dynamics while cutting computation. To ensure effective unidirectional information transfer between branches, we design a memory-retrieval module inspired by neural inference mechanisms. This module recurrently queries a compact memory initialized by the template to retrieve target cues and sharpen target perception over time. Extensive experiments demonstrate that SpikeTrack achieves the state-of-the-art among SNN-based trackers and remains competitive with advanced ANN trackers. Notably, it surpasses TransT on LaSOT dataset while consuming only 1/26 of its energy. To our knowledge, SpikeTrack is the first spike-driven framework to make RGB tracking both accurate and energy efficient. The code and models are available at this https URL.
Abstract (translated)
脉冲神经网络（SNNs）承诺了节能的视觉处理，但将其应用于RGB视频目标跟踪仍然面临挑战：现有的基于SNN的目标追踪框架要么没有完全符合脉冲驱动计算的要求，要么没有充分利用神经元的空间-时间动态特性，导致效率与准确性之间的权衡。为了解决这个问题，我们提出了SpikeTrack，这是一个用于节能的RGB对象跟踪的脉冲驱动框架。 SpikeTrack采用了新颖的不对称设计，利用了不对称的时间步扩展和单向信息流，这样既发挥了空间-时间动态特性又能减少计算量。为了确保分支间有效的一方向信息传递，我们设计了一个受神经推理机制启发的记忆检索模块。此模块通过反复查询模板初始化的小型记忆来获取目标线索，并随着时间推移增强对目标的感知。 广泛的实验表明，SpikeTrack在基于SNN的目标追踪器中达到了最先进的性能，并且与高级人工神经网络（ANN）跟踪器保持了竞争力。尤其值得注意的是，在LaSOT数据集上，它超过了TransT，同时仅消耗后者1/26的能量。 据我们所知，SpikeTrack是第一个能够实现RGB跟踪既准确又节能的脉冲驱动框架。该代码和模型可在此链接获取（在原文中给出的具体链接需要用户自己访问查看）。
URL
https://arxiv.org/abs/2602.23963
PDF
-
UTPTrack: Towards Simple and Unified Token Pruning for Visual Tracking
2026-02-27 06:58:09Hao Wu, Xudong Wang, Jialiang Zhang, Junlong Tong, Xinghao Chen, Junyan Lin, Yunpu Ma, Xiaoyu ShenAbstract
One-stream Transformer-based trackers achieve advanced performance in visual object tracking but suffer from significant computational overhead that hinders real-time deployment. While token pruning offers a path to efficiency, existing methods are fragmented. They typically prune the search region, dynamic template, and static template in isolation, overlooking critical inter-component dependencies, which yields suboptimal pruning and degraded accuracy. To address this, we introduce UTPTrack, a simple and Unified Token Pruning framework that, for the first time, jointly compresses all three components. UTPTrack employs an attention-guided, token type-aware strategy to holistically model redundancy, a design that seamlessly supports unified tracking across multimodal and language-guided tasks within a single model. Extensive evaluations on 10 benchmarks demonstrate that UTPTrack achieves a new state-of-the-art in the accuracy-efficiency trade-off for pruning-based trackers, pruning 65.4% of vision tokens in RGB-based tracking and 67.5% in unified tracking while preserving 99.7% and 100.5% of baseline performance, respectively. This strong performance across both RGB and multimodal scenarios underlines its potential as a robust foundation for future research in efficient visual tracking. Code will be released at this https URL.
Abstract (translated)
基于单流Transformer的跟踪器在视觉对象跟踪中表现出色，但计算开销大，阻碍了其实时部署。虽然令牌剪枝提供了一种提高效率的方法，但现有方法是孤立地对搜索区域、动态模板和静态模板进行剪枝，忽视了各组件间的相互依赖关系，导致次优的剪枝效果及准确率下降。为解决此问题，我们提出了UTPTrack——一个简单且统一化的令牌剪枝框架，首次实现了同时压缩所有三个组成部分的目标。UTPTrack采用注意力引导、令牌类型感知策略来整体建模冗余，设计上可无缝支持单一模型内跨多模态和语言指导任务的统一跟踪。在10个基准测试上的广泛评估表明，UTPTrack在基于剪枝方法的精度效率权衡中达到了新的最先进水平，在基于RGB的跟踪中剪除了65.4%的视觉令牌，并在统一路线中剪除67.5%，同时分别保留了基线性能的99.7%和100.5%。无论是在RGB还是多模态场景下的强大表现，均突显出UTPTrack作为未来高效视觉跟踪研究稳健基础的巨大潜力。代码将在此网址发布。
URL
https://arxiv.org/abs/2602.23734
PDF
-
Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators
2026-02-26 18:40:02Yuhao Liu, Salim Ullah, Akash KumarAbstract
Neural network accelerators have been widely applied to edge devices for complex tasks like object tracking, image recognition, etc. Previous works have explored the quantization technologies in related lightweight accelerator designs to reduce hardware resource consumption. However, low precision leads to high accuracy loss in inference. Therefore, mixed-precision quantization becomes an alternative solution by applying different precision in different layers to trade off resource consumption and accuracy. Because regular designs for multiplication on hardware cannot support the precision reconfiguration for a multi-precision Quantized Neural Network (QNN) model in runtime, we propose a runtime reconfigurable multi-precision multi-channel bitwise systolic array design for QNN accelerators. We have implemented and evaluated our work on the Ultra96 FPGA platform. Results show that our work can achieve 1.3185 to 3.5671 times speedup in inferring mixed-precision models and has less critical path delay, supporting a higher clock frequency (250MHz).
Abstract (translated)
神经网络加速器已被广泛应用于边缘设备，用于执行如目标跟踪、图像识别等复杂任务。此前的研究工作已经探索了在相关轻量级加速器设计中应用量化技术以减少硬件资源消耗的方法。然而，低精度会导致推理时的高准确性损失。因此，混合精度量化成为了一种替代方案，通过在不同的层采用不同的精度来权衡资源消耗和准确性的取舍。由于硬件上常规的设计无法支持多精度量化神经网络（QNN）模型在运行时重新配置精度，我们提出了一种针对QNN加速器的可重构多精度多通道位级阵列设计。我们在Ultra96 FPGA平台上实现了并评估了该设计方案。结果显示，我们的方案可以在推理混合精度模型上实现1.3185到3.5671倍的速度提升，并且减少了关键路径延迟，支持更高的时钟频率（250MHz）。
URL
https://arxiv.org/abs/2602.23334
PDF
-
RT-RMOT: A Dataset and Framework for RGB-Thermal Referring Multi-Object Tracking
2026-02-25 15:41:31Yanqiu Yu, Zhifan Jin, Sijia Chen, Tongfei Chu, En Yu, Liman Liu, Wenbing TaoAbstract
Referring Multi-Object Tracking has attracted increasing attention due to its human-friendly interactive characteristics, yet it exhibits limitations in low-visibility conditions, such as nighttime, smoke, and other challenging scenarios. To overcome this limitation, we propose a new RGB-Thermal RMOT task, named RT-RMOT, which aims to fuse RGB appearance features with the illumination robustness of the thermal modality to enable all-day referring multi-object tracking. To promote research on RT-RMOT, we construct the first Referring Multi-Object Tracking dataset under RGB-Thermal modality, named RefRT. It contains 388 language descriptions, 1,250 tracked targets, and 166,147 Language-RGB-Thermal (L-RGB-T) triplets. Furthermore, we propose RTrack, a framework built upon a multimodal large language model (MLLM) that integrates RGB, thermal, and textual features. Since the initial framework still leaves room for improvement, we introduce a Group Sequence Policy Optimization (GSPO) strategy to further exploit the model's potential. To alleviate training instability during RL fine-tuning, we introduce a Clipped Advantage Scaling (CAS) strategy to suppress gradient explosion. In addition, we design Structured Output Reward and Comprehensive Detection Reward to balance exploration and exploitation, thereby improving the completeness and accuracy of target perception. Extensive experiments on the RefRT dataset demonstrate the effectiveness of the proposed RTrack framework.
Abstract (translated)
基于参考的多对象跟踪（Referring Multi-Object Tracking，RMOT）由于其人性化的互动特性而备受关注，然而，在低可见度条件下如夜间、烟雾和其他具有挑战性的场景中，该技术表现出明显的局限性。为了克服这一限制，我们提出了一项新的RGB-热成像RMOT任务，命名为RT-RMOT（RGB-Thermal Referring Multi-Object Tracking），旨在融合RGB外观特征与热成像模态的照明鲁棒性，以实现全天候参考多对象跟踪。 为促进RT-RMOT的研究，我们构建了首个基于RGB-热成像模式下的参考多目标跟踪数据集，命名为RefRT。该数据集包含388条语言描述、1250个追踪目标和166,147组语言-RGB-热成像（L-RGB-T）三元组。 此外，我们提出了RTrack框架，它是基于一个多模态大型语言模型（Multimodal Large Language Model，MLLM），集成了RGB、热成像和文本特征。尽管初始的框架仍有改进空间，我们引入了分组序列策略优化（Group Sequence Policy Optimization，GSPO）策略来进一步挖掘该模型的潜力。 为了缓解强化学习微调过程中的训练不稳定性问题，我们提出了一种裁剪优势缩放（Clipped Advantage Scaling，CAS）策略以抑制梯度爆炸。同时，设计结构化输出奖励和综合检测奖励机制来平衡探索与利用的关系，从而提高目标感知的完整性和准确性。 在RefRT数据集上进行的广泛实验表明了所提出的RTrack框架的有效性。
URL
https://arxiv.org/abs/2602.22033
PDF
-
A Two-Stage Detection-Tracking Framework for Stable Apple Quality Inspection in Dense Conveyor-Belt Environments
2026-02-22 17:24:13Keonvin Park, Aditya Pal, Jin Hong MokAbstract
Industrial fruit inspection systems must operate reliably under dense multi-object interactions and continuous motion, yet most existing works evaluate detection or classification at the image level without ensuring temporal stability in video streams. We present a two-stage detection-tracking framework for stable multi-apple quality inspection in conveyor-belt environments. An orchard-trained YOLOv8 model performs apple localization, followed by ByteTrack multi-object tracking to maintain persistent identities. A ResNet18 defect classifier, fine-tuned on a healthy-defective fruit dataset, is applied to cropped apple regions. Track-level aggregation is introduced to enforce temporal consistency and reduce prediction oscillation across frames. We define video-level industrial metrics such as track-level defect ratio and temporal consistency to evaluate system robustness under realistic processing conditions. Results demonstrate improved stability compared to frame-wise inference, suggesting that integrating tracking is essential for practical automated fruit grading systems.
Abstract (translated)
工业水果检测系统必须在密集的多对象交互和连续运动下可靠运行，然而大多数现有的研究工作仅通过图像层面来评估检测或分类效果，并未确保视频流中的时间稳定性。我们提出了一种两阶段的检测-跟踪框架，用于传送带环境中稳定地进行苹果质量检查。该框架首先使用经过果园数据训练的YOLOv8模型定位苹果位置，然后利用ByteTrack多对象跟踪技术保持持久的身份标识。接下来，一个基于健康与缺陷果实数据集微调后的ResNet18缺陷分类器被应用于裁剪出的苹果区域。此外，我们引入了轨迹级别的聚合方法来强制执行时间一致性，并减少帧间预测的波动。 为了评估系统在真实处理条件下的鲁棒性，我们定义了一些视频层面的工业指标，如轨迹级别上的缺陷比率和时间一致性等。实验结果表明，与基于单帧的推理相比，我们的方法显著提高了系统的稳定性，这说明对于实际的自动化水果分级系统而言，集成跟踪技术是至关重要的。
URL
https://arxiv.org/abs/2602.19278
PDF
-
MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method
2026-02-20 05:43:47Ahsan Baidar Bakht, Mohamad Alansari, Muhayy Ud Din, Muzammal Naseer, Sajid Javed, Irfan Hussain, Jiri Matas, Arif MahmoodAbstract
Underwater Object Tracking (UOT) is crucial for efficient marine robotics, large scale ecological monitoring, and ocean exploration; however, progress has been hindered by the scarcity of large, multimodal, and diverse datasets. Existing benchmarks remain small and RGB only, limiting robustness under severe color distortion, turbidity, and low visibility conditions. We introduce MUOT_3M, the first pseudo multimodal UOT benchmark comprising 3 million frames from 3,030 videos (27.8h) annotated with 32 tracking attributes, 677 fine grained classes, and synchronized RGB, estimated enhanced RGB, estimated depth, and language modalities validated by a marine biologist. Building upon MUOT_3M, we propose MUTrack, a SAM-based multimodal to unimodal tracker featuring visual geometric alignment, vision language fusion, and four level knowledge distillation that transfers multimodal knowledge into a unimodal student model. Extensive evaluations across five UOT benchmarks demonstrate that MUTrack achieves up to 8.40% higher AUC and 7.80% higher precision than the strongest SOTA baselines while running at 24 FPS. MUOT_3M and MUTrack establish a new foundation for scalable, multimodally trained yet practically deployable underwater tracking.
Abstract (translated)
水下目标跟踪（UOT）对于高效的海洋机器人技术、大规模生态监测和海洋探索至关重要；然而，由于缺乏大型、多模态且多样化的数据集，这一领域的进步受到了阻碍。现有的基准测试规模较小，并且仅限于RGB模式，这限制了其在严重色彩失真、浑浊及低能见度条件下的鲁棒性。我们引入了MUOT_3M，这是第一个伪多模态UOT基准库，包含来自3,030段视频（共计27.8小时）的3百万帧，并且这些视频被标注有32个跟踪属性、677种精细分类以及同步的RGB图像、估计增强后的RGB图像、估计深度图和语言模态数据，由海洋生物学家验证。基于MUOT_3M，我们提出了MUTrack，这是一种基于SAM（Segment Anything Model）的多模态到单模态跟踪器，具有视觉几何对齐、视觉与语言融合以及四层知识蒸馏功能，能够将多模态的知识传递给单一模态的学生模型中。 在五个UOT基准库上进行广泛的评估表明，MUTrack比最强的SOTA（State-of-the-Art）基线最高提升了8.40%的AUC和7.80%的精度，并且能够在24 FPS的速度下运行。MUOT_3M与MUTrack为可扩展、多模态训练但实际部署可行的水下跟踪技术奠定了新的基础。
URL
https://arxiv.org/abs/2602.18006
PDF
-
HiMAP: History-aware Map-occupancy Prediction with Fallback
2026-02-19 10:24:02Yiming Xu, Yi Yang, Hao Cheng, Monika SesterAbstract
Accurate motion forecasting is critical for autonomous driving, yet most predictors rely on multi-object tracking (MOT) with identity association, assuming that objects are correctly and continuously tracked. When tracking fails due to, e.g., occlusion, identity switches, or missed detections, prediction quality degrades and safety risks increase. We present \textbf{HiMAP}, a tracking-free, trajectory prediction framework that remains reliable under MOT failures. HiMAP converts past detections into spatiotemporally invariant historical occupancy maps and introduces a historical query module that conditions on the current agent state to iteratively retrieve agent-specific history from unlabeled occupancy representations. The retrieved history is summarized by a temporal map embedding and, together with the final query and map context, drives a DETR-style decoder to produce multi-modal future trajectories. This design lifts identity reliance, supports streaming inference via reusable encodings, and serves as a robust fallback when tracking is unavailable. On Argoverse~2, HiMAP achieves performance comparable to tracking-based methods while operating without IDs, and it substantially outperforms strong baselines in the no-tracking setting, yielding relative gains of 11\% in FDE, 12\% in ADE, and a 4\% reduction in MR over a fine-tuned QCNet. Beyond aggregate metrics, HiMAP delivers stable forecasts for all agents simultaneously without waiting for tracking to recover, highlighting its practical value for safety-critical autonomy. The code is available under: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2602.17231
PDF
-
GOT-JEPA: Generic Object Tracking with Model Adaptation and Occlusion Handling using Joint-Embedding Predictive Architecture
2026-02-16 14:26:07Shih-Fang Chen, Jun-Cheng Chen, I-Hong Jhuo, Yen-Yu LinAbstract
The human visual system tracks objects by integrating current observations with previously observed information, adapting to target and scene changes, and reasoning about occlusion at fine granularity. In contrast, recent generic object trackers are often optimized for training targets, which limits robustness and generalization in unseen scenarios, and their occlusion reasoning remains coarse, lacking detailed modeling of occlusion patterns. To address these limitations in generalization and occlusion perception, we propose GOT-JEPA, a model-predictive pretraining framework that extends JEPA from predicting image features to predicting tracking models. Given identical historical information, a teacher predictor generates pseudo-tracking models from a clean current frame, and a student predictor learns to predict the same pseudo-tracking models from a corrupted version of the current frame. This design provides stable pseudo supervision and explicitly trains the predictor to produce reliable tracking models under occlusions, distractors, and other adverse observations, improving generalization to dynamic environments. Building on GOT-JEPA, we further propose OccuSolver to enhance occlusion perception for object tracking. OccuSolver adapts a point-centric point tracker for object-aware visibility estimation and detailed occlusion-pattern capture. Conditioned on object priors iteratively generated by the tracker, OccuSolver incrementally refines visibility states, strengthens occlusion handling, and produces higher-quality reference labels that progressively improve subsequent model predictions. Extensive evaluations on seven benchmarks show that our method effectively enhances tracker generalization and robustness.
Abstract (translated)
URL
https://arxiv.org/abs/2602.14771
PDF
-
Offline-Poly: A Polyhedral Framework For Offline 3D Multi-Object Tracking
2026-02-14 13:34:21Xiaoyu Li, Yitao Wu, Xian Wu, Haolin Zhuo, Lijun Zhao, Lining SunAbstract
Offline 3D multi-object tracking (MOT) is a critical component of the 4D auto-labeling (4DAL) process. It enhances pseudo-labels generated by high-performance detectors through the incorporation of temporal context. However, existing offline 3D MOT approaches are direct extensions of online frameworks and fail to fully exploit the advantages of offline setting. Moreover, these methods often depend on fixed upstream and customized architectures, limiting their adaptability. To address these limitations, we propose Offline-Poly, a general offline 3D MOT method based on a tracking-centric design. We introduce a standardized paradigm termed Tracking-by-Tracking (TBT), which operates exclusively on arbitrary off-the-shelf tracking outputs and produces offline-refined tracklets. This formulation decouples offline tracker from specific upstream detectors or trackers. Under the TBT paradigm, Offline-Poly accepts one or multiple coarse tracking results and processes them through a structured pipeline comprising pre-processing, hierarchical matching and fusion, and tracklet refinement. Each module is designed to capitalize on the two fundamental properties of offline tracking: resource unconstrainedness, which permits global optimization beyond real-time limits, and future observability, which enables tracklet reasoning over the full temporal horizon. Offline-Poly first eliminates short-term ghost tracklets and re-identifies fragmented segments using global scene context. It then constructs scene-level similarity to associate tracklets across multiple input sources. Finally, Offline-Poly refines tracklets by jointly leveraging local and global motion patterns. On nuScenes, we achieve SOTA performance with 77.6% AMOTA. On KITTI, it achieves leading results with 83.00% HOTA. Comprehensive experiments further validate the flexibility, generalizability, and modular effectiveness of Offline-Poly.
Abstract (translated)
URL
https://arxiv.org/abs/2602.13772
PDF
-
Layer-Guided UAV Tracking: Enhancing Efficiency and Occlusion Robustness
2026-02-14 07:02:25Yang Zhou, Derui Ding, Ran Sun, Ying Sun, Haohua ZhangAbstract
Visual object tracking (VOT) plays a pivotal role in unmanned aerial vehicle (UAV) applications. Addressing the trade-off between accuracy and efficiency, especially under challenging conditions like unpredictable occlusion, remains a significant challenge. This paper introduces LGTrack, a unified UAV tracking framework that integrates dynamic layer selection, efficient feature enhancement, and robust representation learning for occlusions. By employing a novel lightweight Global-Grouped Coordinate Attention (GGCA) module, LGTrack captures long-range dependencies and global contexts, enhancing feature discriminability with minimal computational overhead. Additionally, a lightweight Similarity-Guided Layer Adaptation (SGLA) module replaces knowledge distillation, achieving an optimal balance between tracking precision and inference efficiency. Experiments on three datasets demonstrate LGTrack's state-of-the-art real-time speed (258.7 FPS on UAVDT) while maintaining competitive tracking accuracy (82.8\% precision). Code is available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2602.13636
PDF
-
Detecting Object Tracking Failure via Sequential Hypothesis Testing
2026-02-13 14:57:15Alejandro Monroy Mu\~noz, Rajeev Verma, Alexander TimansAbstract
Real-time online object tracking in videos constitutes a core task in computer vision, with wide-ranging applications including video surveillance, motion capture, and robotics. Deployed tracking systems usually lack formal safety assurances to convey when tracking is reliable and when it may fail, at best relying on heuristic measures of model confidence to raise alerts. To obtain such assurances we propose interpreting object tracking as a sequential hypothesis test, wherein evidence for or against tracking failures is gradually accumulated over time. Leveraging recent advancements in the field, our sequential test (formalized as an e-process) quickly identifies when tracking failures set in whilst provably containing false alerts at a desired rate, and thus limiting potentially costly re-calibration or intervention steps. The approach is computationally light-weight, requires no extra training or fine-tuning, and is in principle model-agnostic. We propose both supervised and unsupervised variants by leveraging either ground-truth or solely internal tracking information, and demonstrate its effectiveness for two established tracking models across four video benchmarks. As such, sequential testing can offer a statistically grounded and efficient mechanism to incorporate safety assurances into real-time tracking systems.
Abstract (translated)
URL
https://arxiv.org/abs/2602.12983
PDF
-
GOT-Edit: Geometry-Aware Generic Object Tracking via Online Model Editing
2026-02-09 11:50:29Shih-Fang Chen, Jun-Cheng Chen, I-Hong Jhuo, Yen-Yu LinAbstract
Human perception for effective object tracking in a 2D video stream arises from the implicit use of prior 3D knowledge combined with semantic reasoning. In contrast, most generic object tracking (GOT) methods primarily rely on 2D features of the target and its surroundings while neglecting 3D geometric cues, which makes them susceptible to partial occlusion, distractors, and variations in geometry and appearance. To address this limitation, we introduce GOT-Edit, an online cross-modality model editing approach that integrates geometry-aware cues into a generic object tracker from a 2D video stream. Our approach leverages features from a pre-trained Visual Geometry Grounded Transformer to enable geometric cue inference from only a few 2D images. To tackle the challenge of seamlessly combining geometry and semantics, GOT-Edit performs online model editing with null-space constrained updates that incorporate geometric information while preserving semantic discrimination, yielding consistently better performance across diverse scenarios. Extensive experiments on multiple GOT benchmarks demonstrate that GOT-Edit achieves superior robustness and accuracy, particularly under occlusion and clutter, establishing a new paradigm for combining 2D semantics with 3D geometric reasoning for generic object tracking.
Abstract (translated)
人类在二维视频流中进行有效物体跟踪的能力来源于对隐含的三维知识和语义推理的运用。相比之下，大多数通用对象跟踪（GOT）方法主要依赖于目标及其周围环境的二维特征，并忽略几何线索，这使它们容易受到部分遮挡、干扰物以及形状和外观变化的影响。为了克服这一局限性，我们引入了GOT-Edit，这是一种在线跨模态模型编辑方法，它将几何感知线索融入到从二维视频流中获取的通用对象跟踪器中。我们的方法利用预训练的视觉几何基础变换器（Visual Geometry Grounded Transformer）中的特征，仅通过少数几幅二维图像就能推断出几何线索。 为了应对无缝结合几何和语义的挑战，GOT-Edit采用在线模型编辑技术，并使用零空间约束更新来整合几何信息同时保持语义区分性，从而在各种场景中获得更优性能。在多个GOT基准测试中的广泛实验表明，GOT-Edit实现了卓越的鲁棒性和准确性，特别是在遮挡和杂乱环境下表现尤为突出，这标志着结合二维语义与三维几何推理的新范式用于通用对象跟踪。
URL
https://arxiv.org/abs/2602.08550
PDF
-
A 96pJ/Frame/Pixel and 61pJ/Event Anti-UAV System with Hybrid Object Tracking Modes
2026-02-05 15:04:39Yuncheng Lu, Yucen Shi, Aobo Li, Zehao Li, Junying Li, Bo Wang, Tony Tae-Hyoung KimAbstract
We present an energy-efficient anti-UAV system that integrates frame-based and event-driven object tracking to enable reliable detection of small and fast-moving drones. The system reconstructs binary event frames using run-length encoding, generates region proposals, and adaptively switches between frame mode and event mode based on object size and velocity. A Fast Object Tracking Unit improves robustness for high-speed targets through adaptive thresholding and trajectory-based classification. The neural processing unit supports both grayscale-patch and trajectory inference with a custom instruction set and a zero-skipping MAC architecture, reducing redundant neural computations by more than 97 percent. Implemented in 40 nm CMOS technology, the 2 mm^2 chip achieves 96 pJ per frame per pixel and 61 pJ per event at 0.8 V, and reaches 98.2 percent recognition accuracy on public UAV datasets across 50 to 400 m ranges and 5 to 80 pixels per second speeds. The results demonstrate state-of-the-art end-to-end energy efficiency for anti-UAV systems.
Abstract (translated)
我们提出了一种节能的反无人机系统，该系统结合了基于帧和事件驱动的对象跟踪技术，以实现对小型快速移动无人机的可靠检测。该系统使用行程长度编码重构二进制事件帧，并生成区域提案。根据目标大小和速度，它可以自适应地在帧模式和事件模式之间切换。快速物体跟踪单元通过自适应阈值调整和基于轨迹分类提高了高速目标的鲁棒性。神经处理单元支持灰度补丁和轨迹推理，采用了一套定制指令集以及零跳过MAC（矩阵乘法累加）架构，在减少冗余神经计算方面超过了97%的效率。 在40纳米CMOS技术中实现的芯片面积为2平方毫米，该系统在0.8伏电压下每帧每像素的能量消耗为96皮焦耳，每个事件的能量消耗为61皮焦耳。它在公开的无人机数据集上实现了高达98.2%的识别准确率，在50到400米的距离范围内和每秒5至80个像素的速度时性能均表现良好。 实验结果表明该系统达到了反无人机系统的端到端能耗效率的新水平。
URL
https://arxiv.org/abs/2512.17939
PDF
-
UniTrack: Differentiable Graph Representation Learning for Multi-Object Tracking
2026-02-04 20:44:16Bishoy Galoaa, Xiangyu Bai, Utsav Nandi, Sai Siddhartha Vivek Dhir Rangoju, Somaieh Amraee, Sarah OstadabbasAbstract
We present UniTrack, a plug-and-play graph-theoretic loss function designed to significantly enhance multi-object tracking (MOT) performance by directly optimizing tracking-specific objectives through unified differentiable learning. Unlike prior graph-based MOT methods that redesign tracking architectures, UniTrack provides a universal training objective that integrates detection accuracy, identity preservation, and spatiotemporal consistency into a single end-to-end trainable loss function, enabling seamless integration with existing MOT systems without architectural modifications. Through differentiable graph representation learning, UniTrack enables networks to learn holistic representations of motion continuity and identity relationships across frames. We validate UniTrack across diverse tracking models and multiple challenging benchmarks, demonstrating consistent improvements across all tested architectures and datasets including Trackformer, MOTR, FairMOT, ByteTrack, GTR, and MOTE. Extensive evaluations show up to 53\% reduction in identity switches and 12\% IDF1 improvements across challenging benchmarks, with GTR achieving peak performance gains of 9.7\% MOTA on SportsMOT.
Abstract (translated)
我们介绍 UniTrack，这是一种即插即用的图论损失函数，旨在通过统一可微学习直接优化特定于跟踪的目标来显著提升多目标跟踪（MOT）性能。与以往需要重新设计跟踪架构的基于图形的方法不同，UniTrack 提供了一个通用的训练目标，将检测精度、身份保持和时空一致性整合为一个单一的端到端可训练损失函数，从而无需对现有 MOT 系统进行架构修改即可实现无缝集成。 通过可微图表示学习，UniTrack 使网络能够跨帧学习运动连续性和身份关系的整体表示。我们在各种跟踪模型和多个具有挑战性的基准测试中验证了 UniTrack 的性能，展示了所有测试的架构和数据集（包括 Trackformer、MOTR、FairMOT、ByteTrack、GTR 和 MOTE）上的一致改进。 广泛的评估结果显示，在具有挑战性的基准测试中身份切换减少了高达 53%，IDF1 改进达 12%。在 SportsMOT 基准测试中，GTR 达到了峰值性能提升，即 MOTA（多目标跟踪精度）提高了 9.7%。
URL
https://arxiv.org/abs/2602.05037
PDF
-
DRMOT: A Dataset and Framework for RGBD Referring Multi-Object Tracking
2026-02-04 15:56:16Sijia Chen, Lijuan Ma, Yanqiu Yu, En Yu, Liman Liu, Wenbing TaoAbstract
Referring Multi-Object Tracking (RMOT) aims to track specific targets based on language descriptions and is vital for interactive AI systems such as robotics and autonomous driving. However, existing RMOT models rely solely on 2D RGB data, making it challenging to accurately detect and associate targets characterized by complex spatial semantics (e.g., ``the person closest to the camera'') and to maintain reliable identities under severe occlusion, due to the absence of explicit 3D spatial information. In this work, we propose a novel task, RGBD Referring Multi-Object Tracking (DRMOT), which explicitly requires models to fuse RGB, Depth (D), and Language (L) modalities to achieve 3D-aware tracking. To advance research on the DRMOT task, we construct a tailored RGBD referring multi-object tracking dataset, named DRSet, designed to evaluate models' spatial-semantic grounding and tracking capabilities. Specifically, DRSet contains RGB images and depth maps from 187 scenes, along with 240 language descriptions, among which 56 descriptions incorporate depth-related information. Furthermore, we propose DRTrack, a MLLM-guided depth-referring tracking framework. DRTrack performs depth-aware target grounding from joint RGB-D-L inputs and enforces robust trajectory association by incorporating depth cues. Extensive experiments on the DRSet dataset demonstrate the effectiveness of our framework.
Abstract (translated)
基于语言描述的多目标跟踪（RMOT）旨在追踪特定的目标，这对于机器人和自动驾驶等互动式AI系统至关重要。然而，现有的RMOT模型仅依赖于2D RGB数据，在处理具有复杂空间语义特征的目标时面临挑战（例如，“离相机最近的人”），并且在严重遮挡情况下难以保持可靠的身份识别，因为缺乏明确的3D空间信息。 为此，我们提出了一项新的任务——RGBD基于语言描述的多目标跟踪（DRMOT），该任务要求模型融合RGB、深度（D）和语言（L）模态来实现具有3D感知能力的目标追踪。为了推进DRMOT的研究，我们构建了一个名为DRSet的定制化RGBD基于语言描述的多对象跟踪数据集，专门用于评估模型的空间语义接地能力和追踪性能。具体来说，DRSet包含来自187个场景中的RGB图像和深度图，以及240条语言描述，其中56条描述包含了与深度相关的信息。 此外，我们提出了DRTrack框架，该框架由多模态语言-视觉大规模预训练模型（MLLM）指导，并利用深度信息进行目标跟踪。DRTrack从联合的RGB-D-L输入中执行具有深度感知的目标定位，并通过整合深度线索来强化轨迹关联，从而实现稳健的追踪性能。 在DRSet数据集上的广泛实验验证了我们框架的有效性。
URL
https://arxiv.org/abs/2602.04692
PDF
-
GTATrack: Winner Solution to SoccerTrack 2025 with Deep-EIoU and Global Tracklet Association
2026-01-31 03:08:48Rong-Lin Jian, Ming-Chi Luo, Chen-Wei Huang, Chia-Ming Lee, Yu-Fan Lin, Chih-Chung HsuAbstract
Multi-object tracking (MOT) in sports is highly challenging due to irregular player motion, uniform appearances, and frequent occlusions. These difficulties are further exacerbated by the geometric distortion and extreme scale variation introduced by static fisheye cameras. In this work, we present GTATrack, a hierarchical tracking framework that win first place in the SoccerTrack Challenge 2025. GTATrack integrates two core components: Deep Expansion IoU (Deep-EIoU) for motion-agnostic online association and Global Tracklet Association (GTA) for trajectory-level refinement. This two-stage design enables both robust short-term matching and long-term identity consistency. Additionally, a pseudo-labeling strategy is used to boost detector recall on small and distorted targets. The synergy between local association and global reasoning effectively addresses identity switches, occlusions, and tracking fragmentation. Our method achieved a winning HOTA score of 0.60 and significantly reduced false positives to 982, demonstrating state-of-the-art accuracy in fisheye-based soccer tracking. Our code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2602.00484
PDF
