Abstract
Recent advances in transformer-based lightweight object tracking have established new standards across benchmarks, leveraging the global receptive field and powerful feature extraction capabilities of attention mechanisms. Despite these achievements, existing methods universally employ sparse sampling during training--utilizing only one template and one search image per sequence--which fails to comprehensively explore spatiotemporal information in videos. This limitation constrains performance and cause the gap between lightweight and high-performance trackers. To bridge this divide while maintaining real-time efficiency, we propose STDTrack, a framework that pioneers the integration of reliable spatiotemporal dependencies into lightweight trackers. Our approach implements dense video sampling to maximize spatiotemporal information utilization. We introduce a temporally propagating spatiotemporal token to guide per-frame feature extraction. To ensure comprehensive target state representation, we disign the Multi-frame Information Fusion Module (MFIFM), which augments current dependencies using historical context. The MFIFM operates on features stored in our constructed Spatiotemporal Token Maintainer (STM), where a quality-based update mechanism ensures information reliability. Considering the scale variation among tracking targets, we develop a multi-scale prediction head to dynamically adapt to objects of different sizes. Extensive experiments demonstrate state-of-the-art results across six benchmarks. Notably, on GOT-10k, STDTrack rivals certain high-performance non-real-time trackers (e.g., MixFormer) while operating at 192 FPS(GPU) and 41 FPS(CPU).
Abstract (translated)
最近基于Transformer的轻量级目标跟踪技术取得了显著进展，在各项基准测试中确立了新的标准。这些进步得益于注意力机制提供的全局接受域和强大的特征提取能力。尽管已有方法取得了成就，但它们在训练过程中普遍采用稀疏采样策略——每条序列仅使用一个模板图像和一个搜索图像——这无法全面探索视频中的时空信息。这种局限性限制了性能，并导致轻量级跟踪器与高性能跟踪器之间的差距。 为了弥合这一鸿沟同时保持实时效率，我们提出了一种新的框架STDTrack，它首次将可靠的时空依赖关系引入到轻量级目标跟踪中。我们的方法通过密集视频采样最大化时空信息的利用。为指导逐帧特征提取，我们引入了可随时间传播的时空令牌。为了确保全面的目标状态表示，我们设计了一个多帧信息融合模块（MFIFM），该模块使用历史上下文来增强当前依赖关系。 MFIFM操作于由我们构建的时空令牌维护器（STM）中的特征上，其中基于质量的更新机制保证了信息的可靠性。考虑到跟踪目标之间的尺度变化，我们开发了一种多尺度预测头以动态适应不同大小的对象。广泛的实验显示，在六个基准测试中实现了当前最佳结果。值得注意的是，在GOT-10k数据集上，STDTrack在运行速度为每秒192帧（GPU）和41帧（CPU）的情况下，其性能可媲美某些高性能但非实时的目标跟踪器（例如MixFormer）。
URL
https://arxiv.org/abs/2601.09078
