Abstract
Multi-modal object tracking has attracted considerable attention by integrating multiple complementary inputs (e.g., thermal, depth, and event data) to achieve outstanding performance. Although current general-purpose multi-modal trackers primarily unify various modal tracking tasks (i.e., RGB-Thermal infrared, RGB-Depth or RGB-Event tracking) through prompt learning, they still overlook the effective capture of spatio-temporal cues. In this work, we introduce a novel multi-modal tracking framework based on a mamba-style state space model, termed UBATrack. Our UBATrack comprises two simple yet effective modules: a Spatio-temporal Mamba Adapter (STMA) and a Dynamic Multi-modal Feature Mixer. The former leverages Mamba's long-sequence modeling capability to jointly model cross-modal dependencies and spatio-temporal visual cues in an adapter-tuning manner. The latter further enhances multi-modal representation capacity across multiple feature dimensions to improve tracking robustness. In this way, UBATrack eliminates the need for costly full-parameter fine-tuning, thereby improving the training efficiency of multi-modal tracking algorithms. Experiments show that UBATrack outperforms state-of-the-art methods on RGB-T, RGB-D, and RGB-E tracking benchmarks, achieving outstanding results on the LasHeR, RGBT234, RGBT210, DepthTrack, VOT-RGBD22, and VisEvent datasets.
Abstract (translated)
多模态对象跟踪通过融合多种互补输入（如热成像、深度和事件数据）来实现卓越的性能，因此吸引了广泛的关注。尽管目前的一般用途多模态追踪器主要通过提示学习将各种模态追踪任务（即RGB-热红外、RGB-深度或RGB-事件追踪）统一起来，但它们仍忽视了对时空线索的有效捕捉。为此，我们引入了一种基于响尾蛇式状态空间模型的新型多模态跟踪框架，称为UBATrack。我们的UBATrack包含两个简单而有效的模块：时空响尾蛇适配器（STMA）和动态多模态特征混合器。前者利用了响尾蛇对长序列建模的能力，在适配器调优方式下联合建模跨模式依赖关系以及时空视觉线索。后者进一步增强了在多个特征维度上的多模态表示能力，从而提高了跟踪的鲁棒性。这样，UBATrack消除了成本高昂的全参数微调需求，从而提升了多模态追踪算法的训练效率。实验表明，UBATrack在RGB-T、RGB-D和RGB-E追踪基准测试上优于现有最佳方法，在LasHeR、RGBT234、RGBT210、DepthTrack、VOT-RGBD22以及VisEvent数据集上取得了卓越的结果。 这段文字概述了最新的多模态跟踪技术UBATrack，强调其在处理多种传感器输入时的高效性与准确性，并通过实验结果证明该框架的有效性和优越性。
URL
https://arxiv.org/abs/2601.14799
