Abstract
Existing RGB-Event visual object tracking approaches primarily rely on conventional feature-level fusion, failing to fully exploit the unique advantages of event cameras. In particular, the high dynamic range and motion-sensitive nature of event cameras are often overlooked, while low-information regions are processed uniformly, leading to unnecessary computational overhead for the backbone network. To address these issues, we propose a novel tracking framework that performs early fusion in the frequency domain, enabling effective aggregation of high-frequency information from the event modality. Specifically, RGB and event modalities are transformed from the spatial domain to the frequency domain via the Fast Fourier Transform, with their amplitude and phase components decoupled. High-frequency event information is selectively fused into RGB modality through amplitude and phase attention, enhancing feature representation while substantially reducing backbone computation. In addition, a motion-guided spatial sparsification module leverages the motion-sensitive nature of event cameras to capture the relationship between target motion cues and spatial probability distribution, filtering out low-information regions and enhancing target-relevant features. Finally, a sparse set of target-relevant features is fed into the backbone network for learning, and the tracking head predicts the final target position. Extensive experiments on three widely used RGB-Event tracking benchmark datasets, including FE108, FELT, and COESOT, demonstrate the high performance and efficiency of our method. The source code of this paper will be released on this https URL
Abstract (translated)
现有的RGB-事件视觉对象跟踪方法主要依赖于传统的特征级融合，未能充分利用事件相机的独特优势。特别是，高动态范围和运动敏感特性经常被忽视，而低信息区域则进行了统一处理，导致骨干网络产生了不必要的计算开销。为了解决这些问题，我们提出了一种新的追踪框架，在频域中进行早期融合，以有效地聚合来自事件模态的高频信息。具体来说，RGB和事件模态通过快速傅里叶变换从空间域转换到频域，并解耦了它们的振幅和相位成分。通过振幅和相位注意力机制选择性地将高频事件信息融合到RGB模态中，从而增强特征表示的同时大幅减少骨干网络计算量。此外，运动引导的空间稀疏化模块利用了事件相机的运动敏感特性来捕捉目标运动线索与空间概率分布之间的关系，过滤掉低信息区域，并增强相关的目标特征。最终，一组稀疏的相关目标特征被输入到骨干网络中进行学习，而追踪头则预测最终的目标位置。 在三个广泛使用的RGB-事件跟踪基准数据集（FE108、FELT和COESOT）上进行了大量实验，证明了我们方法的高性能和高效率。本文的源代码将在以下网址发布：[此URL]
URL
https://arxiv.org/abs/2601.01022
