Abstract
Underwater Object Tracking (UOT) is crucial for efficient marine robotics, large scale ecological monitoring, and ocean exploration; however, progress has been hindered by the scarcity of large, multimodal, and diverse datasets. Existing benchmarks remain small and RGB only, limiting robustness under severe color distortion, turbidity, and low visibility conditions. We introduce MUOT_3M, the first pseudo multimodal UOT benchmark comprising 3 million frames from 3,030 videos (27.8h) annotated with 32 tracking attributes, 677 fine grained classes, and synchronized RGB, estimated enhanced RGB, estimated depth, and language modalities validated by a marine biologist. Building upon MUOT_3M, we propose MUTrack, a SAM-based multimodal to unimodal tracker featuring visual geometric alignment, vision language fusion, and four level knowledge distillation that transfers multimodal knowledge into a unimodal student model. Extensive evaluations across five UOT benchmarks demonstrate that MUTrack achieves up to 8.40% higher AUC and 7.80% higher precision than the strongest SOTA baselines while running at 24 FPS. MUOT_3M and MUTrack establish a new foundation for scalable, multimodally trained yet practically deployable underwater tracking.
Abstract (translated)
水下目标跟踪（UOT）对于高效的海洋机器人技术、大规模生态监测和海洋探索至关重要；然而，由于缺乏大型、多模态且多样化的数据集，这一领域的进步受到了阻碍。现有的基准测试规模较小，并且仅限于RGB模式，这限制了其在严重色彩失真、浑浊及低能见度条件下的鲁棒性。我们引入了MUOT_3M，这是第一个伪多模态UOT基准库，包含来自3,030段视频（共计27.8小时）的3百万帧，并且这些视频被标注有32个跟踪属性、677种精细分类以及同步的RGB图像、估计增强后的RGB图像、估计深度图和语言模态数据，由海洋生物学家验证。基于MUOT_3M，我们提出了MUTrack，这是一种基于SAM（Segment Anything Model）的多模态到单模态跟踪器，具有视觉几何对齐、视觉与语言融合以及四层知识蒸馏功能，能够将多模态的知识传递给单一模态的学生模型中。 在五个UOT基准库上进行广泛的评估表明，MUTrack比最强的SOTA（State-of-the-Art）基线最高提升了8.40%的AUC和7.80%的精度，并且能够在24 FPS的速度下运行。MUOT_3M与MUTrack为可扩展、多模态训练但实际部署可行的水下跟踪技术奠定了新的基础。
URL
https://arxiv.org/abs/2602.18006
