Abstract
Camera relocalization, a cornerstone capability of modern computer vision, accurately determines a camera's position and orientation (6-DoF) from images and is essential for applications in augmented reality (AR), mixed reality (MR), autonomous driving, delivery drones, and robotic navigation. Unlike traditional deep learning-based methods that regress camera pose from images in a single scene, which often lack generalization and robustness in diverse environments, we propose MVL-Loc, a novel end-to-end multi-scene 6-DoF camera relocalization framework. MVL-Loc leverages pretrained world knowledge from vision-language models (VLMs) and incorporates multimodal data to generalize across both indoor and outdoor settings. Furthermore, natural language is employed as a directive tool to guide the multi-scene learning process, facilitating semantic understanding of complex scenes and capturing spatial relationships among objects. Extensive experiments on the 7Scenes and Cambridge Landmarks datasets demonstrate MVL-Loc's robustness and state-of-the-art performance in real-world multi-scene camera relocalization, with improved accuracy in both positional and orientational estimates.
Abstract (translated)
相机重定位是现代计算机视觉领域的关键技术之一,它能准确地从图像中确定出相机的位置和姿态(6自由度),在增强现实(AR)、混合现实(MR)、自动驾驶、送货无人机以及机器人导航等领域有着重要的应用价值。不同于传统的基于深度学习的方法通过单场景图像回归相机位姿,这些方法往往缺乏多样环境下的泛化能力和鲁棒性,我们提出了一种名为MVL-Loc的新颖端到端多场景6自由度相机重定位框架。 MVL-Loc利用了视觉语言模型(VLM)预先训练的世界知识,并结合多种模态的数据来实现室内和室外环境下的一致泛化。此外,自然语言被用作引导多场景学习过程的工具,有助于复杂场景的语义理解以及物体间空间关系的捕捉。在7Scenes和Cambridge Landmarks数据集上的大量实验表明,MVL-Loc具备强大的鲁棒性和业界领先的性能,在真实世界的多场景相机重定位任务中实现了位置和姿态估计精度的提升。
URL
https://arxiv.org/abs/2507.04509