Paper Reading AI Learner

MVL-Loc: Leveraging Vision-Language Model for Generalizable Multi-Scene Camera Relocalization

2025-07-06 18:52:16
Zhendong Xiao, Wu Wei, Shujie Ji, Shan Yang, Changhao Chen

Abstract

Camera relocalization, a cornerstone capability of modern computer vision, accurately determines a camera's position and orientation (6-DoF) from images and is essential for applications in augmented reality (AR), mixed reality (MR), autonomous driving, delivery drones, and robotic navigation. Unlike traditional deep learning-based methods that regress camera pose from images in a single scene, which often lack generalization and robustness in diverse environments, we propose MVL-Loc, a novel end-to-end multi-scene 6-DoF camera relocalization framework. MVL-Loc leverages pretrained world knowledge from vision-language models (VLMs) and incorporates multimodal data to generalize across both indoor and outdoor settings. Furthermore, natural language is employed as a directive tool to guide the multi-scene learning process, facilitating semantic understanding of complex scenes and capturing spatial relationships among objects. Extensive experiments on the 7Scenes and Cambridge Landmarks datasets demonstrate MVL-Loc's robustness and state-of-the-art performance in real-world multi-scene camera relocalization, with improved accuracy in both positional and orientational estimates.

Abstract (translated)

相机重定位是现代计算机视觉领域的关键技术之一,它能准确地从图像中确定出相机的位置和姿态(6自由度),在增强现实(AR)、混合现实(MR)、自动驾驶、送货无人机以及机器人导航等领域有着重要的应用价值。不同于传统的基于深度学习的方法通过单场景图像回归相机位姿,这些方法往往缺乏多样环境下的泛化能力和鲁棒性,我们提出了一种名为MVL-Loc的新颖端到端多场景6自由度相机重定位框架。 MVL-Loc利用了视觉语言模型(VLM)预先训练的世界知识,并结合多种模态的数据来实现室内和室外环境下的一致泛化。此外,自然语言被用作引导多场景学习过程的工具,有助于复杂场景的语义理解以及物体间空间关系的捕捉。在7Scenes和Cambridge Landmarks数据集上的大量实验表明,MVL-Loc具备强大的鲁棒性和业界领先的性能,在真实世界的多场景相机重定位任务中实现了位置和姿态估计精度的提升。

URL

https://arxiv.org/abs/2507.04509

PDF

https://arxiv.org/pdf/2507.04509.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot