Abstract
We present VISTA (Viewpoint-based Image selection with Semantic Task Awareness), an active exploration method for robots to plan informative trajectories that improve 3D map quality in areas most relevant for task completion. Given an open-vocabulary search instruction (e.g., "find a person"), VISTA enables a robot to explore its environment to search for the object of interest, while simultaneously building a real-time semantic 3D Gaussian Splatting reconstruction of the scene. The robot navigates its environment by planning receding-horizon trajectories that prioritize semantic similarity to the query and exploration of unseen regions of the environment. To evaluate trajectories, VISTA introduces a novel, efficient viewpoint-semantic coverage metric that quantifies both the geometric view diversity and task relevance in the 3D scene. On static datasets, our coverage metric outperforms state-of-the-art baselines, FisherRF and Bayes' Rays, in computation speed and reconstruction quality. In quadrotor hardware experiments, VISTA achieves 6x higher success rates in challenging maps, compared to baseline methods, while matching baseline performance in less challenging maps. Lastly, we show that VISTA is platform-agnostic by deploying it on a quadrotor drone and a Spot quadruped robot. Open-source code will be released upon acceptance of the paper.
Abstract (translated)
我们介绍了VISTA(基于语义任务感知的视角图像选择方法),这是一种机器人主动探索的方法,用于规划能够提高与任务完成最相关区域3D地图质量的信息轨迹。给定一个开放词汇搜索指令(例如,“找到一个人”),VISTA使机器人能够在环境中进行探索以寻找感兴趣的物体,同时实时构建场景的语义3D高斯点云重建。机器人通过规划递减时间范围内的轨迹在环境内导航,这些轨迹优先考虑与查询的语义相似性和对未见区域的探索。为了评估轨迹,VISTA引入了一种新颖且高效的视角-语义覆盖度量方法,该方法量化了3D场景中的几何视图多样性及任务相关性。 在静态数据集上,我们的覆盖率指标在计算速度和重建质量方面均优于最先进的基准方法FisherRF和Bayes' Rays。在四旋翼飞行器硬件实验中,在具有挑战性的地图环境下,VISTA相比基线方法成功率提高了6倍;而在相对不具挑战性的地图环境中,其性能则与基线方法相当。 最后,我们展示了VISTA的平台无关性,通过将其部署到四旋翼无人机和Spot四足机器人上进行验证。论文被接受后将发布开源代码。
URL
https://arxiv.org/abs/2507.01125