Abstract
Advances in 3D reconstruction and novel view synthesis have enabled efficient, photorealistic rendering, but the data collection process remains largely manual, making it time-consuming and labor-intensive. To address the challenges, this study introduces Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection (Hestia), which leverages reinforcement learning to learn a generalizable policy for 5-DoF next-best viewpoint prediction. Unlike prior approaches, Hestia systematically defines the next-best-view task by proposing core components such as dataset choice, observation design, action space, reward calculation, and learning schemes, forming a foundation for the planner. Hestia goes beyond prior next-best-view approaches and traditional capture systems through integration and validation in a real-world setup, where a drone serves as a mobile sensor for active scene exploration. Experimental results show that Hestia performs robustly across three datasets and translated object settings in the NVIDIA IsaacLab environment, and proves feasible for real-world deployment.
Abstract (translated)
在三维重建和新视角合成方面的进展已经使得高效、逼真的渲染成为可能,但数据采集过程仍然主要是手动进行的,这既耗时又耗费人力。为了应对这一挑战,本研究提出了分层最佳视点探索方法(Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection, 简称Hestia),该方法利用强化学习来学习一种适用于5自由度下最佳视点预测的通用策略。与先前的方法不同,Hestia系统地定义了最佳视点任务,并提出了核心组件,如数据集选择、观测设计、动作空间、奖励计算和学习方案,为规划者奠定了基础。 Hestia超越了之前的最佳视点方法及传统的捕捉系统,在一个实际设置中通过无人机作为移动传感器进行主动场景探索实现了集成与验证。实验结果表明,Hestia在NVIDIA IsaacLab环境中三个数据集和转换后的物体设置上表现稳健,并证明其适合于现实世界的部署。 总结起来,该研究介绍了一种创新的自动数据采集系统(Hestia),它能够高效地预测最佳视点并进行3D重建工作,为自动化、智能化的数据采集提供了新的解决方案。
URL
https://arxiv.org/abs/2508.01014