Paper Reading AI Learner

Hestia: Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection

2025-08-01 18:27:23
Cheng-You Lu, Zhuoli Zhuang, Nguyen Thanh Trung Le, Da Xiao, Yu-Cheng Chang, Thomas Do, Srinath Sridhar, Chin-teng Lin

Abstract

Advances in 3D reconstruction and novel view synthesis have enabled efficient, photorealistic rendering, but the data collection process remains largely manual, making it time-consuming and labor-intensive. To address the challenges, this study introduces Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection (Hestia), which leverages reinforcement learning to learn a generalizable policy for 5-DoF next-best viewpoint prediction. Unlike prior approaches, Hestia systematically defines the next-best-view task by proposing core components such as dataset choice, observation design, action space, reward calculation, and learning schemes, forming a foundation for the planner. Hestia goes beyond prior next-best-view approaches and traditional capture systems through integration and validation in a real-world setup, where a drone serves as a mobile sensor for active scene exploration. Experimental results show that Hestia performs robustly across three datasets and translated object settings in the NVIDIA IsaacLab environment, and proves feasible for real-world deployment.

Abstract (translated)

在三维重建和新视角合成方面的进展已经使得高效、逼真的渲染成为可能,但数据采集过程仍然主要是手动进行的,这既耗时又耗费人力。为了应对这一挑战,本研究提出了分层最佳视点探索方法(Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection, 简称Hestia),该方法利用强化学习来学习一种适用于5自由度下最佳视点预测的通用策略。与先前的方法不同,Hestia系统地定义了最佳视点任务,并提出了核心组件,如数据集选择、观测设计、动作空间、奖励计算和学习方案,为规划者奠定了基础。 Hestia超越了之前的最佳视点方法及传统的捕捉系统,在一个实际设置中通过无人机作为移动传感器进行主动场景探索实现了集成与验证。实验结果表明,Hestia在NVIDIA IsaacLab环境中三个数据集和转换后的物体设置上表现稳健,并证明其适合于现实世界的部署。 总结起来,该研究介绍了一种创新的自动数据采集系统(Hestia),它能够高效地预测最佳视点并进行3D重建工作,为自动化、智能化的数据采集提供了新的解决方案。

URL

https://arxiv.org/abs/2508.01014

PDF

https://arxiv.org/pdf/2508.01014.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot