Paper Reading AI Learner

Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approach

2025-08-22 21:29:59
Marco S. Tayar, Lucas K. de Oliveira, Juliano D. Negri, Thiago H. Segreto, Ricardo V. Godoy, Marcelo Becker

Abstract

Inspecting confined industrial infrastructure, such as ventilation shafts, is a hazardous and inefficient task for humans. Unmanned Aerial Vehicles (UAVs) offer a promising alternative, but GPS-denied environments require robust control policies to prevent collisions. Deep Reinforcement Learning (DRL) has emerged as a powerful framework for developing such policies, and this paper provides a comparative study of two leading DRL algorithms for this task: the on-policy Proximal Policy Optimization (PPO) and the off-policy Soft Actor-Critic (SAC). The training was conducted with procedurally generated duct environments in Genesis simulation environment. A reward function was designed to guide a drone through a series of waypoints while applying a significant penalty for collisions. PPO learned a stable policy that completed all evaluation episodes without collision, producing smooth trajectories. By contrast, SAC consistently converged to a suboptimal behavior that traversed only the initial segments before failure. These results suggest that, in hazard-dense navigation, the training stability of on-policy methods can outweigh the nominal sample efficiency of off-policy algorithms. More broadly, the study provides evidence that procedurally generated, high-fidelity simulations are effective testbeds for developing and benchmarking robust navigation policies.

Abstract (translated)

检查工业设施内部的受限区域,如通风井等,对于人类来说是一项危险且低效的任务。无人飞行器(UAV)提供了一种有前景的替代方案,但在无GPS信号的环境中操作时,需要具备强大的控制策略以防止碰撞事故的发生。深度强化学习(DRL)已成为开发此类控制策略的强大框架,并且本文通过对比研究两种领先于该任务领域的DRL算法——基于策略的近端策略优化(PPO)和非基于策略的软演员-评论家方法(SAC),来展示其效用。 训练过程在Genesis仿真环境中使用程序生成的通风管道环境进行。奖励函数被设计用来引导无人机按照一系列航点飞行,并对碰撞施加重罚。实验结果显示,PPO算法能够学习出一个稳定性的策略,在所有评估阶段均能无碰撞地完成任务,产生平稳的轨迹;而SAC则始终收敛到一种次优行为模式,即只通过最初的一些路段后便无法继续。 这些结果表明,在高风险密度环境中导航时,基于策略方法训练过程中的稳定性可能比非基于策略算法名义上的样本效率更为重要。更广泛地说,这项研究提供了证据,证明程序生成的、高度真实的仿真环境是开发和测试稳健导航策略的有效平台。

URL

https://arxiv.org/abs/2508.16807

PDF

https://arxiv.org/pdf/2508.16807.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot