Abstract
Modern autonomous drone missions increasingly require software frameworks capable of seamlessly integrating structured symbolic planning with adaptive reinforcement learning (RL). Although traditional rule-based architectures offer robust structured reasoning for drone autonomy, their capabilities fall short in dynamically complex operational environments that require adaptive symbolic planning. Symbolic RL (SRL), using the Planning Domain Definition Language (PDDL), explicitly integrates domain-specific knowledge and operational constraints, significantly improving the reliability and safety of unmanned aerial vehicle (UAV) decision making. In this study, we propose the AMAD-SRL framework, an extended and refined version of the Autonomous Mission Agents for Drones (AMAD) cognitive multi-agent architecture, enhanced with symbolic reinforcement learning for dynamic mission planning and execution. We validated our framework in a Software-in-the-Loop (SIL) environment structured identically to an intended Hardware-In-the-Loop Simulation (HILS) platform, ensuring seamless transition to real hardware. Experimental results demonstrate stable integration and interoperability of modules, successful transitions between BDI-driven and symbolic RL-driven planning phases, and consistent mission performance. Specifically, we evaluate a target acquisition scenario in which the UAV plans a surveillance path followed by a dynamic reentry path to secure the target while avoiding threat zones. In this SIL evaluation, mission efficiency improved by approximately 75% over a coverage-based baseline, measured by travel distance reduction. This study establishes a robust foundation for handling complex UAV missions and discusses directions for further enhancement and validation.
Abstract (translated)
现代自主无人机任务越来越需要能够无缝集成结构化符号规划与自适应强化学习(RL)的软件框架。尽管传统的基于规则的架构为无人机自主性提供了稳健的结构化推理能力,但在需要自适应符号规划的动态复杂操作环境中,这些架构的能力显得不足。使用规划领域定义语言(PDDL),符号强化学习(SRL)明确地集成了特定领域的知识和操作约束,显著提高了无人航空器(UAV)决策制定的可靠性和安全性。 在此研究中,我们提出了AMAD-SRL框架,这是自主任务代理无人机(AMAD)认知多代理架构的一个扩展和改进版本,并且加强了动态任务规划与执行方面的符号强化学习功能。我们在一个结构上等同于预期硬件在环仿真(HILS)平台的软件在环(SIL)环境中验证了我们的框架,以确保无缝过渡到真实硬件中。实验结果表明,模块的稳定集成和互操作性、BDI驱动和符号RL驱动规划阶段之间成功的转换以及一致的任务性能均得到了证明。 具体而言,在目标获取场景评估中,UAV计划了一条监视路径,并随后规划一条动态重入路径以确保安全地在避开威胁区域的情况下锁定目标。在此SIL评估中,任务效率相对于基于覆盖度的基线提高了大约75%,通过减少行驶距离来衡量。 这项研究为处理复杂的无人机任务奠定了坚实的基础,并讨论了进一步改进和验证的方向。
URL
https://arxiv.org/abs/2508.11890