Abstract
Human-AI joint planning in Unmanned Aerial Vehicles (UAVs) typically relies on control handover when facing environmental uncertainties, which is often inefficient and cognitively demanding for non-expert operators. To address this, we propose a novel framework that shifts the collaboration paradigm from control takeover to active information elicitation. We introduce the Minimal Information Neuro-Symbolic Tree (MINT), a reasoning mechanism that explicitly structures knowledge gaps regarding obstacles and goals into a queryable format. By leveraging large language models, our system formulates optimal binary queries to resolve specific ambiguities with minimal human interaction. We demonstrate the efficacy of this approach through a comprehensive workflow integrating a vision-language model for perception, voice interfaces, and a low-level UAV control module in both high-fidelity NVIDIA Isaac simulations and real-world deployments. Experimental results show that our method achieves a significant improvement in the success rate for complex search-and-rescue tasks while significantly reducing the frequency of human interaction compared to exhaustive querying baselines.
Abstract (translated)
在无人飞行器(UAV)中,人机联合规划通常依赖于环境不确定情况下的控制权交接,这种方式对于非专业操作员来说既低效又认知负担重。为了解决这一问题,我们提出了一种新的框架,该框架将合作模式从控制接管转变为积极的信息获取。我们引入了最小信息神经符号树(Minimal Information Neuro-Symbolic Tree, MINT),这是一种推理机制,能够明确地将关于障碍物和目标的知识缺口结构化成可查询的格式。 通过利用大型语言模型,我们的系统可以构建最优二元查询来解决特定模糊性,并且只需最少的人类交互。我们展示了这一方法的有效性,它整合了一个视觉-语言模型用于感知、语音接口以及低级无人机控制模块,在高保真的NVIDIA Isaac模拟器和实际部署中均得以实现。 实验结果表明,与全面查询基线相比,我们的方法在复杂搜索和救援任务的成功率上实现了显著提升,并且大幅减少了人类交互的频率。
URL
https://arxiv.org/abs/2603.07824