Abstract
Recent advances in large Vision-Language Models (VLMs) have provided rich semantic understanding that empowers drones to search for open-set objects via natural language instructions. However, prior systems struggle to integrate VLMs into practical aerial systems due to orders-of-magnitude frequency mismatch between VLM inference and real-time planning, as well as VLMs' limited 3D scene understanding. They also lack a unified mechanism to balance semantic guidance with motion efficiency in large-scale environments. To address these challenges, we present AirHunt, an aerial object navigation system that efficiently locates open-set objects with zero-shot generalization in outdoor environments by seamlessly fusing VLM semantic reasoning with continuous path planning. AirHunt features a dual-pathway asynchronous architecture that establishes a synergistic interface between VLM reasoning and path planning, enabling continuous flight with adaptive semantic guidance that evolves through motion. Moreover, we propose an active dual-task reasoning module that exploits geometric and semantic redundancy to enable selective VLM querying, and a semantic-geometric coherent planning module that dynamically reconciles semantic priorities and motion efficiency in a unified framework, enabling seamless adaptation to environmental heterogeneity. We evaluate AirHunt across diverse object navigation tasks and environments, demonstrating a higher success rate with lower navigation error and reduced flight time compared to state-of-the-art methods. Real-world experiments further validate AirHunt's practical capability in complex and challenging environments. Code and dataset will be made publicly available before publication.
Abstract (translated)
近期在大型视觉-语言模型(VLMs)方面的进展为无人机通过自然语言指令搜索开放集对象提供了丰富的语义理解能力。然而,先前的系统由于VLM推断与实时规划之间频率不匹配以及VLM对3D场景理解有限的问题,在将VLM集成到实际空中系统中时遇到了困难。此外,这些系统缺乏一种统一机制来在大规模环境中平衡语义引导和运动效率。为了解决这些问题,我们提出了AirHunt,这是一种高效的空中物体导航系统,它通过无缝融合VLM的语义推理与连续路径规划,在户外环境中实现开放集对象的零样本泛化定位。 AirHunt具有双路径异步架构,建立了VLM推理与路径规划之间的协同接口,允许根据运动不断演变的自适应语义指导进行持续飞行。此外,我们提出了一种主动双重任务推理模块,利用几何和语义冗余实现选择性的VLM查询,并提出了一个语义-几何协调规划模块,在统一框架内动态平衡语义优先级与运动效率,使系统能够无缝地适应环境异质性。 我们在各种物体导航任务和环境中评估了AirHunt,证明它比最先进的方法具有更高的成功率、更低的导航误差以及更短的飞行时间。实际世界中的实验进一步验证了AirHunt在复杂且具挑战性的环境下的实用能力。发布前,代码和数据集将公开提供。 通过这种方式,AirHunt不仅克服了现有系统的局限性,还为无人机的应用开辟了新的可能,特别是在搜索与救援、农业监测等领域。
URL
https://arxiv.org/abs/2601.12742