Drone
-
Agentic AI Meets Edge Computing in Autonomous UAV Swarms
2026-01-20 19:45:33Thuan Minh Nguyen, Vu Tuan Truong, Long Bao LeAbstract
The integration of agentic AI, powered by large language models (LLMs) with autonomous reasoning, planning, and execution, into unmanned aerial vehicle (UAV) swarms opens new operational possibilities and brings the vision of the Internet of Drones closer to reality. However, infrastructure constraints, dynamic environments, and the computational demands of multi-agent coordination limit real-world deployment in high-risk scenarios such as wildfires and disaster response. This paper investigates the integration of LLM-based agentic AI and edge computing to realize scalable and resilient autonomy in UAV swarms. We first discuss three architectures for supporting UAV swarms - standalone, edge-enabled, and edge-cloud hybrid deployment - each optimized for varying autonomy and connectivity levels. Then, a use case for wildfire search and rescue (SAR) is designed to demonstrate the efficiency of the edge-enabled architecture, enabling high SAR coverage, reduced mission completion times, and a higher level of autonomy compared to traditional approaches. Finally, we highlight open challenges in integrating LLMs and edge computing for mission-critical UAV-swarm applications.
Abstract (translated)
基于大型语言模型（LLMs）自主推理、规划和执行能力的代理型人工智能集成到无人飞行器（UAV）集群中，开创了新的操作可能性，并使无人机互联网的概念更加接近现实。然而，基础设施限制、动态环境以及多智能体协调计算需求限制了在野火和灾害响应等高风险场景中的实际部署。本文探讨了将基于LLM的代理型人工智能与边缘计算结合，以实现UAV集群中可扩展且具有弹性的自主性。首先，我们讨论支持UAV集群的三种架构——独立式、边缘增强式以及边缘-云混合部署，每种架构都针对不同级别的自主性和连接性进行了优化。然后，设计了一个用于野火搜索和救援（SAR）的实际案例，以展示边缘增强式架构的有效性，该架构能够提高SAR覆盖范围，减少任务完成时间，并比传统方法实现更高的自主水平。最后，我们强调了在关键任务型UAV集群应用中整合LLMs与边缘计算所面临的开放挑战。
URL
https://arxiv.org/abs/2601.14437
PDF
-
GIC-DLC: Differentiable Logic Circuits for Hardware-Friendly Grayscale Image Compression
2026-01-20 16:29:23Till Aczel, David F. Jenny, Simon B\"uhrer, Andreas Plesner, Antonio Di Maio, Roger WattenhoferAbstract
Neural image codecs achieve higher compression ratios than traditional hand-crafted methods such as PNG or JPEG-XL, but often incur substantial computational overhead, limiting their deployment on energy-constrained devices such as smartphones, cameras, and drones. We propose Grayscale Image Compression with Differentiable Logic Circuits (GIC-DLC), a hardware-aware codec where we train lookup tables to combine the flexibility of neural networks with the efficiency of Boolean operations. Experiments on grayscale benchmark datasets show that GIC-DLC outperforms traditional codecs in compression efficiency while allowing substantial reductions in energy consumption and latency. These results demonstrate that learned compression can be hardware-friendly, offering a promising direction for low-power image compression on edge devices.
Abstract (translated)
神经图像编解码器比传统的手工设计方法（如PNG或JPEG-XL）实现了更高的压缩比率，但通常会带来显著的计算开销，限制了它们在智能手机、相机和无人机等能量受限设备上的部署。我们提出了灰度图像压缩与可微逻辑电路 (GIC-DLC)，这是一种硬件感知编解码器，我们在其中训练查找表以结合神经网络的灵活性和布尔运算的效率。在灰度基准数据集上的实验表明，GIC-DLC 在压缩效率上超越了传统编解码器，并且能够在很大程度上减少能量消耗和延迟。这些结果证明了学习到的压缩技术可以是硬件友好的，为边缘设备上的低功耗图像压缩提供了一个有前景的方向。
URL
https://arxiv.org/abs/2601.14130
PDF
-
CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments
2026-01-20 13:44:02Haotian Xu, Yue Hu, Zhengqiu Zhu, Chen Gao, Ziyou Wang, Junreng Rao, Wenhao Lu, Weishi Li, Quanjun Yin, Yong LiAbstract
Cross-view spatial reasoning is essential for embodied AI, underpinning spatial understanding, mental simulation and planning in complex environments. Existing benchmarks primarily emphasize indoor or street settings, overlooking the unique challenges of open-ended urban spaces characterized by rich semantics, complex geometries, and view variations. To address this, we introduce CityCube, a systematic benchmark designed to probe cross-view reasoning capabilities of current VLMs in urban settings. CityCube integrates four viewpoint dynamics to mimic camera movements and spans a wide spectrum of perspectives from multiple platforms, e.g., vehicles, drones and satellites. For a comprehensive assessment, it features 5,022 meticulously annotated multi-view QA pairs categorized into five cognitive dimensions and three spatial relation expressions. A comprehensive evaluation of 33 VLMs reveals a significant performance disparity with humans: even large-scale models struggle to exceed 54.1% accuracy, remaining 34.2% below human performance. By contrast, small-scale fine-tuned VLMs achieve over 60.0% accuracy, highlighting the necessity of our benchmark. Further analyses indicate the task correlations and fundamental cognitive disparity between VLMs and human-like reasoning.
Abstract (translated)
跨视角空间推理对于具身人工智能（Embodied AI）至关重要，它是复杂环境中空间理解、心理模拟和规划的基础。现有基准测试主要集中在室内或街道场景上，忽略了开放型城市空间的独特挑战，这些地方以其丰富的语义信息、复杂的几何形状和视图变化为特点。为了应对这一问题，我们引入了CityCube，这是一个系统性基准测试工具，旨在评估当前视觉语言模型（VLMs）在城市环境中跨视角推理能力的表现。 CityCube整合了四种视角动态来模拟相机移动，并跨越多个平台的广泛视角范围进行操作，如车辆、无人机和卫星。为了进行全面评价，它包括5,022对细致标注的多视图问答配对，这些配对被分类为五种认知维度和三种空间关系表达方式。 33个VLM模型的综合评估显示了与人类表现之间的显著差异：即使大规模模型也难以超过54.1%的准确率，比人类的表现低34.2个百分点。相比之下，小型精细调优的VLM可以达到60.0%以上的准确率，这凸显了我们基准测试工具的重要性。 进一步分析表明，任务之间存在相关性以及视觉语言模型与人类类似推理之间的基本认知差异。
URL
https://arxiv.org/abs/2601.14339
PDF
-
DroneVLA: VLA based Aerial Manipulation
2026-01-20 10:08:00Fawad Mehboob, Monijesu James, Amir Habel, Jeffrin Sam, Miguel Altamirano Cabrera, Dzmitry TsetserukouAbstract
As aerial platforms evolve from passive observers to active manipulators, the challenge shifts toward designing intuitive interfaces that allow non-expert users to command these systems naturally. This work introduces a novel concept of autonomous aerial manipulation system capable of interpreting high-level natural language commands to retrieve objects and deliver them to a human user. The system is intended to integrate a MediaPipe based on Grounding DINO and a Vision-Language-Action (VLA) model with a custom-built drone equipped with a 1-DOF gripper and an Intel RealSense RGB-D camera. VLA performs semantic reasoning to interpret the intent of a user prompt and generates a prioritized task queue for grasping of relevant objects in the scene. Grounding DINO and dynamic A* planning algorithm are used to navigate and safely relocate the object. To ensure safe and natural interaction during the handover phase, the system employs a human-centric controller driven by MediaPipe. This module provides real-time human pose estimation, allowing the drone to employ visual servoing to maintain a stable, distinct position directly in front of the user, facilitating a comfortable handover. We demonstrate the system's efficacy through real-world experiments for localization and navigation, which resulted in a 0.164m, 0.070m, and 0.084m of max, mean euclidean, and root-mean squared errors, respectively, highlighting the feasibility of VLA for aerial manipulation operations.
Abstract (translated)
随着空中平台从被动观察者发展为主动操作者，设计直观的用户界面成为挑战，以使非专家用户能够自然地控制这些系统。这项工作提出了一种新颖的概念——自主空中操控系统，该系统能理解高级别的自然语言指令来检索物体并将其交付给人类用户。这个系统整合了基于Grounding DINO和MediaPipe的Vision-Language-Action (VLA)模型以及一个配备了1-DOF夹具和Intel RealSense RGB-D相机的定制无人机。 VLA模型执行语义推理，以理解用户的意图，并生成优先级任务队列来抓取场景中的相关物体。使用Grounding DINO和动态A*规划算法进行导航并安全地重新定位对象。为了确保在交接阶段的安全且自然互动，该系统采用了由MediaPipe驱动的人体中心控制器模块。这个模块提供实时人体姿态估计功能，使无人机能够采用视觉伺服技术来保持一个稳定、明显的姿态，直接位于用户前方，从而实现舒适的物体交付。 通过实际实验验证了系统的有效性，在定位和导航方面取得了最大误差为0.164米，平均欧氏距离误差为0.070米，均方根误差为0.084米的成绩。这些结果突显了VLA模型在空中操控操作中的可行性。
URL
https://arxiv.org/abs/2601.13809
PDF
-
HoverAI: An Embodied Aerial Agent for Natural Human-Drone Interaction
2026-01-20 09:59:49Yuhua Jin, Nikita Kuzmin, Georgii Demianchuk, Mariya Lezina, Fawad Mehboob, Issatay Tokmurziyev, Miguel Altamirano Cabrera, Muhammad Ahsan Mustafa, Dzmitry TsetserukouAbstract
Drones operating in human-occupied spaces suffer from insufficient communication mechanisms that create uncertainty about their intentions. We present HoverAI, an embodied aerial agent that integrates drone mobility, infrastructure-independent visual projection, and real-time conversational AI into a unified platform. Equipped with a MEMS laser projector, onboard semi-rigid screen, and RGB camera, HoverAI perceives users through vision and voice, responding via lip-synced avatars that adapt appearance to user demographics. The system employs a multimodal pipeline combining VAD, ASR (Whisper), LLM-based intent classification, RAG for dialogue, face analysis for personalization, and voice synthesis (XTTS v2). Evaluation demonstrates high accuracy in command recognition (F1: 0.90), demographic estimation (gender F1: 0.89, age MAE: 5.14 years), and speech transcription (WER: 0.181). By uniting aerial robotics with adaptive conversational AI and self-contained visual output, HoverAI introduces a new class of spatially-aware, socially responsive embodied agents for applications in guidance, assistance, and human-centered interaction.
Abstract (translated)
在有人类活动的空间中运行的无人机由于通信机制不足，导致人们对它们的意图存在不确定性。我们提出了HoverAI，这是一种集成了无人机移动性、基础设施独立的视觉投影和实时对话人工智能于一体的综合平台的实体空中代理。通过配备MEMS激光投影仪、机载半刚性屏幕以及RGB相机，HoverAI能够通过视觉和语音感知用户，并利用与其人口统计特征相适应的唇同步化身做出回应。 该系统采用了一种多模态管道，结合了语音活动检测（VAD）、自动语音识别（使用Whisper模型）、基于大型语言模型的意图分类、RAG对话系统、面部分析以实现个性化以及语音合成（XTTS v2）等技术。评估结果显示，在命令识别中表现出高准确性（F1: 0.90），人口统计估计准确率在性别上为0.89，年龄误差平均绝对值仅为5.14岁，并且在语音转录中的错误率为0.181。 通过将空中机器人技术与适应性对话人工智能及自包含视觉输出相结合，HoverAI引入了一种新型的具有空间感知和社会响应能力的实体代理。这种新技术适用于指导、协助和以人为中心的人机互动应用中。
URL
https://arxiv.org/abs/2601.13801
PDF
-
MVGD-Net: A Novel Motion-aware Video Glass Surface Detection Network
2026-01-20 08:19:17Yiwei Lu, Hao Huang, Tao YanAbstract
Glass surface ubiquitous in both daily life and professional environments presents a potential threat to vision-based systems, such as robot and drone navigation. To solve this challenge, most recent studies have shown significant interest in Video Glass Surface Detection (VGSD). We observe that objects in the reflection (or transmission) layer appear farther from the glass surfaces. Consequently, in video motion scenarios, the notable reflected (or transmitted) objects on the glass surface move slower than objects in non-glass regions within the same spatial plane, and this motion inconsistency can effectively reveal the presence of glass surfaces. Based on this observation, we propose a novel network, named MVGD-Net, for detecting glass surfaces in videos by leveraging motion inconsistency cues. Our MVGD-Net features three novel modules: the Cross-scale Multimodal Fusion Module (CMFM) that integrates extracted spatial features and estimated optical flow maps, the History Guided Attention Module (HGAM) and Temporal Cross Attention Module (TCAM), both of which further enhances temporal features. A Temporal-Spatial Decoder (TSD) is also introduced to fuse the spatial and temporal features for generating the glass region mask. Furthermore, for learning our network, we also propose a large-scale dataset, which comprises 312 diverse glass scenarios with a total of 19,268 frames. Extensive experiments demonstrate that our MVGD-Net outperforms relevant state-of-the-art methods.
Abstract (translated)
玻璃表面在日常生活中和专业环境中普遍存在，这给基于视觉的系统（如机器人和无人机导航）带来了潜在威胁。为了解决这一挑战，最近的研究对视频玻璃面检测（VGSD）表现出了浓厚的兴趣。我们观察到，在反射层或透射层中的物体似乎距离玻璃更远。因此，在视频运动场景中，相较于同一平面内的非玻璃区域里的对象，玻璃表面上的显著反射（或透射）物体移动得较慢，这种运动不一致性可以有效地揭示玻璃表面的存在。 基于这一观察，我们提出了一种名为MVGD-Net的新网络，用于通过利用运动不一致线索来检测视频中的玻璃面。我们的MVGD-Net具有三个新颖模块：跨尺度多模态融合模块（CMFM），该模块整合了提取的空间特征和估计的光流图；历史引导注意模块（HGAM）以及时间交叉注意模块（TCAM），这两个模块进一步增强了时序特征。此外，还引入了一个时空解码器（TSD），用于融合空间和时间特征以生成玻璃区域掩模。 为了训练我们的网络，我们还提出了一套大规模的数据集，其中包括312种多样的玻璃场景，总计有19,268帧。广泛的实验表明，与相关最先进的方法相比，我们的MVGD-Net在性能上取得了优越的结果。
URL
https://arxiv.org/abs/2601.13715
PDF
-
Exploiting Light To Enhance The Endurance and Navigation of Lighter-Than-Air Micro-Drones
2026-01-19 14:27:15Harry Huang, Talia Xu, Marco Z\'u\~niga ZamalloaAbstract
Micro-Unmanned Aerial Vehicles (UAVs) are rapidly expanding into tasks from inventory to environmental sensing, yet their short endurance and unreliable navigation in GPS-denied spaces limit deployment. Lighter-Than-Air (LTA) drones offer an energy-efficient alternative: they use a helium envelope to provide buoyancy, which enables near-zero-power drain during hovering and much longer operation. LTAs are promising, but their design is complex, and they lack integrated solutions to enable sustained autonomous operations and navigation with simple, low-infrastructure. We propose a compact, self-sustaining LTA drone that uses light for both energy harvesting and navigation. Our contributions are threefold: (i) a high-fidelity simulation framework to analyze LTA aerodynamics and select a stable, efficient configuration; (ii) a framework to integrate solar cells on the envelope to provide net-positive energy; and (iii) a point-and-go navigation system with three light-seeking algorithms operating on a single light beacon. Our LTA-analysis, together with the integrated solar panels, not only saves energy while flying, but also enables sustainable operation: providing 1 minute of flying time for every 4 minutes of energy harvesting, under illuminations of 80klux. We also demonstrate robust single-beacon navigation towards a light source that can be up to 7m away, in indoor and outdoor environments, even with moderate winds. The resulting system indicates a plausible path toward persistent, autonomous operation for indoor and outdoor monitoring. More broadly, this work provides a practical pathway for translating the promise of LTA drones into a persistent, self-sustaining aerial system.
Abstract (translated)
微型无人驾驶飞机（UAV）正迅速扩展到从库存管理到环境监测等各种任务中，但它们的短续航时间和在无GPS区域导航不稳定的特性限制了其部署。轻于空气（LTA）无人机提供了一种节能的替代方案：利用氦气囊提供浮力，使悬停时接近零能量消耗，并且飞行时间更长。尽管LTAs具有潜力，但设计复杂且缺乏集成解决方案以实现持续自主操作和低基础设施依赖导航。 我们提出一种紧凑、自给自足的LTA无人机，它使用光来同时进行能源收集和导航。我们的贡献包括三个方面：（i）一个高保真模拟框架用于分析LTA气动力学并选择稳定高效的配置；（ii）一个将太阳能电池集成到气囊上的框架以提供净正能量；以及（iii）一种基于单光源的“指哪去哪”导航系统，它使用三种寻光算法。 通过我们的LTA分析和集成太阳能板不仅节省了飞行时的能量消耗，还实现了可持续操作：在80klux照明条件下，每4分钟充电可提供1分钟的飞行时间。此外，我们展示了即使在有中等风速的情况下，室内和室外环境中的单光源导航也具有鲁棒性，并且光源距离可达7米。 该系统为实现持续自主的操作（用于室内和室外监测）指明了一条可能的道路。更广泛地说，这项工作提供了一种将LTA无人机的潜力转化为持久自给飞行系统的实用路径。
URL
https://arxiv.org/abs/2601.13088
PDF
-
AirHunt: Bridging VLM Semantics and Continuous Planning for Efficient Aerial Object Navigation
2026-01-19 05:50:03Xuecheng Chen, Zongzhuo Liu, Jianfa Ma, Bang Du, Tiantian Zhang, Xueqian Wang, Boyu ZhouAbstract
Recent advances in large Vision-Language Models (VLMs) have provided rich semantic understanding that empowers drones to search for open-set objects via natural language instructions. However, prior systems struggle to integrate VLMs into practical aerial systems due to orders-of-magnitude frequency mismatch between VLM inference and real-time planning, as well as VLMs' limited 3D scene understanding. They also lack a unified mechanism to balance semantic guidance with motion efficiency in large-scale environments. To address these challenges, we present AirHunt, an aerial object navigation system that efficiently locates open-set objects with zero-shot generalization in outdoor environments by seamlessly fusing VLM semantic reasoning with continuous path planning. AirHunt features a dual-pathway asynchronous architecture that establishes a synergistic interface between VLM reasoning and path planning, enabling continuous flight with adaptive semantic guidance that evolves through motion. Moreover, we propose an active dual-task reasoning module that exploits geometric and semantic redundancy to enable selective VLM querying, and a semantic-geometric coherent planning module that dynamically reconciles semantic priorities and motion efficiency in a unified framework, enabling seamless adaptation to environmental heterogeneity. We evaluate AirHunt across diverse object navigation tasks and environments, demonstrating a higher success rate with lower navigation error and reduced flight time compared to state-of-the-art methods. Real-world experiments further validate AirHunt's practical capability in complex and challenging environments. Code and dataset will be made publicly available before publication.
Abstract (translated)
近期在大型视觉-语言模型（VLMs）方面的进展为无人机通过自然语言指令搜索开放集对象提供了丰富的语义理解能力。然而，先前的系统由于VLM推断与实时规划之间频率不匹配以及VLM对3D场景理解有限的问题，在将VLM集成到实际空中系统中时遇到了困难。此外，这些系统缺乏一种统一机制来在大规模环境中平衡语义引导和运动效率。为了解决这些问题，我们提出了AirHunt，这是一种高效的空中物体导航系统，它通过无缝融合VLM的语义推理与连续路径规划，在户外环境中实现开放集对象的零样本泛化定位。 AirHunt具有双路径异步架构，建立了VLM推理与路径规划之间的协同接口，允许根据运动不断演变的自适应语义指导进行持续飞行。此外，我们提出了一种主动双重任务推理模块，利用几何和语义冗余实现选择性的VLM查询，并提出了一个语义-几何协调规划模块，在统一框架内动态平衡语义优先级与运动效率，使系统能够无缝地适应环境异质性。 我们在各种物体导航任务和环境中评估了AirHunt，证明它比最先进的方法具有更高的成功率、更低的导航误差以及更短的飞行时间。实际世界中的实验进一步验证了AirHunt在复杂且具挑战性的环境下的实用能力。发布前，代码和数据集将公开提供。 通过这种方式，AirHunt不仅克服了现有系统的局限性，还为无人机的应用开辟了新的可能，特别是在搜索与救援、农业监测等领域。
URL
https://arxiv.org/abs/2601.12742
PDF
-
UAVGENT: A Language-Guided Distributed Control Framework
2026-01-18 19:48:57Ziyi Zhang, Xiyu Deng, Guannan Qu, Yorie NakahiraAbstract
We study language-in-the-loop control for multi-drone systems that execute evolving, high-level missions while retaining formal robustness guarantees at the physical layer. We propose a three-layer architecture in which (i) a human operator issues natural-language instructions, (ii) an LLM-based supervisor periodically interprets, verifies, and corrects the commanded task in the context of the latest state and target estimates, and (iii) a distributed inner-loop controller tracks the resulting reference using only local relative information. We derive a theoretical guarantee that characterizes tracking performance under bounded disturbances and piecewise-smooth references with discrete jumps induced by LLM updates. Overall, our results illustrate how centralized language-based task reasoning can be combined with distributed feedback control to achieve complex behaviors with provable robustness and stability.
Abstract (translated)
URL
https://arxiv.org/abs/2602.13212
PDF
-
BirdsEye-RU: A Dataset For Detecting Faces from Overhead Images
2026-01-18 18:36:03Md. Ahanaf Arif Khan, Ariful Islam, Sangeeta Biswas, Md. Iqbal Aziz Khan, Subrata Pramanik, Sanjoy Kumar Chakrabarty, Bimal Kumar PramanikAbstract
Detecting faces in overhead images remains a significant challenge due to extreme scale variations and environmental clutter. To address this, we created the BirdsEye-RU dataset, a comprehensive collection of 2,978 images containing over eight thousand annotated faces. This dataset is specifically designed to capture small and distant faces across diverse environments, containing both drone images and smartphone-captured images from high altitude. We present a detailed description of the BirdsEye-RU dataset in this paper. We made our dataset freely available to the public, and it can be accessed at this https URL.
Abstract (translated)
在顶视图图像中检测人脸仍然是一个重大挑战，原因在于极端的尺度变化和环境杂乱。为了解决这一问题，我们创建了BirdsEye-RU数据集，这是一个包含2978张图片的数据集，其中标注有超过八千个面部。该数据集特别设计用于捕捉不同环境中微小且遥远的人脸，包括无人机拍摄的图像以及从高处用智能手机捕获的照片。本文详细介绍了BirdsEye-RU数据集。我们已将此数据集免费提供给公众使用，并可通过以下链接访问：[请在此插入实际URL]。
URL
https://arxiv.org/abs/2601.12533
PDF
-
Video Individual Counting and Tracking from Moving Drones: A Benchmark and Methods
2026-01-18 17:17:31Yaowu Fan, Jia Wan, Tao Han, Andy J. Ma, Antoni B. ChanAbstract
Counting and tracking dense crowds in large-scale scenes is highly challenging, yet existing methods mainly rely on datasets captured by fixed cameras, which provide limited spatial coverage and are inadequate for large-scale dense crowd analysis. To address this limitation, we propose a flexible solution using moving drones to capture videos and perform video-level crowd counting and tracking of unique pedestrians across entire scenes. We introduce MovingDroneCrowd++, the largest video-level dataset for dense crowd counting and tracking captured by moving drones, covering diverse and complex conditions with varying flight altitudes, camera angles, and illumination. Existing methods fail to achieve satisfactory performance on this dataset. To this end, we propose GD3A (Global Density Map Decomposition via Descriptor Association), a density map-based video individual counting method that avoids explicit localization. GD3A establishes pixel-level correspondences between pedestrian descriptors across consecutive frames via optimal transport with an adaptive dustbin score, enabling the decomposition of global density maps into shared, inflow, and outflow components. Building on this framework, we further introduce DVTrack, which converts descriptor-level matching into instance-level associations through a descriptor voting mechanism for pedestrian tracking. Experimental results show that our methods significantly outperform existing approaches under dense crowds and complex motion, reducing counting error by 47.4 percent and improving tracking performance by 39.2 percent.
Abstract (translated)
在大规模场景中计数和追踪密集人群是一项极具挑战性的任务，但现有的方法主要依赖于固定摄像头捕获的数据集，这些数据集提供的空间覆盖范围有限，并不足以进行大规模密集人群的分析。为了克服这一限制，我们提出了一种灵活的解决方案，即使用移动无人机来捕捉视频并执行整个场景内的视频级人群计数和独特的行人人身追踪。为此，我们引入了MovingDroneCrowd++，这是一个由移动无人机捕获的最大规模视频级数据集，用于密集人群计数和跟踪，涵盖了各种复杂条件，包括变化的飞行高度、相机角度以及光照条件。现有方法在该数据集中未能达到满意的性能表现。 为了解决这个问题，我们提出了GD3A（通过描述符关联进行全局密度图分解），这是一种基于密度图的视频个体计数方法，避免了显式的定位步骤。GD3A利用最优传输和自适应尘盒得分，在连续帧之间建立行人的描述符像素级对应关系，从而将全局密度图分解为共享、流入以及流出成分。 在此框架基础上，我们进一步引入了DVTrack，通过描述符投票机制将描述符级别的匹配转换为实例级别的关联，以实现行人追踪。实验结果表明，我们的方法在密集人群和复杂运动条件下显著优于现有方法，在计数误差上减少了47.4%，并提高了39.2%的跟踪性能。
URL
https://arxiv.org/abs/2601.12500
PDF
-
Towards Airborne Object Detection: A Deep Learning Analysis
2026-01-17 04:47:47Prosenjit Chatterjee, ANK ZamanAbstract
The rapid proliferation of airborne platforms, including commercial aircraft, drones, and UAVs, has intensified the need for real-time, automated threat assessment systems. Current approaches depend heavily on manual monitoring, resulting in limited scalability and operational inefficiencies. This work introduces a dual-task model based on EfficientNetB4 capable of performing airborne object classification and threat-level prediction simultaneously. To address the scarcity of clean, balanced training data, we constructed the AODTA Dataset by aggregating and refining multiple public sources. We benchmarked our approach on both the AVD Dataset and the newly developed AODTA Dataset and further compared performance against a ResNet-50 baseline, which consistently underperformed EfficientNetB4. Our EfficientNetB4 model achieved 96% accuracy in object classification and 90% accuracy in threat-level prediction, underscoring its promise for applications in surveillance, defense, and airspace management. Although the title references detection, this study focuses specifically on classification and threat-level inference using pre-localized airborne object images provided by existing datasets.
Abstract (translated)
随着空中平台（包括商用飞机、无人机和无人飞行器）的迅速增多，实时自动威胁评估系统的需求变得更加迫切。目前的方法主要依赖于人工监控，这导致了可扩展性有限以及运营效率低下。本文介绍了一种基于EfficientNetB4的双任务模型，该模型能够同时执行空中物体分类与威胁等级预测。 为了解决干净、平衡训练数据不足的问题，我们通过整合和优化多个公共来源构建了AODTA数据集。我们在AVD数据集以及新开发的AODTA数据集上对我们的方法进行了基准测试，并且将其性能与ResNet-50基线模型进行了比较，后者在所有情况下均不如EfficientNetB4表现良好。我们的EfficientNetB4模型实现了96%的目标物体分类准确率和90%的威胁等级预测准确率，这表明它具有应用于监控、防御以及空域管理等领域的巨大潜力。 尽管标题提到的是检测问题，本研究主要关注利用现有数据集中提供的预定位空中物体图像进行分类与威胁等级推断。
URL
https://arxiv.org/abs/2601.11907
PDF
-
Adaptive Monitoring of Stochastic Fire Front Processes via Information-seeking Predictive Control
2026-01-16 12:21:27Savvas Papaioannou, Panayiotis Kolios, Christos G. Panayiotou, Marios M. PolycarpouAbstract
We consider the problem of adaptively monitoring a wildfire front using a mobile agent (e.g., a drone), whose trajectory determines where sensor data is collected and thus influences the accuracy of fire propagation estimation. This is a challenging problem, as the stochastic nature of wildfire evolution requires the seamless integration of sensing, estimation, and control, often treated separately in existing methods. State-of-the-art methods either impose linear-Gaussian assumptions to establish optimality or rely on approximations and heuristics, often without providing explicit performance guarantees. To address these limitations, we formulate the fire front monitoring task as a stochastic optimal control problem that integrates sensing, estimation, and control. We derive an optimal recursive Bayesian estimator for a class of stochastic nonlinear elliptical-growth fire front models. Subsequently, we transform the resulting nonlinear stochastic control problem into a finite-horizon Markov decision process and design an information-seeking predictive control law obtained via a lower confidence bound-based adaptive search algorithm with asymptotic convergence to the optimal policy.
Abstract (translated)
我们考虑使用移动代理（例如无人机）自适应监控野火前沿的问题，其轨迹决定了传感器数据的采集地点，并因此影响了火灾蔓延估计的准确性。这是一个具有挑战性的问题，因为野火演化的随机性质要求感测、估计和控制之间的无缝集成，而在现有方法中这些过程通常被单独处理。最先进的方法要么为了建立最优性而假设线性和高斯特性，要么依赖于近似值和启发式方法，往往不提供明确的性能保证。 为了解决这些问题，我们将野火前沿监控任务作为集成了感测、估计和控制的随机最优控制问题进行建模。我们推导出了一类具有随机非线性椭圆增长模型的野火前沿的递归贝叶斯最优估计器。接着，将由此产生的非线性随机控制问题转换为有限时间范围的马尔可夫决策过程，并通过基于下限置信度的自适应搜索算法设计了一个信息导向的预测控制法则，该方法具有渐进收敛于最优策略的特点。
URL
https://arxiv.org/abs/2601.11231
PDF
-
Visual Marker Search for Autonomous Drone Landing in Diverse Urban Environments
2026-01-16 08:24:23Jiaohong Yao, Linfeng Liang, Yao Deng, Xi Zheng, Richard Han, Yuankai QiAbstract
Marker-based landing is widely used in drone delivery and return-to-base systems for its simplicity and reliability. However, most approaches assume idealized landing site visibility and sensor performance, limiting robustness in complex urban settings. We present a simulation-based evaluation suite on the AirSim platform with systematically varied urban layouts, lighting, and weather to replicate realistic operational diversity. Using onboard camera sensors (RGB for marker detection and depth for obstacle avoidance), we benchmark two heuristic coverage patterns and a reinforcement learning-based agent, analyzing how exploration strategy and scene complexity affect success rate, path efficiency, and robustness. Results underscore the need to evaluate marker-based autonomous landing under diverse, sensor-relevant conditions to guide the development of reliable aerial navigation systems.
Abstract (translated)
基于标记的着陆在无人机配送和返航系统中因其简单性和可靠性而被广泛使用。然而，大多数方法假设理想化的着陆点可见度和传感器性能，这限制了其在复杂城市环境中的鲁棒性。我们提出了一种基于AirSim平台的仿真评估套件，该套件通过系统地改变城市布局、光照条件和天气情况来模拟实际操作中的多样性。使用机载摄像头传感器（RGB用于标记检测和深度信息用于障碍物规避），我们对两种启发式覆盖模式和一种基于强化学习的代理进行了基准测试，并分析了探索策略与场景复杂度如何影响成功率、路径效率以及鲁棒性。研究结果强调，需要在多变且与传感器相关的条件下评估基于标记的自主着陆系统，以指导可靠空中导航系统的开发。
URL
https://arxiv.org/abs/2601.11078
PDF
-
Zeros can be Informative: Masked Binary U-Net for Image Segmentation on Tensor Cores
2026-01-15 21:22:16Chunshu Wu, Ruibing Song, Sushant Kondguli, Tong Geng, Ang LiAbstract
Real-time image segmentation is a key enabler for AR/VR, robotics, drones, and autonomous systems, where tight accuracy, latency, and energy budgets must be met on resource-constrained edge devices. While U-Net offers a favorable balance of accuracy and efficiency compared to large transformer-based models, achieving real-time performance on high-resolution input remains challenging due to compute, memory, and power limits. Extreme quantization, particularly binary networks, is appealing for its hardware-friendly operations. However, two obstacles limit practicality: (1) severe accuracy degradation, and (2) a lack of end-to-end implementations that deliver efficiency on general-purpose GPUs. We make two empirical observations that guide our design. (1) An explicit zero state is essential: training with zero masking to binary U-Net weights yields noticeable sparsity. (2) Quantization sensitivity is uniform across layers. Motivated by these findings, we introduce Masked Binary U-Net (MBU-Net), obtained through a cost-aware masking strategy that prioritizes masking where it yields the highest accuracy-per-cost, reconciling accuracy with near-binary efficiency. To realize these gains in practice, we develop a GPU execution framework that maps MBU-Net to Tensor Cores via a subtractive bit-encoding scheme, efficiently implementing masked binary weights with binary activations. This design leverages native binary Tensor Core BMMA instructions, enabling high throughput and energy savings on widely available GPUs. Across 3 segmentation benchmarks, MBU-Net attains near full-precision accuracy (3% average drop) while delivering 2.04x speedup and 3.54x energy reductions over a 16-bit floating point U-Net.
Abstract (translated)
实时图像分割是增强现实（AR）/虚拟现实（VR）、机器人技术、无人机和自主系统的关键推动力，这些应用需要在计算资源受限的边缘设备上实现高精度、低延迟和节能。虽然U-Net架构相比基于大型变换器模型而言，在准确性与效率之间提供了一个有利的平衡，但在处理高分辨率输入时要实现实时性能仍然面临挑战，因为这会受到计算能力、内存以及能耗限制的影响。极端量化（特别是二值网络）因其硬件友好的操作而具有吸引力。然而，有两个障碍阻碍了其实用性：一是准确性的严重下降；二是缺乏在通用GPU上实现高效能的端到端实施方案。 基于两个经验观察结果，我们设计出Masked Binary U-Net (MBU-Net)架构以克服上述挑战： 1. 显式的零态是必不可少的：使用零掩码训练二值U-Net权重能够显著提高稀疏性。 2. 量化敏感度在各层间均匀分布。 受到这些发现的启发，我们通过一种成本感知掩蔽策略引入了MBU-Net架构，这种策略优先选择那些能提供最高准确性和效率比的位置进行掩码处理，从而实现了准确性与近似二值化效率之间的协调。为了实现这一设计的实际效益，我们开发了一个GPU执行框架，该框架通过减法位编码方案将MBU-Net映射到Tensor Core上，并高效地实施了带掩蔽的二进制权重和激活函数。这种设计利用了原生的二进制Tensor Core BMMA指令，从而在广泛可用的GPU上实现了高吞吐量和节能。 在三个分割基准测试中，MBU-Net实现了接近全精度（平均下降3%）的同时，相较于16位浮点数U-Net分别提高了2.04倍的速度和3.54倍的能量效率。
URL
https://arxiv.org/abs/2601.11660
PDF
-
DeepUrban: Interaction-Aware Trajectory Prediction and Planning for Automated Driving by Aerial Imagery
2026-01-15 16:18:42Constantin Selzer, Fabian B. FlohrAbstract
The efficacy of autonomous driving systems hinges critically on robust prediction and planning capabilities. However, current benchmarks are impeded by a notable scarcity of scenarios featuring dense traffic, which is essential for understanding and modeling complex interactions among road users. To address this gap, we collaborated with our industrial partner, DeepScenario, to develop DeepUrban-a new drone dataset designed to enhance trajectory prediction and planning benchmarks focusing on dense urban settings. DeepUrban provides a rich collection of 3D traffic objects, extracted from high-resolution images captured over urban intersections at approximately 100 meters altitude. The dataset is further enriched with comprehensive map and scene information to support advanced modeling and simulation tasks. We evaluate state-of-the-art (SOTA) prediction and planning methods, and conducted experiments on generalization capabilities. Our findings demonstrate that adding DeepUrban to nuScenes can boost the accuracy of vehicle predictions and planning, achieving improvements up to 44.1 % / 44.3% on the ADE / FDE metrics. Website: this https URL
Abstract (translated)
自动驾驶系统的有效性在很大程度上取决于强大的预测和规划能力。然而，当前的基准测试受到缺乏密集交通场景数据的限制，而这些场景对于理解并建模道路使用者之间的复杂交互至关重要。为解决这一问题，我们与我们的工业合作伙伴DeepScenario合作，开发了DeepUrban——一个新的无人机数据集，旨在增强在密集城市环境下的轨迹预测和规划基准测试。DeepUrban提供了一套丰富的3D交通对象集合，这些对象是从高空约100米处拍摄的城市交叉口高分辨率图像中提取出来的。该数据集还通过详细的地图和场景信息得到了进一步的丰富，以支持高级建模和仿真任务。 我们评估了最先进的预测和规划方法，并进行了泛化能力实验。研究结果表明，在nuScenes基础上加入DeepUrban可以显著提高车辆预测和规划的准确性，ADE/FDE指标分别提高了44.1% / 44.3%。网站：[此处插入实际链接]
URL
https://arxiv.org/abs/2601.10554
PDF
-
Online parameter estimation for the Crazyflie quadcopter through an EM algorithm
Abstract
Drones are becoming more and more popular nowadays. They are small in size, low in cost, and reliable in operation. They contain a variety of sensors and can perform a variety of flight tasks, reaching places that are difficult or inaccessible for humans. Earthquakes damage a lot of infrastructure, making it impossible for rescuers to reach some areas. But drones can help. Many amateur and professional photographers like to use drones for aerial photography. Drones play a non-negligible role in agriculture and transportation too. Drones can be used to spray pesticides, and they can also transport supplies. A quadcopter is a four-rotor drone and has been studied in this paper. In this paper, random noise is added to the quadcopter system and its effects on the drone system are studied. An extended Kalman filter has been used to estimate the state based on noisy observations from the sensor. Based on a SDE system, a linear quadratic Gaussian controller has been implemented. The expectation maximization algorithm has been applied for parameter estimation of the quadcopter. The results of offline parameter estimation and online parameter estimation are presented. The results show that the online parameter estimation has a slightly larger range of convergence values than the offline parameter estimation.
Abstract (translated)
URL
https://arxiv.org/abs/2601.17009
PDF
-
Hybrid guided variational autoencoder for visual place recognition
2026-01-14 07:33:53Ni Wang, Zihan You, Emre Neftci, Thorben SchoepeAbstract
Autonomous agents such as cars, robots and drones need to precisely localize themselves in diverse environments, including in GPS-denied indoor environments. One approach for precise localization is visual place recognition (VPR), which estimates the place of an image based on previously seen places. State-of-the-art VPR models require high amounts of memory, making them unwieldy for mobile deployment, while more compact models lack robustness and generalization capabilities. This work overcomes these limitations for robotics using a combination of event-based vision sensors and an event-based novel guided variational autoencoder (VAE). The encoder part of our model is based on a spiking neural network model which is compatible with power-efficient low latency neuromorphic hardware. The VAE successfully disentangles the visual features of 16 distinct places in our new indoor VPR dataset with a classification performance comparable to other state-of-the-art approaches while, showing robust performance also under various illumination conditions. When tested with novel visual inputs from unknown scenes, our model can distinguish between these places, which demonstrates a high generalization capability by learning the essential features of location. Our compact and robust guided VAE with generalization capabilities poses a promising model for visual place recognition that can significantly enhance mobile robot navigation in known and unknown indoor environments.
Abstract (translated)
自主代理（如汽车、机器人和无人机）需要在各种环境中精确定位自己，包括无GPS信号的室内环境。视觉位置识别（VPR）是一种精确定位的方法，它基于之前看到的位置来估计图像的位置。最先进的VPR模型需要大量内存，使其难以部署到移动设备中，而更紧凑的模型则缺乏鲁棒性和泛化能力。 这项研究通过结合事件驱动视觉传感器和一种新的事件驱动引导变分自编码器（VAE）的方法，解决了机器人技术中的这些限制。我们模型的编码部分基于脉冲神经网络模型，该模型与节能低延迟神经形态硬件兼容。我们的VAE在新的室内VPR数据集上成功分离了16个不同位置的视觉特征，并且其分类性能与其他最先进的方法相当，在各种光照条件下也表现出良好的鲁棒性。 当使用未知场景的新视觉输入进行测试时，我们的模型能够区分这些位置，这表明它具有很高的泛化能力，可以学习到地点的关键特征。我们紧凑、稳健并具备泛化的引导VAE为视觉位置识别提供了一个有前景的模型，能够在已知和未知的室内环境中显著增强移动机器人的导航能力。
URL
https://arxiv.org/abs/2601.09248
PDF
-
Hybrid Distillation with CoT Guidance for Edge-Drone Control Code Generation
2026-01-13 10:31:09Yizhan Feng, Hichem Snoussi, Yuhang Wang, Jing Teng, Abel Cherouat, Tian WangAbstract
With large language models demonstrating significant potential in code generation tasks, their application to onboard control of resource-constrained Unmanned Aerial Vehicles has emerged as an important research direction. However, a notable contradiction exists between the high resource consumption of large models and the real-time, lightweight requirements of UAV platforms. This paper proposes an integrated approach that combines knowledge distillation, chain-of-thought guidance, and supervised fine-tuning for UAV multi-SDK control tasks, aiming to efficiently transfer complex reasoning and code generation capabilities to smaller models. Firstly, a high-quality dataset covering various mainstream UAV SDKs is constructed, featuring instruction-code-reasoning chains, and incorporates counterfactual negative samples for data augmentation, guiding the model to learn the end-to-end logic from instruction parsing to code generation. Secondly, leveraging DeepSeek-Coder-V2-Lite quantized via QLoRA as the teacher model, and based on a hybrid black-box and white-box distillation strategy, high-quality chain-of-thought soft labels are generated. These are combined with a weighted cross-entropy loss using hard labels to transfer complex reasoning capabilities to the smaller student model. Finally, through prompt tuning engineering optimized for the UAV control scenario, the model performance on core tasks such as SDK type recognition and function call matching is enhanced. Experimental results indicate that the distilled lightweight model maintains high code generation accuracy while achieving significant improvements in deployment and inference efficiency, effectively demonstrating the feasibility and superiority of our approach in achieving precise and lightweight intelligent control for UAVs
Abstract (translated)
随着大型语言模型在代码生成任务中展现出巨大潜力，它们被应用于资源受限的无人驾驶飞行器（UAV）的在线控制，成为了一个重要的研究方向。然而，大型模型高资源消耗与无人机平台实时、轻量级需求之间存在明显的矛盾。本文提出了一种综合方法，结合知识蒸馏、链式思维引导和监督微调技术，以实现将复杂推理和代码生成能力高效转移至小型模型的目标。具体而言： 首先，构建了一个高质量的数据集，涵盖了各种主流UAV软件开发工具包（SDK），该数据集中包含了指令-代码-推理链条，并且引入了反事实负样本进行数据增强，引导模型从指令解析到代码生成学习端到端逻辑。 其次，利用通过QLoRA量化后的DeepSeek-Coder-V2-Lite作为教师模型，并基于混合黑盒和白盒蒸馏策略生成高质量的链式思维软标签。这些软标签与硬标签结合使用加权交叉熵损失函数，以将复杂推理能力转移至较小的学生模型。 最后，通过针对UAV控制场景优化提示调优工程，增强了模型在SDK类型识别、功能调用匹配等核心任务上的性能表现。 实验结果表明，蒸馏后的轻量级模型能够保持高代码生成准确性的同时，在部署和推断效率上实现了显著改进。这有效地展示了我们的方法在实现精确且轻量级的UAV智能控制方面的可行性和优越性。
URL
https://arxiv.org/abs/2601.08412
PDF
-
Edge-Optimized Multimodal Learning for UAV Video Understanding via BLIP-2
2026-01-13 10:26:10Yizhan Feng, Hichem Snoussi, Jing Teng, Jian Liu, Yuyang Wang, Abel Cherouat, Tian WangAbstract
The demand for real-time visual understanding and interaction in complex scenarios is increasingly critical for unmanned aerial vehicles. However, a significant challenge arises from the contradiction between the high computational cost of large Vision language models and the limited computing resources available on UAV edge devices. To address this challenge, this paper proposes a lightweight multimodal task platform based on BLIP-2, integrated with YOLO-World and YOLOv8-Seg models. This integration extends the multi-task capabilities of BLIP-2 for UAV applications with minimal adaptation and without requiring task-specific fine-tuning on drone data. Firstly, the deep integration of BLIP-2 with YOLO models enables it to leverage the precise perceptual results of YOLO for fundamental tasks like object detection and instance segmentation, thereby facilitating deeper visual-attention understanding and reasoning. Secondly, a content-aware key frame sampling mechanism based on K-Means clustering is designed, which incorporates intelligent frame selection and temporal feature concatenation. This equips the lightweight BLIP-2 architecture with the capability to handle video-level interactive tasks effectively. Thirdly, a unified prompt optimization scheme for multi-task adaptation is implemented. This scheme strategically injects structured event logs from the YOLO models as contextual information into BLIP-2's input. Combined with output constraints designed to filter out technical details, this approach effectively guides the model to generate accurate and contextually relevant outputs for various tasks.
Abstract (translated)
在复杂场景中，无人驾驶航空器对实时视觉理解和交互的需求日益重要。然而，大型视觉语言模型的高计算成本与无人机边缘设备有限的计算资源之间的矛盾成为了一个重大挑战。为解决这一问题，本文提出了一种基于BLIP-2的轻量级多模态任务平台，并集成了YOLO-World和YOLOv8-Seg模型。这种集成使BLIP-2在无人飞行器应用中能够扩展其多任务功能，而无需对无人机数据进行特定任务的微调。 首先，BLIP-2与YOLO模型的深度融合使其能利用YOLO提供的精确感知结果来执行对象检测和实例分割等基本任务，从而促进更深层次的视觉注意力理解和推理。其次，设计了一种基于K-Means聚类的内容感知关键帧采样机制，该机制结合了智能帧选择和时间特征串联功能，使轻量级BLIP-2架构能够有效处理视频级别的交互式任务。最后，实施了一个统一的多任务适应性提示优化方案。此方案战略性地将YOLO模型中的结构化事件日志作为上下文信息注入到BLIP-2的输入中，并结合输出限制以过滤掉技术细节，从而有效地指导模型生成准确且与上下文相关的多种任务结果。
URL
https://arxiv.org/abs/2601.08408
PDF
