Drone
-
HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Traffic
2026-02-03 12:12:47Yu-Hsiang Chen, Wei-Jer Chang, Christian Kotulla, Thomas Keutgens, Steffen Runde, Tobias Moers, Christoph Klas, Wei Zhan, Masayoshi Tomizuka, Yi-Ting ChenAbstract
We present HetroD, a dataset and benchmark for developing autonomous driving systems in heterogeneous environments. HetroD targets the critical challenge of navi- gating real-world heterogeneous traffic dominated by vulner- able road users (VRUs), including pedestrians, cyclists, and motorcyclists that interact with vehicles. These mixed agent types exhibit complex behaviors such as hook turns, lane splitting, and informal right-of-way negotiation. Such behaviors pose significant challenges for autonomous vehicles but remain underrepresented in existing datasets focused on structured, lane-disciplined traffic. To bridge the gap, we collect a large- scale drone-based dataset to provide a holistic observation of traffic scenes with centimeter-accurate annotations, HD maps, and traffic signal states. We further develop a modular toolkit for extracting per-agent scenarios to support downstream task development. In total, the dataset comprises over 65.4k high- fidelity agent trajectories, 70% of which are from VRUs. HetroD supports modeling of VRU behaviors in dense, het- erogeneous traffic and provides standardized benchmarks for forecasting, planning, and simulation tasks. Evaluation results reveal that state-of-the-art prediction and planning models struggle with the challenges presented by our dataset: they fail to predict lateral VRU movements, cannot handle unstructured maneuvers, and exhibit limited performance in dense and multi-agent scenarios, highlighting the need for more robust approaches to heterogeneous traffic. See our project page for more examples: this https URL
Abstract (translated)
我们介绍了HetroD，这是一个用于开发异质环境中自主驾驶系统的数据集和基准。HetroD旨在解决在由易受伤害的道路使用者（VRU）主导的真实世界异质交通环境中的导航挑战，包括行人、骑自行车者和摩托车手与车辆的交互行为。这些混合代理类型表现出复杂的动态行为，如钩形转弯、车道分隔以及非正式的优先权协商等。这类行为对自动驾驶汽车构成了重大挑战，但在现有的专注于结构化且遵守车道规则交通的数据集中却严重不足。 为了弥补这一差距，我们收集了一个大规模基于无人机的数据集，为异质交通场景提供了全面观察，并带有厘米级精度的标注、高清地图和交通信号状态信息。此外，我们还开发了一套模块化的工具包来提取单个代理的情景以支持下游任务的发展。总体而言，该数据集中包含了超过65,400条高保真度代理轨迹，其中70%来源于VRU。HetroD支持对密集和异质交通中VRU行为的建模，并为预测、规划及模拟任务提供了标准化基准。 评估结果显示，最先进的预测和规划模型在处理我们数据集提出的挑战时存在困难：它们无法准确预测侧向VRU运动轨迹，不能应对非结构化的操作动作，且在高密度与多代理场景中的性能表现有限。这突显了开发更稳健的方法来处理异质交通的必要性。 更多信息请参见我们的项目页面: [此URL](this https URL)
URL
https://arxiv.org/abs/2602.03447
PDF
-
LEVIO: Lightweight Embedded Visual Inertial Odometry for Resource-Constrained Devices
2026-02-03 09:20:57Jonas K\"uhne, Christian Vogt, Michele Magno, Luca BeniniAbstract
Accurate, infrastructure-less sensor systems for motion tracking are essential for mobile robotics and augmented reality (AR) applications. The most popular state-of-the-art visual-inertial odometry (VIO) systems, however, are too computationally demanding for resource-constrained hardware, such as micro-drones and smart glasses. This work presents LEVIO, a fully featured VIO pipeline optimized for ultra-low-power compute platforms, allowing six-degrees-of-freedom (DoF) real-time sensing. LEVIO incorporates established VIO components such as Oriented FAST and Rotated BRIEF (ORB) feature tracking and bundle adjustment, while emphasizing a computationally efficient architecture with parallelization and low memory usage to suit embedded microcontrollers and low-power systems-on-chip (SoCs). The paper proposes and details the algorithmic design choices and the hardware-software co-optimization approach, and presents real-time performance on resource-constrained hardware. LEVIO is validated on a parallel-processing ultra-low-power RISC-V SoC, achieving 20 FPS while consuming less than 100 mW, and benchmarked against public VIO datasets, offering a compelling balance between efficiency and accuracy. To facilitate reproducibility and adoption, the complete implementation is released as open-source.
Abstract (translated)
准确且无需基础设施的传感器系统对于移动机器人和增强现实（AR）应用中的运动跟踪至关重要。然而，最先进的视觉惯性里程计（VIO）系统对资源受限硬件（如微型无人机和智能眼镜）来说计算需求过高。这项工作提出了LEVIO，这是一种针对超低功耗计算平台优化的全功能VIO流水线，能够实现六自由度（DoF）实时感知。 LEVIO融合了成熟的VIO组件，例如定向FAST与旋转BRIEF（ORB）特征跟踪和束调整技术，并且特别强调采用高效的并行化架构和低内存使用量设计，以适应嵌入式微控制器和低功耗系统级芯片（SoC）。论文详细介绍了算法的设计选择及硬件-软件协同优化方法，并展示了在资源受限硬件上的实时性能。 LEVIO在一个支持并行处理的超低能耗RISC-V SoC上进行了测试，在每秒20帧的速度下仅消耗不到100毫瓦，同时在公开的VIO数据集上与现有系统进行了基准测试。结果表明，LEVIO在效率和准确性之间提供了良好的平衡。 为了促进重现性和采用，LEVIO的完整实现已作为开源代码发布。
URL
https://arxiv.org/abs/2602.03294
PDF
-
Vision-only UAV State Estimation for Fast Flights Without External Localization Systems: A2RL Drone Racing Finalist Approach
2026-02-02 09:32:19Filip Nov\'ak, Mat\v{e}j Petrl\'ik, Matej Novosad, Parakh M. Gupta, Robert P\v{e}ni\v{c}ka, Martin SaskaAbstract
Fast flights with aggressive maneuvers in cluttered GNSS-denied environments require fast, reliable, and accurate UAV state estimation. In this paper, we present an approach for onboard state estimation of a high-speed UAV using a monocular RGB camera and an IMU. Our approach fuses data from Visual-Inertial Odometry (VIO), an onboard landmark-based camera measurement system, and an IMU to produce an accurate state estimate. Using onboard measurement data, we estimate and compensate for VIO drift through a novel mathematical drift model. State-of-the-art approaches often rely on more complex hardware (e.g., stereo cameras or rangefinders) and use uncorrected drifting VIO velocities, orientation, and angular rates, leading to errors during fast maneuvers. In contrast, our method corrects all VIO states (position, orientation, linear and angular velocity), resulting in accurate state estimation even during rapid and dynamic motion. Our approach was thoroughly validated through 1600 simulations and numerous real-world experiments. Furthermore, we applied the proposed method in the A2RL Drone Racing Challenge 2025, where our team advanced to the final four out of 210 teams and earned a medal.
Abstract (translated)
URL
https://arxiv.org/abs/2602.01860
PDF
-
SkySim: A ROS2-based Simulation Environment for Natural Language Control of Drone Swarms using Large Language Models
2026-02-01 13:34:34Aditya Shibu, Marah Saleh, Mohamed Al-Musleh, Nidhal AbdulazizAbstract
Unmanned Aerial Vehicle (UAV) swarms offer versatile applications in logistics, agriculture, and surveillance, yet controlling them requires expert knowledge for safety and feasibility. Traditional static methods limit adaptability, while Large Language Models (LLMs) enable natural language control but generate unsafe trajectories due to lacking physical grounding. This paper introduces SkySim, a ROS2-based simulation framework in Gazebo that decouples LLM high-level planning from low-level safety enforcement. Using Gemini 3.5 Pro, SkySim translates user commands (e.g., "Form a circle") into spatial waypoints, informed by real-time drone states. An Artificial Potential Field (APF) safety filter applies minimal adjustments for collision avoidance, kinematic limits, and geo-fencing, ensuring feasible execution at 20 Hz. Experiments with swarms of 3, 10, and 30 Crazyflie drones validate spatial reasoning accuracy (100% across tested geometric primitives), real-time collision prevention, and scalability. SkySim empowers non-experts to iteratively refine behaviors, bridging AI cognition with robotic safety for dynamic environments. Future work targets hardware integration.
Abstract (translated)
URL
https://arxiv.org/abs/2602.01226
PDF
-
Stealthy Coverage Control for Human-enabled Real-Time 3D Reconstruction
2026-01-31 02:41:13Reiji Terunuma, Yuta Nakamura, Takuma Abe, Takeshi HatanakaAbstract
In this paper, we propose a novel semi-autonomous image sampling strategy, called stealthy coverage control, for human-enabled 3D structure reconstruction. The present mission involves a fundamental problem: while the number of images required to accurately reconstruct a 3D model depends on the structural complexity of the target scene to be reconstructed, it is not realistic to assume prior knowledge of the spatially non-uniform structural complexity. We approach this issue by leveraging human flexible reasoning and situational recognition capabilities. Specifically, we design a semi-autonomous system that leaves identification of regions that need more images and navigation of the drones to such regions to a human operator. To this end, we first present a way to reflect the human intention in autonomous coverage control. Subsequently, in order to avoid operational conflicts between manual control and autonomous coverage control, we develop the stealthy coverage control that decouples the drone motion for efficient image sampling from navigation by the human. Simulation studies on a Unity/ROS2-based simulator demonstrate that the present semi-autonomous system outperforms the one without human interventions in the sense of the reconstructed model quality.
Abstract (translated)
URL
https://arxiv.org/abs/2602.00466
PDF
-
FlyAware: Inertia-Aware Aerial Manipulation via Vision-Based Estimation and Post-Grasp Adaptation
2026-01-30 08:02:33Biyu Ye, Na Fan, Zhengping Fan, Weiliang Deng, Hongming Chen, Qifeng Chen, Ximin LyuAbstract
Aerial manipulators (AMs) are gaining increasing attention in automated transportation and emergency services due to their superior dexterity compared to conventional multirotor drones. However, their practical deployment is challenged by the complexity of time-varying inertial parameters, which are highly sensitive to payload variations and manipulator configurations. Inspired by human strategies for interacting with unknown objects, this letter presents a novel onboard framework for robust aerial manipulation. The proposed system integrates a vision-based pre-grasp inertia estimation module with a post-grasp adaptation mechanism, enabling real-time estimation and adaptation of inertial dynamics. For control, we develop an inertia-aware adaptive control strategy based on gain scheduling, and assess its robustness via frequency-domain system identification. Our study provides new insights into post-grasp control for AMs, and real-world experiments validate the effectiveness and feasibility of the proposed framework.
Abstract (translated)
URL
https://arxiv.org/abs/2601.22686
PDF
-
Efficient Spike-driven Transformer for High-performance Drone-View Geo-Localization
2026-01-29 07:10:08Zhongwei Chen, Hai-Jun Rong, Zhao-Xu Yang, Guoqi LiAbstract
Traditional drone-view geo-localization (DVGL) methods based on artificial neural networks (ANNs) have achieved remarkable performance. However, ANNs rely on dense computation, which results in high power consumption. In contrast, spiking neural networks (SNNs), which benefit from spike-driven computation, inherently provide low power consumption. Regrettably, the potential of SNNs for DVGL has yet to be thoroughly investigated. Meanwhile, the inherent sparsity of spike-driven computation for representation learning scenarios also results in loss of critical information and difficulties in learning long-range dependencies when aligning heterogeneous visual data sources. To address these, we propose SpikeViMFormer, the first SNN framework designed for DVGL. In this framework, a lightweight spike-driven transformer backbone is adopted to extract coarse-grained features. To mitigate the loss of critical information, the spike-driven selective attention (SSA) block is designed, which uses a spike-driven gating mechanism to achieve selective feature enhancement and highlight discriminative regions. Furthermore, a spike-driven hybrid state space (SHS) block is introduced to learn long-range dependencies using a hybrid state space. Moreover, only the backbone is utilized during the inference stage to reduce computational cost. To ensure backbone effectiveness, a novel hierarchical re-ranking alignment learning (HRAL) strategy is proposed. It refines features via neighborhood re-ranking and maintains cross-batch consistency to directly optimize the backbone. Experimental results demonstrate that SpikeViMFormer outperforms state-of-the-art SNNs. Compared with advanced ANNs, it also achieves competitive this http URL code is available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2512.19365
PDF
-
Mam-App: A Novel Parameter-Efficient Mamba Model for Apple Leaf Disease Classification
2026-01-29 05:59:53Md Nadim Mahamood, Md Imran Hasan, Md Rasheduzzaman, Ausrukona Ray, Md Shafi Ud Doula, Kamrul HasanAbstract
The rapid growth of the global population, alongside exponential technological advancement, has intensified the demand for food production. Meeting this demand depends not only on increasing agricultural yield but also on minimizing food loss caused by crop diseases. Diseases account for a substantial portion of apple production losses, despite apples being among the most widely produced and nutritionally valuable fruits worldwide. Previous studies have employed machine learning techniques for feature extraction and early diagnosis of apple leaf diseases, and more recently, deep learning-based models have shown remarkable performance in disease recognition. However, most state-of-the-art deep learning models are highly parameter-intensive, resulting in increased training and inference time. Although lightweight models are more suitable for user-friendly and resource-constrained applications, they often suffer from performance degradation. To address the trade-off between efficiency and performance, we propose Mam-App, a parameter-efficient Mamba-based model for feature extraction and leaf disease classification. The proposed approach achieves competitive state-of-the-art performance on the PlantVillage Apple Leaf Disease dataset, attaining 99.58% accuracy, 99.30% precision, 99.14% recall, and a 99.22% F1-score, while using only 0.051M parameters. This extremely low parameter count makes the model suitable for deployment on drones, mobile devices, and other low-resource platforms. To demonstrate the robustness and generalizability of the proposed model, we further evaluate it on the PlantVillage Corn Leaf Disease and Potato Leaf Disease datasets. The model achieves 99.48%, 99.20%, 99.34%, and 99.27% accuracy, precision, recall, and F1-score on the corn dataset and 98.46%, 98.91%, 95.39%, and 97.01% on the potato dataset, respectively.
Abstract (translated)
URL
https://arxiv.org/abs/2601.21307
PDF
-
A Methodology for Designing Knowledge-Driven Missions for Robots
2026-01-28 17:39:03Guillermo GP-Lenza, Carmen DR. Pita-Romero, Miguel Fernandez-Cortizas, Pascual CampoyAbstract
This paper presents a comprehensive methodology for implementing knowledge graphs in ROS 2 systems, aiming to enhance the efficiency and intelligence of autonomous robotic missions. The methodology encompasses several key steps: defining initial and target conditions, structuring tasks and subtasks, planning their sequence, representing task-related data in a knowledge graph, and designing the mission using a high-level language. Each step builds on the previous one to ensure a cohesive process from initial setup to final execution. A practical implementation within the Aerostack2 framework is demonstrated through a simulated search and rescue mission in a Gazebo environment, where drones autonomously locate a target. This implementation highlights the effectiveness of the methodology in improving decision-making and mission performance by leveraging knowledge graphs.
Abstract (translated)
URL
https://arxiv.org/abs/2601.20797
PDF
-
Learned split-spectrum metalens for obstruction-free broadband imaging in the visible
2026-01-27 09:38:26Seungwoo Yoon, Dohyun Kang, Eunsue Choi, Sohyun Lee, Seoyeon Kim, Minho Choi, Hyeonsu Heo, Dong-ha Shin, Suha Kwak, Arka Majumdar, Junsuk Rho, Seung-Hwan BaekAbstract
Obstructions such as raindrops, fences, or dust degrade captured images, especially when mechanical cleaning is infeasible. Conventional solutions to obstructions rely on a bulky compound optics array or computational inpainting, which compromise compactness or fidelity. Metalenses composed of subwavelength meta-atoms promise compact imaging, but simultaneous achievement of broadband and obstruction-free imaging remains a challenge, since a metalens that images distant scenes across a broadband spectrum cannot properly defocus near-depth occlusions. Here, we introduce a learned split-spectrum metalens that enables broadband obstruction-free imaging. Our approach divides the spectrum of each RGB channel into pass and stop bands with multi-band spectral filtering and learns the metalens to focus light from far objects through pass bands, while filtering focused near-depth light through stop bands. This optical signal is further enhanced using a neural network. Our learned split-spectrum metalens achieves broadband and obstruction-free imaging with relative PSNR gains of 32.29% and improves object detection and semantic segmentation accuracies with absolute gains of +13.54% mAP, +48.45% IoU, and +20.35% mIoU over a conventional hyperbolic design. This promises robust obstruction-free sensing and vision for space-constrained systems, such as mobile robots, drones, and endoscopes.
Abstract (translated)
URL
https://arxiv.org/abs/2601.19403
PDF
-
Perception-to-Pursuit: Track-Centric Temporal Reasoning for Open-World Drone Detection and Autonomous Chasing
2026-01-27 07:57:29Venkatakrishna Reddy OrugantiAbstract
Autonomous drone pursuit requires not only detecting drones but also predicting their trajectories in a manner that enables kinematically feasible interception. Existing tracking methods optimize for prediction accuracy but ignore pursuit feasibility, resulting in trajectories that are physically impossible to intercept 99.9% of the time. We propose Perception-to-Pursuit (P2P), a track-centric temporal reasoning framework that bridges detection and actionable pursuit planning. Our method represents drone motion as compact 8-dimensional tokens capturing velocity, acceleration, scale, and smoothness, enabling a 12-frame causal transformer to reason about future behavior. We introduce the Intercept Success Rate (ISR) metric to measure pursuit feasibility under realistic interceptor constraints. Evaluated on the Anti-UAV-RGBT dataset with 226 real drone sequences, P2P achieves 28.12 pixel average displacement error and 0.597 ISR, representing a 77% improvement in trajectory prediction and 597x improvement in pursuit feasibility over tracking-only baselines, while maintaining perfect drone classification accuracy (100%). Our work demonstrates that temporal reasoning over motion patterns enables both accurate prediction and actionable pursuit planning.
Abstract (translated)
URL
https://arxiv.org/abs/2601.19318
PDF
-
EFSI-DETR: Efficient Frequency-Semantic Integration for Real-Time Small Object Detection in UAV Imagery
2026-01-26 15:41:37Yu Xia, Chang Liu, Tianqi Xiang, Zhigang TuAbstract
Real-time small object detection in Unmanned Aerial Vehicle (UAV) imagery remains challenging due to limited feature representation and ineffective multi-scale fusion. Existing methods underutilize frequency information and rely on static convolutional operations, which constrain the capacity to obtain rich feature representations and hinder the effective exploitation of deep semantic features. To address these issues, we propose EFSI-DETR, a novel detection framework that integrates efficient semantic feature enhancement with dynamic frequency-spatial guidance. EFSI-DETR comprises two main components: (1) a Dynamic Frequency-Spatial Unified Synergy Network (DyFusNet) that jointly exploits frequency and spatial cues for robust multi-scale feature fusion, (2) an Efficient Semantic Feature Concentrator (ESFC) that enables deep semantic extraction with minimal computational cost. Furthermore, a Fine-grained Feature Retention (FFR) strategy is adopted to incorporate spatially rich shallow features during fusion to preserve fine-grained details, crucial for small object detection in UAV imagery. Extensive experiments on VisDrone and CODrone benchmarks demonstrate that our EFSI-DETR achieves the state-of-the-art performance with real-time efficiency, yielding improvement of \textbf{1.6}\% and \textbf{5.8}\% in AP and AP$_{s}$ on VisDrone, while obtaining \textbf{188} FPS inference speed on a single RTX 4090 GPU.
Abstract (translated)
URL
https://arxiv.org/abs/2601.18597
PDF
-
DiffusionCinema: Text-to-Aerial Cinematography
2026-01-24 11:12:30Valerii Serpiva, Artem Lykov, Jeffrin Sam, Aleksey Fedoseev, Dzmitry TsetserukouAbstract
We propose a novel Unmanned Aerial Vehicles (UAV) assisted creative capture system that leverages diffusion models to interpret high-level natural language prompts and automatically generate optimal flight trajectories for cinematic video recording. Instead of manually piloting the drone, the user simply describes the desired shot (e.g., "orbit around me slowly from the right and reveal the background waterfall"). Our system encodes the prompt along with an initial visual snapshot from the onboard camera, and a diffusion model samples plausible spatio-temporal motion plans that satisfy both the scene geometry and shot semantics. The generated flight trajectory is then executed autonomously by the UAV to record smooth, repeatable video clips that match the prompt. User evaluation using NASA-TLX showed a significantly lower overall workload with our interface (M = 21.6) compared to a traditional remote controller (M = 58.1), demonstrating a substantial reduction in perceived effort. Mental demand (M = 11.5 vs. 60.5) and frustration (M = 14.0 vs. 54.5) were also markedly lower for our system, confirming clear usability advantages in autonomous text-driven flight control. This project demonstrates a new interaction paradigm: text-to-cinema flight, where diffusion models act as the "creative operator" converting story intentions directly into aerial motion.
Abstract (translated)
URL
https://arxiv.org/abs/2601.17412
PDF
-
Boundary and Position Information Mining for Aerial Small Object Detection
2026-01-23 10:15:12Rongxin Huang, Guangfeng Lin, Wenbo Zhou, Zhirong Li, Wenhuan WuAbstract
Unmanned Aerial Vehicle (UAV) applications have become increasingly prevalent in aerial photography and object recognition. However, there are major challenges to accurately capturing small targets in object detection due to the imbalanced scale and the blurred edges. To address these issues, boundary and position information mining (BPIM) framework is proposed for capturing object edge and location cues. The proposed BPIM includes position information guidance (PIG) module for obtaining location information, boundary information guidance (BIG) module for extracting object edge, cross scale fusion (CSF) module for gradually assembling the shallow layer image feature, three feature fusion (TFF) module for progressively combining position and boundary information, and adaptive weight fusion (AWF) module for flexibly merging the deep layer semantic feature. Therefore, BPIM can integrate boundary, position, and scale information in image for small object detection using attention mechanisms and cross-scale feature fusion strategies. Furthermore, BPIM not only improves the discrimination of the contextual feature by adaptive weight fusion with boundary, but also enhances small object perceptions by cross-scale position fusion. On the VisDrone2021, DOTA1.0, and WiderPerson datasets, experimental results show the better performances of BPIM compared to the baseline Yolov5-P2, and obtains the promising performance in the state-of-the-art methods with comparable computation load.
Abstract (translated)
URL
https://arxiv.org/abs/2601.16617
PDF
-
A Beacon Based Solution for Autonomous UUVs GNSS-Denied Stealthy Navigation
2026-01-22 09:34:56Alexandre Albore, Humbert Fiorino, Damien PellierAbstract
Autonomous Unmanned Underwater Vehicles (UUVs) enable military and civilian covert operations in coastal areas without relying on support vessels or Global Navigation Satellite Systems (GNSS). Such operations are critical when surface access is not possible and stealthy navigation is required in restricted environments such as protected zones or dangerous areas under access ban. GNSS denied navigation is then essential to maintaining concealment as surfacing could expose UUVs to detection. To ensure a precise fleet positioning a constellation of beacons deployed by aerial or surface drones establish a synthetic landmark network that will guide the fleet of UUVs along an optimized path from the continental shelf to the goal on the shore. These beacons either submerged or floating emit acoustic signals for UUV localisation and navigation. A hierarchical planner generates an adaptive route for the drones executing primitive actions while continuously monitoring and replanning as needed to maintain trajectory accuracy.
Abstract (translated)
自主无人水下航行器（UUVs）能够在沿海区域执行军事和民用秘密行动，无需依赖支援舰船或全球导航卫星系统（GNSS）。这种操作在表面无法进入并且需要隐蔽导航的受限环境中尤为重要，例如保护区域或禁止进入的危险地区。在这种情况下，没有GNSS信号的自主导航对于保持隐藏至关重要，因为浮出水面可能会使UUV暴露于探测风险中。 为了确保精确的舰队定位，通过无人机（空中或水面）部署的一系列信标建立了一个合成地标网络，该网络将引导UUV舰队从大陆架到岸边目标沿优化路径航行。这些信标无论是沉入水下的还是漂浮在水面上的，都会发出声波信号用于UUV的定位和导航。 一个分层规划器生成了一条适应性路线供无人机执行基本动作，并且会不断监控并根据需要重新规划路线以保持航迹精度。
URL
https://arxiv.org/abs/2601.15802
PDF
-
AION: Aerial Indoor Object-Goal Navigation Using Dual-Policy Reinforcement Learning
2026-01-22 03:35:34Zichen Yan, Yuchen Hou, Shenao Wang, Yichao Gao, Rui Huang, Lin ZhaoAbstract
Object-Goal Navigation (ObjectNav) requires an agent to autonomously explore an unknown environment and navigate toward target objects specified by a semantic label. While prior work has primarily studied zero-shot ObjectNav under 2D locomotion, extending it to aerial platforms with 3D locomotion capability remains underexplored. Aerial robots offer superior maneuverability and search efficiency, but they also introduce new challenges in spatial perception, dynamic control, and safety assurance. In this paper, we propose AION for vision-based aerial ObjectNav without relying on external localization or global maps. AION is an end-to-end dual-policy reinforcement learning (RL) framework that decouples exploration and goal-reaching behaviors into two specialized policies. We evaluate AION on the AI2-THOR benchmark and further assess its real-time performance in IsaacSim using high-fidelity drone models. Experimental results show that AION achieves superior performance across comprehensive evaluation metrics in exploration, navigation efficiency, and safety. The video can be found at this https URL.
Abstract (translated)
物体目标导航（ObjectNav）要求代理自主探索未知环境，并根据语义标签向特定的目标对象移动。尽管先前的工作主要研究了在二维运动情况下零样本的ObjectNav问题，但将其扩展到具有三维运动能力的空中平台仍然是一个未充分探讨的研究领域。空中机器人提供了更好的机动性和搜索效率，但也引入了空间感知、动态控制和安全保证等方面的新挑战。在这篇论文中，我们提出了AION，这是一种基于视觉的无人飞行器（UAV）物体导航框架，无需依赖外部定位或全局地图。AION是一个端到端的双策略强化学习（RL）框架，它将探索行为与目标达到行为分解为两个专门化的策略。我们在AI2-THOR基准上评估了AION，并进一步在IsaacSim中使用高保真无人机模型对其实时性能进行了测试。实验结果表明，AION在探索、导航效率和安全性等综合评价指标方面取得了卓越的性能。相关视频可以在此URL找到（原文中的链接未提供，您可以访问原始论文或提供的URL以获取视频）。
URL
https://arxiv.org/abs/2601.15614
PDF
-
A Universal Large Language Model -- Drone Command and Control Interface
2026-01-21 21:37:54Javier N. Ramos-Silva, Peter J. BurkeAbstract
The use of artificial intelligence (AI) for drone control can have a transformative impact on drone capabilities, especially when real world information can be integrated with drone sensing, command, and control, part of a growing field of physical AI. Large language models (LLMs) can be advantageous if trained at scale on general knowledge, but especially and in particular when the training data includes information such as detailed map geography topology of the entire planet, as well as the ability to access real time situational data such as weather. However, challenges remain in the interface between drones and LLMs in general, with each application requiring a tedious, labor intensive effort to connect the LLM trained knowledge to drone command and control. Here, we solve that problem, using an interface strategy that is LLM agnostic and drone agnostic, providing the first universal, versatile, comprehensive and easy to use drone control interface. We do this using the new model context protocol (MCP) standard, an open standard that provides a universal way for AI systems to access external data, tools, and services. We develop and deploy a cloud based Linux machine hosting an MCP server that supports the Mavlink protocol, an ubiquitous drone control language used almost universally by millions of drones including Ardupilot and PX4 this http URL demonstrate flight control of a real unmanned aerial vehicle. In further testing, we demonstrate extensive flight planning and control capability in a simulated drone, integrated with a Google Maps MCP server for up to date, real time navigation information. This demonstrates a universal approach to integration of LLMs with drone command and control, a paradigm that leverages and exploits virtually all of modern AI industry with drone technology in an easy to use interface that translates natural language to drone control.
Abstract (translated)
使用人工智能（AI）进行无人机控制可以在很大程度上改变无人机的能力，特别是在将现实世界的信息与无人机的感知、指令和控制系统相结合时，这已成为物理AI这一新兴领域的一部分。大规模语言模型（LLMs），如果经过全面的知识训练，并且训练数据包括整个地球的地图地理学拓扑信息以及实时环境数据如天气情况，会特别有用。 然而，在无人机和LLM之间接口方面仍然存在挑战，每个应用都需要费时、劳力密集的过程来将训练好的知识连接到无人机的指令与控制。在这里，我们解决了这个问题，采用了一种对LLM和无人机都无特定依赖的界面策略，提供了一个通用的、灵活的、全面且易于使用的无人机控制系统接口。为此，我们使用了新的模型上下文协议（Model Context Protocol, MCP）标准，这是一个开放的标准，为AI系统提供了访问外部数据、工具和服务的统一方式。 我们在云端部署了一台基于Linux的操作服务器，该服务器支持Mavlink协议——一种几乎被全球数百万无人机使用的通用无人机控制语言。我们通过此接口展示了一个真实无人驾驶飞行器的飞行控制演示。在进一步测试中，我们还在模拟无人机上展示了广泛的飞行规划和控制系统能力，并与Google Maps MCP服务器集成以获得最新的实时导航信息。 这证明了将LLMs与无人机指令和控制系统进行集成的一种通用方法，这种方法利用并挖掘现代AI产业几乎所有的资源来支持无人机技术，同时提供了一种易于使用的界面，能够将自然语言转换为无人机控制。
URL
https://arxiv.org/abs/2601.15486
PDF
-
MonoRace: Winning Champion-Level Drone Racing with Robust Monocular AI
2026-01-21 17:53:29Stavrow A. Bahnam, Robin Ferede, Till M. Blaha, Anton E. Lang, Erin Lucassen, Quentin Missinne, Aderik E. C. Verraest, Christophe De Wagter, Guido C. H. E. de CroonAbstract
Autonomous drone racing represents a major frontier in robotics research. It requires an Artificial Intelligence (AI) that can run on board light-weight flying robots under tight resource and time constraints, while pushing the physical system to its limits. The state of the art in this area consists of a system with a stereo camera and an inertial measurement unit (IMU) that beat human drone racing champions in a controlled indoor environment. Here, we present MonoRace: an onboard drone racing approach that uses a monocular, rolling-shutter camera and IMU that generalizes to a competition environment without any external motion tracking system. The approach features robust state estimation that combines neural-network-based gate segmentation with a drone model. Moreover, it includes an offline optimization procedure that leverages the known geometry of gates to refine any state estimation parameter. This offline optimization is based purely on onboard flight data and is important for fine-tuning the vital external camera calibration parameters. Furthermore, the guidance and control are performed by a neural network that foregoes inner loop controllers by directly sending motor commands. This small network runs on the flight controller at 500Hz. The proposed approach won the 2025 Abu Dhabi Autonomous Drone Racing Competition (A2RL), outperforming all competing AI teams and three human world champion pilots in a direct knockout tournament. It set a new milestone in autonomous drone racing research, reaching speeds up to 100 km/h on the competition track and successfully coping with problems such as camera interference and IMU saturation.
Abstract (translated)
自主无人机竞速代表了机器人研究的一个重要前沿。它需要能够在资源和时间受限的条件下运行于轻量级飞行机器人上的人工智能，同时将物理系统推向极限。该领域的最新技术包括一个配备了立体摄像头和惯性测量单元（IMU）的系统，在受控室内环境中击败了人类无人机竞速冠军。在这里，我们介绍了MonoRace：一种使用单目滚动快门相机和IMU、适用于无外部运动跟踪系统的竞争环境中的机载无人机竞速方法。这种方法的特点是结合神经网络栅栏分割与无人机模型的稳健状态估计，并且包括一个离线优化过程，该过程利用已知的栅栏几何形状来微调任何状态估算参数。此离线优化完全基于飞行数据，对于精细调整重要的外部相机校准参数至关重要。 此外，引导和控制由神经网络执行，该网络通过直接发送电机命令而跳过了内部回路控制器。这个小型网络在飞行控制器上以每秒500赫兹的频率运行。所提出的方法赢得了2025年阿布扎比自主无人机竞速比赛（A2RL），超过了所有参赛的人工智能团队和三名世界冠军飞行员，后者在直接淘汰赛中被击败。它在自主无人机竞速研究方面设立了新的里程碑，在竞赛赛道上达到了高达100公里/小时的速度，并成功应对了诸如相机干扰和IMU饱和等问题。
URL
https://arxiv.org/abs/2601.15222
PDF
-
From Observation to Prediction: LSTM for Vehicle Lane Change Forecasting on Highway On/Off-Ramps
2026-01-21 10:31:03Mohamed Abouras, Catherine M. EliasAbstract
On and off-ramps are understudied road sections even though they introduce a higher level of variation in highway interactions. Predicting vehicles' behavior in these areas can decrease the impact of uncertainty and increase road safety. In this paper, the difference between this Area of Interest (AoI) and a straight highway section is studied. Multi-layered LSTM architecture to train the AoI model with ExiD drone dataset is utilized. In the process, different prediction horizons and different models' workflow are tested. The results show great promise on horizons up to 4 seconds with prediction accuracy starting from about 76% for the AoI and 94% for the general highway scenarios on the maximum horizon.
Abstract (translated)
匝道（上、下坡路段）虽然是高速公路互动中变异性较高的部分，但一直以来却未得到充分的研究。预测这些区域内的车辆行为可以降低不确定性的影响，并提高道路安全。本文研究了这一兴趣区域（AoI）与直线高速公路段之间的差异。利用ExiD无人机数据集和多层LSTM架构来训练AoI模型，在此过程中测试了不同的预测时长和不同模型的工作流程。结果显示，对于长达4秒的预测范围，AoI的预测准确率可达到约76%，而一般高速公路场景在最大预测时长下的预测准确率为94%左右，这表明该方法具有很大的潜力。
URL
https://arxiv.org/abs/2601.14848
PDF
-
SimD3: A Synthetic drone Dataset with Payload and Bird Distractor Modeling for Robust Detection
2026-01-21 07:56:25Ami Pandat, Kanyala Muvva, Punna Rajasekhar, Gopika Vinod, Rohit ShuklaAbstract
Reliable drone detection is challenging due to limited annotated real-world data, large appearance variability, and the presence of visually similar distractors such as birds. To address these challenges, this paper introduces SimD3, a large-scale high-fidelity synthetic dataset designed for robust drone detection in complex aerial environments. Unlike existing synthetic drone datasets, SimD3 explicitly models drones with heterogeneous payloads, incorporates multiple bird species as realistic distractors, and leverages diverse Unreal Engine 5 environments with controlled weather, lighting, and flight trajectories captured using a 360 six-camera rig. Using SimD3, we conduct an extensive experimental evaluation within the YOLOv5 detection framework, including an attention-enhanced variant termed Yolov5m+C3b, where standard bottleneck-based C3 blocks are replaced with C3b modules. Models are evaluated on synthetic data, combined synthetic and real data, and multiple unseen real-world benchmarks to assess robustness and generalization. Experimental results show that SimD3 provides effective supervision for small-object drone detection and that Yolov5m+C3b consistently outperforms the baseline across in-domain and cross-dataset evaluations. These findings highlight the utility of SimD3 for training and benchmarking robust drone detection models under diverse and challenging conditions.
Abstract (translated)
可靠的无人机检测由于缺乏注释的真实世界数据、外观变化大以及视觉上与鸟类等干扰物相似，因而面临挑战。为了解决这些问题，本文介绍了一种名为SimD3的大规模高保真合成数据集，旨在复杂空域环境中实现稳健的无人机检测。不同于现有的合成无人机数据集，SimD3明确地模拟了携带不同类型负载的无人机，并引入多种鸟类作为现实中的干扰物。此外，SimD3利用多样化的Unreal Engine 5环境，这些环境包括通过360度六相机装置捕捉到的不同天气、光照和飞行轨迹。 使用SimD3数据集，我们在YOLOv5检测框架中进行了广泛的实验评估，其中包括一种增强注意力的变体Yolov5m+C3b，在这种变体中，标准瓶颈基的C3块被替换为C3b模块。模型在合成数据、组合了合成和真实世界的数据以及多个未见过的真实世界基准上进行评估，以测试其鲁棒性和泛化能力。实验结果表明，SimD3提供了有效的监督用于检测小型无人机，并且Yolov5m+C3b在领域内（in-domain）和跨数据集（cross-dataset）的评估中始终优于基线模型。 这些发现突显了SimD3在多样化的复杂条件下训练和基准测试稳健的无人机检测模型方面的实用性。
URL
https://arxiv.org/abs/2601.14742
PDF
