Drone
-
Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning
2026-03-09 20:58:43Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario, Lars Lundqvist, Christine H. Diepenbrock, Brian N. Bailey, J. Mason EarlesAbstract
This paper introduces a synthetic benchmark to evaluate the performance of vision language models (VLMs) in generating plant simulation configurations for digital twins. While functional-structural plant models (FSPMs) are useful tools for simulating biophysical processes in agricultural environments, their high complexity and low throughput create bottlenecks for deployment at scale. We propose a novel approach that leverages state-of-the-art open-source VLMs -- Gemma 3 and Qwen3-VL -- to directly generate simulation parameters in JSON format from drone-based remote sensing images. Using a synthetic cowpea plot dataset generated via the Helios 3D procedural plant generation library, we tested five in-context learning methods and evaluated the models across three categories: JSON integrity, geometric evaluations, and biophysical evaluations. Our results show that while VLMs can interpret structural metadata and estimate parameters like plant count and sun azimuth, they often exhibit performance degradation due to contextual bias or rely on dataset means when visual cues are insufficient. Validation on a real-world drone orthophoto dataset and an ablation study using a blind baseline further characterize the models' reasoning capabilities versus their reliance on contextual priors. To the best of our knowledge, this is the first study to utilize VLMs to generate structural JSON configurations for plant simulations, providing a scalable framework for reconstruction 3D plots for digital twin in agriculture.
Abstract (translated)
这篇论文介绍了一个合成基准，用于评估视觉语言模型（VLM）在为数字孪生生成植物仿真配置方面的性能。尽管功能性-结构性植株模型（FSPM）是模拟农业环境中生物物理过程的有用工具，但它们的高度复杂性和低吞吐量限制了其大规模部署的能力。我们提出了一种新颖的方法，利用最先进的开源视觉语言模型——Gemma 3 和 Qwen3-VL，直接从基于无人机的遥感图像生成JSON格式的仿真参数。通过使用Helios 3D程序化植物生成库生成的合成豇豆地块数据集，我们测试了五种上下文学习方法，并在三个类别中评估了这些模型：JSON完整性、几何评价和生物物理评价。我们的研究结果表明，虽然VLM可以解释结构元数据并估计植株数量和太阳方位角等参数，但在视觉线索不足的情况下，它们往往会因为上下文偏差而表现不佳或依赖于数据集平均值。 在真实世界的无人机正射影像数据集上进行验证以及使用盲基线的消融研究进一步刻画了模型推理能力与其对上下文先验依赖之间的关系。据我们所知，这是首次利用VLM生成植物仿真的结构化JSON配置的研究，为农业中数字孪生体3D地块的重建提供了一个可扩展框架。
URL
https://arxiv.org/abs/2603.08930
PDF
-
Adaptive SINDy: Residual Force System Identification Based UAV Disturbance Rejection
2026-03-09 19:26:13Fawad Mehboob, Amir Atef Habel, Roohan Ahmed Khan, Mikhail Derevianchenko, Clement Fortin, Dzmitry TsetserukouAbstract
The stability and control of Unmanned Aerial Vehicles (UAVs) in a turbulent environment is a matter of great concern. Devising a robust control algorithm to reject disturbances is challenging due to the highly nonlinear nature of wind dynamics, and modeling the dynamics using analytical techniques is not straightforward. While traditional techniques using disturbance observers and classical adaptive control have shown some progress, they are mostly limited to relatively non-complex environments. On the other hand, learning based approaches are increasingly being used for modeling of residual forces and disturbance rejection; however, their generalization and interpretability is a factor of concern. To this end, we propose a novel integration of data-driven system identification using Sparse Identification of Non-Linear Dynamics (SINDy) with a Recursive Least Square (RLS) adaptive control to adapt and reject wind disturbances in a turbulent environment. We tested and validated our approach on Gazebo harmonic environment and on real flights with wind speeds of up to 2 m/s from four directions, creating a highly dynamic and turbulent environment. Adaptive SINDy outperformed the baseline PID and INDI controllers on several trajectory tracking error metrics without crashing. A root mean square error (RMSE) of up to 12.2 cm and 17.6 cm, and a mean absolute error (MAE) of 13.7 cm and 10.5 cm were achieved on circular and lemniscate trajectories, respectively. The validation was performed on a very lightweight Crazyflie drone under a highly dynamic environment for complex trajectory tracking.
Abstract (translated)
在湍流环境中，无人驾驶航空器（UAV）的稳定性和控制是一个重要的问题。由于风动力学的高度非线性特性，设计能够拒绝干扰的强大控制算法具有挑战性，并且使用分析技术来建模这些动态过程也不容易。虽然传统的扰动观测器和经典自适应控制系统已经取得了一些进展，但它们主要适用于相对简单的环境。另一方面，基于学习的方法越来越多地被用来模拟残余力和干扰排斥；然而，这些方法的泛化能力和可解释性是一个值得关注的因素。 为此，我们提出了一种新颖的数据驱动系统识别技术与递归最小二乘（RLS）自适应控制相结合的新方案，用于在湍流环境中调整并拒绝风扰动。我们在Gazebo谐波环境以及实际飞行测试中验证了这一方法的有效性，在这些实际飞行测试中，来自四个方向的风速高达2米/秒，从而创建了一个高度动态和湍流的环境。自适应SINDy控制算法在多个轨迹跟踪误差指标上优于基准PID和INDI控制器，并且未发生任何坠机事故。对于圆形和反向八字形（lemniscate）轨迹，我们分别实现了最高为12.2厘米和17.6厘米的均方根误差（RMSE），以及分别为13.7厘米和10.5厘米的平均绝对误差（MAE）。这种验证是在一个非常轻量级的Crazyflie无人机上进行的，在高度动态环境中完成复杂轨迹跟踪任务。
URL
https://arxiv.org/abs/2603.08863
PDF
-
Online Sparse Synthetic Aperture Radar Imaging
2026-03-09 16:35:58Conor Flynn, Radoslav Ivanov, Birsen YaziciAbstract
With modern defense applications increasingly relying on inexpensive, autonomous drones, lies the major challenge of designing computationally and memory-efficient onboard algorithms to fulfill mission objectives. This challenge is particularly significant in Synthetic Aperture Radar (SAR), where large volumes of data must be collected and processed for downstream tasks. We propose an online reconstruction method, the Online Fast Iterative Shrinkage-Thresholding Algorithm (Online FISTA), which incrementally reconstructs a scene with limited data through sparse coding. Rather than requiring storage of all received signal data, the algorithm recursively updates storage matrices for each iteration, greatly reducing memory demands. Online SAR image reconstruction facilitates more complex downstream tasks, such as Automatic Target Recognition (ATR), in an online manner, resulting in a more versatile and integrated framework compared to existing post-collection reconstruction and ATR approaches.
Abstract (translated)
随着现代防御应用越来越多地依赖于廉价、自主的无人机，设计计算和内存效率高的机载算法以实现任务目标成为了一个重大挑战。尤其是在合成孔径雷达（SAR）领域，由于需要收集和处理大量的数据来进行后续任务，这一挑战尤为突出。我们提出了一种在线重建方法——在线快速迭代收缩阈值法（Online FISTA），该方法通过稀疏编码逐步使用有限的数据重建场景。与存储所有接收到的信号数据相比，算法递归地更新每次迭代中的存储矩阵，大大减少了内存需求。在线SAR图像重建使得像自动目标识别（ATR）等更复杂的后续任务能够实时进行，从而形成一个比现有的事后收集重建和ATR方法更为灵活和集成的框架。
URL
https://arxiv.org/abs/2603.08582
PDF
-
Real-Time Drone Detection in Event Cameras via Per-Pixel Frequency Analysis
2026-03-09 13:45:00Michael Bezick, Majid SahinAbstract
Detecting fast-moving objects, such as unmanned aerial vehicle (UAV), from event camera data is challenging due to the sparse, asynchronous nature of the input. Traditional Discrete Fourier Transforms (DFT) are effective at identifying periodic signals, such as spinning rotors, but they assume uniformly sampled data, which event cameras do not provide. We propose a novel per-pixel temporal analysis framework using the Non-uniform Discrete Fourier Transform (NDFT), which we call Drone Detection via Harmonic Fingerprinting (DDHF). Our method uses purely analytical techniques that identify the frequency signature of drone rotors, as characterized by frequency combs in their power spectra, enabling a tunable and generalizable algorithm that achieves accurate real-time localization of UAV. We compare against a YOLO detector under equivalent conditions, demonstrating improvement in accuracy and latency across a difficult array of drone speeds, distances, and scenarios. DDHF achieves an average localization F1 score of 90.89% and average latency of 2.39ms per frame, while YOLO achieves an F1 score of 66.74% and requires 12.40ms per frame. Through utilization of purely analytic techniques, DDHF is quickly tuned on small data, easily interpretable, and achieves competitive accuracies and latencies to deep learning alternatives.
Abstract (translated)
从事件相机数据中检测快速移动的物体（如无人驾驶飞行器(UAV)）是一项挑战，原因是输入数据稀疏且非同步。传统的离散傅立叶变换(DFT)能够有效识别周期信号，例如旋转叶片，但它们假设数据是均匀采样的，而这与事件相机提供的数据不符。我们提出了一种新颖的基于像素的时间分析框架，使用非均匀离散傅里叶变换(NDFT)，并将其命名为通过谐波指纹法进行无人机检测（DDHF）。我们的方法仅采用纯分析技术来识别无人机旋翼的独特频率特征，具体表现为它们功率谱中的频梳。该算法具有可调性和通用性，能够在实时条件下实现UAV的精准定位。 与在同等条件下的YOLO探测器相比，我们在一系列不同的速度、距离和场景中展示了DDHF在准确度和延迟上的改进。DDHF达到了平均90.89%的定位F1分数以及每帧2.39ms的平均延迟，而YOLO则为66.74%的F1分数及每帧12.40ms的处理时间。 通过仅使用纯分析技术，DDHF可以快速在小规模数据上进行调整，并且易于解释。同时，在精度和延迟方面，它与深度学习方法相比具有竞争力。
URL
https://arxiv.org/abs/2603.08386
PDF
-
Synthetic Defect Image Generation for Power Line Insulator Inspection Using Multimodal Large Language Models
2026-03-09 08:06:27Xuesong Wang, Caisheng WangAbstract
Utility companies increasingly rely on drone imagery for post-event and routine inspection, but training accurate defect-type classifiers remains difficult because defect examples are rare and inspection datasets are often limited or proprietary. We address this data-scarcity setting by using an off-the-shelf multimodal large language model (MLLM) as a training-free image generator to synthesize defect images from visual references and text prompts. Our pipeline increases diversity via dual-reference conditioning, improves label fidelity with lightweight human verification and prompt refinement, and filters the resulting synthetic pool using an embedding-based selection rule based on distances to class centroids computed from the real training split. We evaluate on ceramic insulator defect-type classification (shell vs. glaze) using a public dataset with a realistic low training-data regime (104 real training images; 152 validation; 308 test). Augmenting the 10% real training set with embedding-selected synthetic images improves test F1 score (harmonic mean of precision and recall) from 0.615 to 0.739 (20% relative), corresponding to an estimated 4--5x data-efficiency gain, and the gains persist with stronger backbone models and frozen-feature linear-probe baselines. These results suggest a practical, low-barrier path for improving defect recognition when collecting additional real defects is slow or infeasible.
Abstract (translated)
公用事业公司越来越多地依赖无人机图像进行事后的和常规的检查，但是由于缺陷实例稀少且检查数据集往往有限或专有，训练准确的缺陷分类器仍然具有挑战性。我们通过使用现成的多模态大型语言模型（MLLM）作为无需训练的图像生成器来解决这一数据稀缺问题，该方法可以从视觉参考和文本提示中合成缺陷图像。我们的流程通过双参考条件增加多样性、通过轻量级的人工验证和提示改进提高标签准确性，并利用基于嵌入的选择规则过滤出最终的合成池，此选择规则基于计算真实训练集各类质心的距离。 我们在一个具有现实低数据量设定（104张真实的训练图像；152张验证图像；308张测试图像）的公共陶瓷绝缘子缺陷类型分类（壳体 vs. 釉面）的数据集上进行了评估。将10%的真实训练集与基于嵌入选择的合成图像相结合，使测试F1分数（精度和召回率的调和平均数）从0.615提高到了0.739（相对提高了20%），相当于估计出4-5倍的数据效率增益，并且这种改进在使用更强的基础模型和支持冻结特征线性探测基准时依然持续存在。 这些结果表明，在收集更多真实缺陷变得缓慢或不可行的情况下，有一条实用、门槛较低的路径可以提高缺陷识别能力。
URL
https://arxiv.org/abs/2603.08069
PDF
-
Enhancing Cross-View UAV Geolocalization via LVLM-Driven Relational Modeling
2026-03-09 07:57:29Bowen Liu, Pengyue Jia, Wanyu Wang, Derong Xu, Jiawei Cheng, Jiancheng Dong, Xiao Han, Zimo Zhao, Chao Zhang, Bowen Yu, Fangyu Hong, Xiangyu ZhaoAbstract
The primary objective of cross-view UAV geolocalization is to identify the exact spatial coordinates of drone-captured imagery by aligning it with extensive, geo-referenced satellite databases. Current approaches typically extract features independently from each perspective and rely on basic heuristics to compute similarity, thereby failing to explicitly capture the essential interactions between different views. To address this limitation, we introduce a novel, plug-and-play ranking architecture designed to explicitly perform joint relational modeling for improved UAV-to-satellite image matching. By harnessing the capabilities of a Large Vision-Language Model (LVLM), our framework effectively learns the deep visual-semantic correlations linking UAV and satellite imagery. Furthermore, we present a novel relational-aware loss function to optimize the training phase. By employing soft labels, this loss provides fine-grained supervision that avoids overly penalizing near-positive matches, ultimately boosting both the model's discriminative power and training stability. Comprehensive evaluations across various baseline architectures and standard benchmarks reveal that the proposed method substantially boosts the retrieval accuracy of existing models, yielding superior performance even under highly demanding conditions.
Abstract (translated)
跨视角无人机地理定位的主要目标是通过将其与广泛的、地理参考的卫星数据库对齐，来识别由无人机捕获影像的确切空间坐标。当前的方法通常是从每个视角独立提取特征，并依赖于基本启发式方法计算相似度，从而未能明确捕捉不同视图之间的关键交互作用。为了解决这一限制，我们引入了一种新颖的即插即用排名架构，该架构旨在进行联合关系建模，以改善无人机与卫星图像匹配的效果。通过利用大型视觉-语言模型（LVLM）的能力，我们的框架能够有效学习连接无人机和卫星影像之间的深度视觉语义关联。 此外，我们提出了一种新型的关系感知损失函数，用于优化训练阶段。通过使用软标签，这种损失可以提供细粒度的监督，并且避免过度惩罚接近正匹配的情况，从而最终增强模型的判别能力和训练稳定性。在各种基线架构和标准基准测试中的全面评估表明，所提出的方法显著提高了现有模型的检索准确性，在高度苛刻条件下也能表现出色。
URL
https://arxiv.org/abs/2603.08063
PDF
-
Vector Field Augmented Differentiable Policy Learning for Vision-Based Drone Racing
2026-03-09 06:49:41Yang Su, Feng Yu, Yu Hu, Xinze Niu, Linzuo Zhang, Fangyu Sun, Danping ZouAbstract
Autonomous drone racing in complex environments requires agile, high-speed flight while maintaining reliable obstacle avoidance. Differentiable-physics-based policy learning has recently demonstrated high sample efficiency and remarkable performance across various tasks, including agile drone flight and quadruped locomotion. However, applying such methods to drone racing remains difficult, as key objective like gate traversal are inherently hard to express as smooth, differentiable losses. To address these challenges, we propose DiffRacing, a novel vector field-augmented differentiable policy learning framework. DiffRacing integrates differentiable losses and vector fields into the training process to provide continuous and stable gradient signals, balancing obstacle avoidance and high-speed gate traversal. In addition, a differentiable Delta Action Model compensates for dynamics mismatch, enabling efficient sim-to-real transfer without explicit system identification. Extensive simulation and real-world experiments demonstrate that DiffRacing achieves superior sample efficiency, faster convergence, and robust flight performance, thereby demonstrating that vector fields can augment traditional gradient-based policy learning with a task-specific geometric prior.
Abstract (translated)
在复杂环境中进行自主无人机竞速需要高速飞行并保持可靠的障碍物规避。基于可微物理的策略学习方法最近展示了高样本效率和跨多个任务（包括敏捷无人机飞行和四足机器人运动）的出色性能。然而，将这些方法应用于无人机竞赛仍然具有挑战性，因为诸如穿越门等关键目标本质上难以表达为平滑、可微分的损失函数。 为了应对这一挑战，我们提出了一种新的框架——DiffRacing，这是一种基于向量场增强的可微策略学习框架。DiffRacing 将可微分损失和向量场整合到训练过程中，提供连续且稳定的梯度信号，在障碍物规避与高速门穿越之间取得平衡。 此外，一种可微的Delta动作模型补偿了动力学不匹配问题，使得无需显式系统识别即可实现高效的仿真到现实环境转换。 大量的模拟实验和真实世界的测试表明，DiffRacing 在样本效率、更快的收敛速度以及稳健的飞行性能方面表现出色。这证明了向量场能够增强传统的梯度基策略学习，并为特定任务引入几何先验知识。
URL
https://arxiv.org/abs/2603.08019
PDF
-
Reasoning Knowledge-Gap in Drone Planning via LLM-based Active Elicitation
2026-03-08 22:00:32Zeyu Fang, Beomyeol Yu, Cheng Liu, Zeyuan Yang, Rongqian Chen, Yuxin Lin, Mahdi Imani, Tian LanAbstract
Human-AI joint planning in Unmanned Aerial Vehicles (UAVs) typically relies on control handover when facing environmental uncertainties, which is often inefficient and cognitively demanding for non-expert operators. To address this, we propose a novel framework that shifts the collaboration paradigm from control takeover to active information elicitation. We introduce the Minimal Information Neuro-Symbolic Tree (MINT), a reasoning mechanism that explicitly structures knowledge gaps regarding obstacles and goals into a queryable format. By leveraging large language models, our system formulates optimal binary queries to resolve specific ambiguities with minimal human interaction. We demonstrate the efficacy of this approach through a comprehensive workflow integrating a vision-language model for perception, voice interfaces, and a low-level UAV control module in both high-fidelity NVIDIA Isaac simulations and real-world deployments. Experimental results show that our method achieves a significant improvement in the success rate for complex search-and-rescue tasks while significantly reducing the frequency of human interaction compared to exhaustive querying baselines.
Abstract (translated)
在无人飞行器（UAV）中，人机联合规划通常依赖于环境不确定情况下的控制权交接，这种方式对于非专业操作员来说既低效又认知负担重。为了解决这一问题，我们提出了一种新的框架，该框架将合作模式从控制接管转变为积极的信息获取。我们引入了最小信息神经符号树（Minimal Information Neuro-Symbolic Tree, MINT），这是一种推理机制，能够明确地将关于障碍物和目标的知识缺口结构化成可查询的格式。 通过利用大型语言模型，我们的系统可以构建最优二元查询来解决特定模糊性，并且只需最少的人类交互。我们展示了这一方法的有效性，它整合了一个视觉-语言模型用于感知、语音接口以及低级无人机控制模块，在高保真的NVIDIA Isaac模拟器和实际部署中均得以实现。 实验结果表明，与全面查询基线相比，我们的方法在复杂搜索和救援任务的成功率上实现了显著提升，并且大幅减少了人类交互的频率。
URL
https://arxiv.org/abs/2603.07824
PDF
-
Vision-Guided MPPI for Agile Drone Racing: Navigating Arbitrary Gate Poses via Neural Signed Distance Fields
2026-03-07 13:18:07Fangguo Zhao, Hanbing Zhang, Zhouheng Li, Xin Guan, Shuo LiAbstract
Autonomous drone racing requires the tight coupling of perception, planning, and control under extreme agility. However, recent approaches typically rely on precomputed spatial reference trajectories or explicit 6-DoF gate pose estimation, rendering them brittle to spatial perturbations, unmodeled track changes, and sensor noise. Conversely, end-to-end learning policies frequently overfit to specific track layouts and struggle with zero-shot generalization. To address these fundamental limitations, we propose a fully onboard, vision guided optimal control framework that enables reference-free agile flight through arbitrarily placed and oriented gates. Central to our approach is Gate-SDF, a novel, implicitly learned neural signed distance field. Gate-SDF directly processes raw, noisy depth images to predict a continuous spatial field that provides both collision repulsion and active geometric guidance toward the valid traversal area. We seamlessly integrate this representation into a sampling-based Model Predictive Path Integral (MPPI) controller. By fully exploiting GPU parallelism, the framework evaluates these continuous spatial constraints across thousands of simulated trajectory rollouts simultaneously in real time. Furthermore, our formulation inherently maintains spatial consistency, ensuring robust navigation even under severe visual occlusion during aggressive maneuvers. Extensive simulations and real-world experiments demonstrate that the proposed system achieves high-speed agile flight and successfully navigates unseen tracks subject to severe unmodeled gate displacements and orientation perturbations. Videos are available at this https URL
Abstract (translated)
自主无人机竞速要求在极高灵活性条件下，感知、规划和控制的紧密耦合。然而，最近的方法通常依赖于预计算的空间参考轨迹或显式的6自由度门姿态估计，这使它们对空间扰动、未建模赛道变化及传感器噪声非常敏感。相比之下，端到端学习策略往往会过度拟合并难以实现零样本泛化能力。为了解决这些根本性的限制，我们提出了一种完全在机载设备上运行的、由视觉引导的最优控制框架，该框架能够通过任意放置和定向的门进行无参考自由飞行。 我们的方法的核心是Gate-SDF，这是一种新颖的隐式学习神经签名距离场。Gate-SDF可以直接处理原始且含噪的深度图像，并预测一个连续的空间场，提供碰撞排斥以及对有效穿越区域的主动几何引导。我们将这种表示无缝地整合到了基于采样的模型预测路径积分（MPPI）控制器中。通过充分利用GPU并行性，该框架能够在实时状态下同时评估数千个模拟轨迹回放下的连续空间约束条件。此外，我们的方法固有地保持了空间一致性，在进行剧烈机动操作时即使遭遇严重视觉遮挡也能确保稳健导航。 广泛的仿真和现实世界实验表明，所提出的系统能够实现高速灵巧飞行，并成功地在存在严重未建模门位移及姿态扰动的未知赛道上导航。视频可在提供的链接中找到（请将“this https URL”替换为实际的视频网址）。
URL
https://arxiv.org/abs/2603.07199
PDF
-
Small Target Detection Based on Mask-Enhanced Attention Fusion of Visible and Infrared Remote Sensing Images
2026-03-06 22:53:27Qianqian Zhang, Xiaolong Jia, Ahmed M. Abdelmoniem, Li Zhou, Junshe AnAbstract
Targets in remote sensing images are usually small, weakly textured, and easily disturbed by complex backgrounds, challenging high-precision detection with general algorithms. Building on our earlier ESM-YOLO, this work presents ESM-YOLO+ as a lightweight visible infrared fusion network. To enhance detection, ESM-YOLO+ includes two key innovations. (1) A Mask-Enhanced Attention Fusion (MEAF) module fuses features at the pixel level via learnable spatial masks and spatial attention, effectively aligning RGB and infrared features, enhancing small-target representation, and alleviating cross-modal misalignment and scale heterogeneity. (2) Training-time Structural Representation (SR) enhancement provides auxiliary supervision to preserve fine-grained spatial structures during training, boosting feature discriminability without extra inference cost. Extensive experiments on the VEDAI and DroneVehicle datasets validate ESM-YOLO+'s superiority. The model achieves 84.71\% mAP on VEDAI and 74.0\% mAP on DroneVehicle, while greatly reducing model complexity, with 93.6\% fewer parameters and 68.0\% lower GFLOPs than the baseline. These results confirm that ESM-YOLO+ integrates strong performance with practicality for real-time deployment, providing an effective solution for high-performance small-target detection in complex remote sensing scenes.
Abstract (translated)
在遥感图像中，目标通常很小、纹理较弱，并且容易受到复杂背景的干扰，这给一般算法进行高精度检测带来了挑战。在此前的工作 ESM-YOLO 的基础上，我们提出了 ESM-YOLO+，这是一种轻量级的可见光与红外融合网络。为了提升检测效果，ESM-YOLO+ 包含了两项关键创新： 1. **Mask-Enhanced Attention Fusion (MEAF) 模块**：该模块通过可学习的空间掩码和空间注意力机制在像素级别上融合特征，有效地对齐 RGB 和红外特性，增强小目标的表现，并缓解跨模态错配及尺度异质性问题。 2. **训练时间结构表示（SR）增强**：这种辅助监督提供了在训练期间保持精细的时空结构的方法，在不增加推理成本的情况下提高特征判别能力。 在 VEDAI 和 DroneVehicle 数据集上的广泛实验验证了 ESM-YOLO+ 的优越性能。该模型在 VEDAI 上实现了 84.71% mAP，而在 DroneVehicle 上达到了 74.0% mAP，并且极大地减少了模型复杂度，参数量比基线少了93.6%，GFLOPs（十亿次浮点运算）则低了68.0%。这些结果表明 ESM-YOLO+ 结合了强大的性能和实际应用性，可以实现实时部署，为在复杂的遥感场景中进行高性能小目标检测提供了有效的解决方案。
URL
https://arxiv.org/abs/2603.06925
PDF
-
Fly360: Omnidirectional Obstacle Avoidance within Drone View
2026-03-06 18:59:43Xiangkai Zhang, Dizhe Zhang, WenZhuo Cao, Zhaoliang Wan, Yingjie Niu, Lu Qi, Xu Yang, Zhiyong LiuAbstract
Obstacle avoidance in unmanned aerial vehicles (UAVs), as a fundamental capability, has gained increasing attention with the growing focus on spatial intelligence. However, current obstacle-avoidance methods mainly depend on limited field-of-view sensors and are ill-suited for UAV scenarios which require full-spatial awareness when the movement direction differs from the UAV's heading. This limitation motivates us to explore omnidirectional obstacle avoidance for panoramic drones with full-view perception. We first study an under explored problem setting in which a UAV must generate collision-free motion in environments with obstacles from arbitrary directions, and then construct a benchmark that consists of three representative flight tasks. Based on such settings, we propose Fly360, a two-stage perception-decision pipeline with a fixed random-yaw training strategy. At the perception stage, panoramic RGB observations are input and converted into depth maps as a robust intermediate representation. For the policy network, it is lightweight and used to output body-frame velocity commands from depth inputs. Extensive simulation and real-world experiments demonstrate that Fly360 achieves stable omnidirectional obstacle avoidance and outperforms forward-view baselines across all tasks. Our model is available at this https URL
Abstract (translated)
无人机（UAV）中的避障能力作为一项基本技能，随着对空间智能的关注日益增加而受到了更多的重视。然而，目前的避障方法主要依赖于视场有限的传感器，在需要全方位感知的情况下尤其不适合，尤其是在移动方向与无人机航向不一致时。这种局限性促使我们探索适用于全视角无人机的全方位避障问题。 本文首先研究了一个尚待深入探讨的问题设置：即在存在来自任意方向障碍物的环境中生成无碰撞运动的能力，然后构建一个包含三种代表飞行任务的基准测试集。基于这些设定，我们提出了Fly360系统，这是一种两阶段感知-决策管道，并采用了固定随机偏航训练策略。 在感知阶段，全景RGB观察数据作为输入并转换为深度图，以形成一种稳健的中间表示形式。对于策略网络而言，它设计得轻量级且直接从深度图像输入输出体框架的速度命令。 通过广泛的模拟和现实世界实验，Fly360展示出能够实现稳定的全方位避障，并在所有任务中都优于向前视图基准线模型的表现。我们的模型可在指定链接获取（注：原文中的链接未能在此显示，请访问原始文档以获得链接）。
URL
https://arxiv.org/abs/2603.06573
PDF
-
ODD-SEC: Onboard Drone Detection with a Spinning Event Camera
2026-03-06 13:30:40Kuan Dai, Hongxin Zhang, Sheng Zhong, Yi ZhouAbstract
The rapid proliferation of drones requires balancing innovation with regulation. To address security and privacy concerns, techniques for drone detection have attracted significant this http URL solutions, such as frame camera-based systems, offer versatility and energy efficiency under typical conditions but are fundamentally constrained by their operational principles in scenarios involving fast-moving targets or adverse this http URL by biological vision, event cameras asynchronously detect per-pixel brightness changes, offering high dynamic range and microsecond-level responsiveness that make them uniquely suited for drone detection in conditions beyond the reach of conventional frame-based this http URL, the design of most existing event-based solutions assumes a static camera, greatly limiting their applicability to moving carriers--such as quadrupedal robots or unmanned ground vehicles--during field this http URL this paper, we introduce a real-time drone detection system designed for deployment on moving carriers. The system utilizes a spinning event-based camera, providing a 360° horizontal field of view and enabling bearing estimation of detected drones. A key contribution is a novel image-like event representation that operates without motion compensation, coupled with a lightweight neural network architecture for efficient spatiotemporal learning. Implemented on an onboard Jetson Orin NX, the system can operate in real time. Outdoor experimental results validate reliable detection with a mean angular error below 2° under challenging conditions, underscoring its suitability for real-world surveillance applications. We will open-source our complete pipeline to support future research.
Abstract (translated)
无人机的迅速普及要求在创新与监管之间取得平衡。为了应对安全和隐私方面的担忧，无人机检测技术引起了广泛关注。目前的一些解决方案，如基于帧相机的系统，在常规条件下提供灵活性和能源效率，但从根本上受到操作原理的限制，在涉及快速移动目标或不利环境条件的情况下效果不佳。相比之下，生物视觉驱动的事件相机异步地检测像素亮度的变化，提供了高动态范围和微秒级响应速度，使它们特别适合在传统帧基系统难以发挥作用的条件下进行无人机检测。 然而，现有基于事件的大多数解决方案的设计假设摄像机是静态的，极大地限制了其在移动载体（如四足机器人或无人驾驶地面车辆）上的现场应用。在这篇论文中，我们介绍了一种为部署在移动载体上而设计的实时无人机检测系统。该系统使用旋转式的事件相机，提供360°水平视场，并能够估算被检无人机的方向。一个重要贡献是一种新颖的类似图像的事件表示方法，它无需运动补偿即可工作，并结合了一个轻量级神经网络架构以进行高效的时空学习。在车载Jetson Orin NX上实施后，该系统可以在实时环境中运行。 户外实验结果验证了其在挑战性条件下的可靠检测能力，平均角度误差低于2°，表明其适用于现实世界的监控应用。我们将开源我们的完整流程以支持未来的研究。
URL
https://arxiv.org/abs/2603.06265
PDF
-
AMP2026: A Multi-Platform Marine Robotics Dataset for Tracking and Mapping
2026-03-04 16:07:19Edwin Meriaux, Shuo Wen, David Widhalm, Zhizun Wang, Junming Shi, Mariana Sosa Guzm\'an, Kalvik Jakkala, Bennett Carley, Elias Sokolova, Yogesh Girdhar, Monika Roznere, Jason O'Kane, Junaed Sattar, Gregory DudekAbstract
Marine environments present significant challenges for perception and autonomy due to dynamic surfaces, limited visibility, and complex interactions between aerial, surface, and submerged sensing modalities. This paper introduces the Aerial Marine Perception Dataset (AMP2026), a multi-platform marine robotics dataset collected across multiple field deployments designed to support research in two primary areas: multi-view tracking and marine environment mapping. The dataset includes synchronized data from aerial drones, boat-mounted cameras, and submerged robotic platforms, along with associated localization and telemetry information. The goal of this work is to provide a publicly available dataset enabling research in marine perception and multi-robot observation scenarios. This paper describes the data collection methodology, sensor configurations, dataset organization, and intended research tasks supported by the dataset.
Abstract (translated)
海洋环境由于动态表面、有限的能见度以及空中、水面和水下感知模式之间的复杂交互，为感知和自主性带来了显著挑战。本文介绍了Aerial Marine Perception Dataset（AMP2026），这是一个多平台海上机器人数据集，在多个实地部署中收集而成，旨在支持两个主要研究领域：多视图跟踪和海洋环境测绘。该数据集包括来自空中无人机、船载摄像机和水下机器人平台的同步数据，以及相关的定位和遥测信息。这项工作的目标是提供一个公开可用的数据集，以促进海洋感知及多机器人观测场景的研究。本文描述了数据收集方法、传感器配置、数据集组织以及该数据集支持的研究任务。
URL
https://arxiv.org/abs/2603.04225
PDF
-
Right in Time: Reactive Reasoning in Regulated Traffic Spaces
2026-03-04 12:18:36Simon Kohaut, Benedict Flade, Julian Eggert, Kristian Kersting, Devendra Singh DhamiAbstract
Exact inference in probabilistic First-Order Logic offers a promising yet computationally costly approach for regulating the behavior of autonomous agents in shared traffic spaces. While prior methods have combined logical and probabilistic data into decision-making frameworks, their application is often limited to pre-flight checks due to the complexity of reasoning across vast numbers of possible universes. In this work, we propose a reactive mission design framework that jointly considers uncertain environmental data and declarative, logical traffic regulations. By synthesizing Probabilistic Mission Design (ProMis) with reactive reasoning facilitated by Reactive Circuits (RC), we enable online, exact probabilistic inference over hybrid domains. Our approach leverages the Frequency of Change inherent in heterogeneous data streams to subdivide inference formulas into memoized, isolated tasks, ensuring that only the specific components affected by new sensor data are re-evaluated. In experiments involving both real-world vessel data and simulated drone traffic in dense urban scenarios, we demonstrate that our approach provides orders of magnitude in speedup over ProMis without reactive paradigms. This allows intelligent transportation systems, such as Unmanned Aircraft Systems (UAS), to actively assert safety and legal compliance during operations rather than relying solely on preparation procedures.
Abstract (translated)
在概率一阶逻辑中进行精确推理为调节共享交通空间中的自主代理行为提供了一种有前景但计算成本高昂的方法。尽管先前的方法将逻辑和概率数据结合到决策框架中，但由于跨大量可能世界的复杂推理问题，它们的应用往往仅限于飞行前检查。在此工作中，我们提出了一种反应式任务设计框架，该框架同时考虑不确定的环境数据和声明性的、基于逻辑的交通规则。通过将概率任务设计（ProMis）与由反应电路（RC）支持的反应性推理相结合，我们能够在混合领域中实现在线精确的概率推理。 我们的方法利用异构数据流中的变化频率，将推理公式分解为可以缓存并隔离的任务，从而确保仅重新评估受新传感器数据影响的具体部分。通过使用真实世界船只数据和密集城市环境中模拟无人机交通进行的实验，我们展示了与不采用反应式范式的ProMis相比，我们的方法在速度上提高了几个数量级。这使得智能运输系统（如无人驾驶飞机系统UAS）能够在操作期间主动确保安全性和合规性，而不仅仅是依赖于准备程序。 这段翻译保留了原文的主要内容和技术细节，并尽量使其通俗易懂。
URL
https://arxiv.org/abs/2603.03977
PDF
-
Self-supervised Domain Adaptation for Visual 3D Pose Estimation of Nano-drone Racing Gates by Enforcing Geometric Consistency
2026-03-03 12:46:25Nicholas Carlotti, Michele Antonazzi, Elia Cereda, Mirko Nava, Nicola Basilico, Daniele Palossi, Alessandro GiustiAbstract
We consider the task of visually estimating the relative pose of a drone racing gate in front of a nano-quadrotor, using a convolutional neural network pre-trained on simulated data to regress the gate's pose. Due to the sim-to-real gap, the pre-trained model underperforms in the real world and must be adapted to the target domain. We propose an unsupervised domain adaptation (UDA) approach using only real image sequences collected by the drone flying an arbitrary trajectory in front of a gate; sequences are annotated in a self-supervised fashion with the drone's odometry as measured by its onboard sensors. On this dataset, a state consistency loss enforces that two images acquired at different times yield pose predictions that are consistent with the drone's odometry. Results indicate that our approach outperforms other SoA UDA approaches, has a low mean absolute error in position (x=26, y=28, z=10 cm) and orientation ($\psi$=13${^{\circ}}$), an improvement of 40% in position and 37% in orientation over a baseline. The approach's effectiveness is appreciable with as few as 10 minutes of real-world flight data and yields models with an inference time of 30.4ms (33 fps) when deployed aboard the Crazyflie 2.1 Brushless nano-drone.
Abstract (translated)
我们考虑使用卷积神经网络（CNN）来估计纳米四旋翼无人机前方赛车门的相对姿态，该网络在模拟数据上进行了预训练以回归门的姿态。由于从仿真到现实之间的差距，预训练模型在实际世界中的表现不佳，必须适应目标领域。为此，我们提出了一种无监督域适应（UDA）方法，仅使用由无人机沿任意轨迹飞行并收集的实图序列；序列通过将无人机的里程计信息作为标注来自我监督地进行注释。在这个数据集上，状态一致性损失确保在不同时刻获取的两张图像所预测的姿态与无人机的里程计测量结果一致。实验结果显示，我们提出的方法优于其他最先进的UDA方法，在位置（x=26厘米, y=28厘米, z=10厘米）和姿态（$\psi$=13°）方面的平均绝对误差较低，相对于基线模型而言，在位置上提高了40%，在姿态上提高了37%。这种方法即使使用最少仅十分钟的现实飞行数据也能显示出显著效果，并且部署到Crazyflie 2.1无刷纳米无人机上的推理时间仅为30.4毫秒（每秒33帧）。
URL
https://arxiv.org/abs/2603.02936
PDF
-
Robust Tightly-Coupled Filter-Based Monocular Visual-Inertial State Estimation and Graph-Based Evaluation for Autonomous Drone Racing
2026-03-03 08:43:49Maulana Bisyir Azhari, Donghun Han, SungJun Park, David Hyunchul ShimAbstract
Autonomous drone racing (ADR) demands state estimation that is simultaneously computationally efficient and resilient to the perceptual degradation experienced during extreme velocity and maneuvers. Traditional frameworks typically rely on conventional visual-inertial pipelines with loosely-coupled gate-based Perspective-n-Points (PnP) corrections that suffer from a rigid requirement for four visible features and information loss in intermediate steps. Furthermore, the absence of GNSS and Motion Capture systems in uninstrumented, competitive racing environments makes the objective evaluation of such systems remarkably difficult. To address these limitations, we propose ADR-VINS, a robust, monocular visual-inertial state estimation framework based on an Error-State Kalman Filter (ESKF) tailored for autonomous drone racing. Our approach integrates direct pixel reprojection errors from gate corners features as innovation terms within the filter. By bypassing intermediate PnP solvers, ADR-VINS maintains valid state updates with as few as two visible corners and utilizes robust reweighting instead of RANSAC-based schemes to handle outliers, enhancing computational efficiency. Furthermore, we introduce ADR-FGO, an offline Factor-Graph Optimization framework to generate high-fidelity reference trajectories that facilitate post-flight performance evaluation and analysis on uninstrumented, GNSS-denied environments. The proposed system is validated using TII-RATM dataset, where ADR-VINS achieves an average RMS translation error of 0.134 m, while ADR-FGO yields 0.060 m as a smoothing-based reference. Finally, ADR-VINS was successfully deployed in the A2RL Drone Championship Season 2, maintaining stable and robust estimation despite noisy detections during high-agility flight at top speeds of 20.9 m/s. We further utilize ADR-FGO for post-flight evaluation in uninstrumented racing environments.
Abstract (translated)
自主无人机竞速（ADR）需要一种同时计算效率高且能抵抗高速度和机动过程中感知退化影响的状态估计方法。传统框架通常依赖于传统的视觉惯性管道，这些管道采用松耦合的门基Perspective-n-Points (PnP) 校正，这种校正方式严格要求四个可见特征，并在中间步骤中丢失信息。此外，在没有GNSS和运动捕捉系统的非仪器化竞争环境中，这样的系统的目标评估非常困难。 为了解决这些问题，我们提出了ADR-VINS，这是一个基于误差状态卡尔曼滤波器（ESKF）的稳健单目视觉惯性状态估计框架，专为自主无人机竞速设计。我们的方法将直接像素重投影误差从门角特征作为创新项整合到滤波器中。通过绕过中间的PnP求解器，ADR-VINS即使在只有两个可见角落的情况下也能保持有效状态更新，并且采用鲁棒重新加权方案处理离群值，而不是基于RANSAC的方法，这提高了计算效率。 此外，我们还引入了ADR-FGO，这是一个离线因子图优化框架，用于生成高保真参考轨迹，在非仪器化、无GNSS的环境中进行飞行后的性能评估和分析。该系统在TII-RATM数据集上得到了验证，其中ADR-VINS实现了平均0.134米的RMS平移误差，而ADR-FGO作为基于平滑的参考轨迹生成了0.060米的结果。 最后，在A2RL无人机锦标赛第二季中，ADR-VINS在最高飞行速度达到20.9m/s的情况下成功部署，并且即使检测噪声大也能保持稳定和稳健的状态估计。我们进一步使用ADR-FGO进行非仪器化竞速环境中的飞行后评估。
URL
https://arxiv.org/abs/2603.02742
PDF
-
Event-Only Drone Trajectory Forecasting with RPM-Modulated Kalman Filtering
2026-03-02 15:48:51Hari Prasanth S. M., Pejman Habibiroudkenar, Eerik Alamikkotervo, Dimitrios Bouzoulas, Risto OjalaAbstract
Event cameras provide high-temporal-resolution visual sensing that is well suited for observing fast-moving aerial objects; however, their use for drone trajectory prediction remains limited. This work introduces an event-only drone forecasting method that exploits propeller-induced motion cues. Propeller rotational speed are extracted directly from raw event data and fused within an RPM-aware Kalman filtering framework. Evaluations on the FRED dataset show that the proposed method outperforms learning-based approaches and vanilla kalman filter in terms of average distance error and final distance error at 0.4s and 0.8s forecasting horizons. The results demonstrate robust and accurate short- and medium-horizon trajectory forecasting without reliance on RGB imagery or training data.
Abstract (translated)
事件相机提供高时间分辨率的视觉感知，非常适合观察快速移动的空中物体；然而，它们在无人机轨迹预测中的应用仍然有限。本研究提出了一种仅基于事件数据的无人机预测方法，利用了螺旋桨引起的运动线索。从原始事件数据中直接提取螺旋桨旋转速度，并将其融合到一个考虑转速（RPM）的卡尔曼滤波框架内。在FRED数据集上的评估显示，在0.4秒和0.8秒的预测时间范围内，所提出的方法在平均距离误差和最终距离误差方面优于基于学习的方法和普通卡尔曼滤波器。实验结果表明，该方法能够在不依赖于RGB图像或训练数据的情况下实现稳健且准确的短时和中等时长轨迹预测。
URL
https://arxiv.org/abs/2603.01997
PDF
-
Tiny-DroNeRF: Tiny Neural Radiance Fields aboard Federated Learning-enabled Nano-drones
2026-03-02 13:30:56Ilenia Carboni, Elia Cereda, Lorenzo Lamberti, Daniele Malpetti, Francesco Conti, Daniele PalossiAbstract
Sub-30g nano-sized aerial robots can leverage their agility and form factor to autonomously explore cluttered and narrow environments, like in industrial inspection and search and rescue missions. However, the price for their tiny size is a strong limit in their resources, i.e., sub-100 mW microcontroller units (MCUs) delivering $\sim$100 GOps/s at best, and memory budgets well below 100 MB. Despite these strict constraints, we aim to enable complex vision-based tasks aboard nano-drones, such as dense 3D scene reconstruction: a key robotic task underlying fundamental capabilities like spatial awareness and motion planning. Top-performing 3D reconstruction methods leverage neural radiance fields (NeRF) models, which require GBs of memory and massive computation, usually delivered by high-end GPUs consuming 100s of Watts. Our work introduces Tiny-DroNeRF, a lightweight NeRF model, based on Instant-NGP, and optimized for running on a GAP9 ultra-low-power (ULP) MCU aboard our nano-drones. Then, we further empower our Tiny-DroNeRF by leveraging a collaborative federated learning scheme, which distributes the model training among multiple nano-drones. Our experimental results show a 96% reduction in Tiny-DroNeRF's memory footprint compared to Instant-NGP, with only a 5.7 dB drop in reconstruction accuracy. Finally, our federated learning scheme allows Tiny-DroNeRF to train with an amount of data otherwise impossible to keep in a single drone's memory, increasing the overall reconstruction accuracy. Ultimately, our work combines, for the first time, NeRF training on an ULP MCU with federated learning on nano-drones.
Abstract (translated)
重量低于30克的纳米级空中机器人可以利用其敏捷性和外形尺寸自主探索拥挤和狭窄的环境，例如在工业检查和搜索救援任务中。然而，它们的小尺寸带来了资源限制的代价，即微控制器单元（MCU）提供的计算能力不超过100毫瓦，提供大约100 GOPs/s 的处理速度，并且内存预算低于100MB。尽管存在这些严格的约束条件，我们的目标是在纳米无人机上启用复杂的基于视觉的任务，例如密集的3D场景重建：这是机器人技术中支持空间感知和运动规划等基本能力的关键任务之一。 高性能的3D重建方法通常依赖于神经辐射场（NeRF）模型，这类模型需要数GB的内存以及大量的计算量，这通常由消耗数百瓦电力的高端GPU提供。我们的研究工作介绍了一种基于Instant-NGP且轻量化优化过的Tiny-DroNeRF NeRF模型，并针对我们纳米无人机上搭载的GAP9超低功耗（ULP）MCU进行了专门优化。 接下来，通过利用一种协作联邦学习方案，我们将进一步增强我们的Tiny-DroNeRF，该方案将模型训练任务分发到多个纳米无人机中。实验结果显示，与Instant-NGP相比，Tiny-DroNeRF的内存占用减少了96%，同时仅导致重建精度下降了5.7dB。此外，通过联邦学习方案，Tiny-DroNeRF能够使用单个无人机无法存储的数据量进行训练，从而提高了整体重建精度。 最终，我们的工作首次实现了在超低功耗微控制器单元（ULP MCU）上对神经辐射场模型进行训练，并结合纳米级无人机上的联邦学习。这一组合为资源受限的微型机器人系统带来了重要的技术突破，开启了新的可能性。
URL
https://arxiv.org/abs/2603.01850
PDF
-
SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testing
2026-03-02 09:06:28Anjali Parashar, Yingke Li, Eric Yang Yu, Fei Chen, James Neidhoefer, Devesh Upadhyay, Chuchu FanAbstract
As autonomous systems such as drones, become increasingly deployed in high-stakes, human-centric domains, it is critical to evaluate the ethical alignment since failure to do so imposes imminent danger to human lives, and long term bias in decision-making. Automated ethical benchmarking of these systems is understudied due to the lack of ubiquitous, well-defined metrics for evaluation, and stakeholder-specific subjectivity, which cannot be modeled analytically. To address these challenges, we propose SEED-SET, a Bayesian experimental design framework that incorporates domain-specific objective evaluations, and subjective value judgments from stakeholders. SEED-SET models both evaluation types separately with hierarchical Gaussian Processes, and uses a novel acquisition strategy to propose interesting test candidates based on learnt qualitative preferences and objectives that align with the stakeholder preferences. We validate our approach for ethical benchmarking of autonomous agents on two applications and find our method to perform the best. Our method provides an interpretable and efficient trade-off between exploration and exploitation, by generating up to $2\times$ optimal test candidates compared to baselines, with $1.25\times$ improvement in coverage of high dimensional search spaces.
Abstract (translated)
随着无人机等自主系统在高风险、以人为中心的领域中的部署日益增多，评估其伦理一致性变得至关重要。这是因为未能进行此类评估会立即对人类生命构成危险，并可能导致长期决策偏差。由于缺乏普遍适用且定义明确的评价指标以及不同利益相关者特有的主观性（这些无法通过分析建模），自动化伦理基准测试的研究尚处于初级阶段。 为解决这一挑战，我们提出了一种名为SEED-SET的贝叶斯实验设计框架，该框架结合了特定领域的客观评估和来自利益相关者的主观价值判断。SEED-SET使用分层高斯过程分别建模这两种类型的评价，并采用一种新颖的方法来根据学习到的质量偏好和符合利益相关者偏好的目标提出有趣的测试候选方案。 我们在两个应用上验证了我们的伦理基准测试方法的有效性，发现该方法优于其他基线方法。我们的方法提供了一个可解释且高效的探索与利用之间的权衡机制，在生成最优测试候选方案方面比基线提高了多达两倍，并在高维搜索空间的覆盖率上提升了1.25倍。 通过SEED-SET框架，我们能够更好地理解自主系统中的伦理问题，并为其制定合适的基准测试策略，从而促进这些系统的安全和可靠部署。
URL
https://arxiv.org/abs/2603.01630
PDF
-
Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles
2026-02-27 13:56:59Fangyu Sun, Fanxing Li, Yu Hu, Linzuo Zhang, Yueqian Liu, Wenxian Yu, Danping ZouAbstract
Autonomous drone racing has attracted increasing interest as a research topic for exploring the limits of agile flight. However, existing studies primarily focus on obstacle-free racetracks, while the perception and dynamic challenges introduced by obstacles remain underexplored, often resulting in low success rates and limited robustness in real-world flight. To this end, we propose a novel vision-based curriculum reinforcement learning framework for training a robust controller capable of addressing unseen obstacles in drone racing. We combine multi-stage cu rriculum learning, domain randomization, and a multi-scene updating strategy to address the conflicting challenges of obstacle avoidance and gate traversal. Our end-to-end control policy is implemented as a single network, allowing high-speed flight of quadrotors in environments with variable obstacles. Both hardware-in-the-loop and real-world experiments demonstrate that our method achieves faster lap times and higher success rates than existing approaches, effectively advancing drone racing in obstacle-rich environments. The video and code are available at: this https URL.
Abstract (translated)
自主无人机竞速作为一种探索敏捷飞行极限的研究课题，已经吸引了越来越多的关注。然而，现有的研究主要集中在无障碍赛道上，而障碍物带来的感知和动态挑战则较少被探讨，这导致实际飞行中成功率较低且鲁棒性有限。为此，我们提出了一种基于视觉的课程强化学习框架，旨在训练出一种能够应对无人机竞速中未见过障碍物的稳健控制器。通过结合多阶段课程学习、领域随机化以及多场景更新策略，我们的方法可以解决避障与穿越门道之间的冲突挑战。整个控制策略以单个网络实现，使四旋翼飞机能够在具有可变障碍物的环境中进行高速飞行。 无论是硬件在环还是真实环境实验都表明，相较于现有方法，本框架能够取得更快的圈速和更高的成功率，在充满障碍物的竞速环境下显著推进了无人机技术的发展。视频及代码可在以下链接获取：[此URL]。
URL
https://arxiv.org/abs/2602.24030
PDF
