Autonomous drone racing presents a challenging control problem, requiring real-time decision-making and robust handling of nonlinear system dynamics. While iterative learning model predictive control~(LMPC) offers a promising framework for iterative performance improvement, its direct application to drone racing faces challenges like real-time compatibility or the trade-off between time-optimal and safe traversal. In this paper, we enhance LMPC with three key innovations:~(1) an adaptive cost function that dynamically weights time-optimal tracking against centerline adherence,~(2)~a shifted local safe set to prevent excessive shortcutting and enable more robust iterative updates, and~(3) a Cartesian-based formulation that accommodates safety constraints without the singularities or integration errors associated with Frenet-frame transformations. Results from extensive simulation and real-world experiments demonstrate that our improved algorithm can optimize initial trajectories generated by a wide range of controllers with varying levels of tuning for a maximum improvement in lap time by 60.85\%. Even applied to the most aggressively tuned state-of-the-art model-based controller, MPCC++, on a real drone, a 6.05\% improvement is still achieved. Overall, the proposed method pushes the drone toward faster traversal and avoids collisions in simulation and real-world experiments, making it a practical solution to improve the peak performance of drone racing.
自主无人机竞速提出了一个具有挑战性的控制问题,需要实时决策和非线性系统动态的稳健处理。虽然迭代学习模型预测控制(LMPC)为性能改进提供了有前景的框架,但将其直接应用于无人机竞速面临诸如实时兼容性和时间最优与安全穿越之间的权衡等问题。在本文中,我们通过三项关键创新增强了LMPC:(1)一个自适应成本函数,该函数动态地平衡了时间最优跟踪和中心线遵循之间的权重;(2)移位局部安全集,以防止过度捷径并使迭代更新更加稳健;以及 (3) 一种基于笛卡尔坐标的公式化方法,能够容纳安全性约束而不会出现与Frenet框架转换相关的奇点或积分误差。广泛的模拟和现实世界实验的结果表明,我们的改进算法可以优化由各种控制器产生的初始轨迹,并在不同调整水平下实现60.85%的最大圈速改善。即使应用到最先进的模型基控制器MPCC++(最激进调优情况下)的实飞无人机上,仍然能取得6.05%的性能提升。总的来说,所提出的方法使无人机能够在模拟和现实世界实验中更快地穿越并避免碰撞,从而成为提高无人机竞速峰值性能的实际解决方案。
https://arxiv.org/abs/2508.01103
Advances in 3D reconstruction and novel view synthesis have enabled efficient, photorealistic rendering, but the data collection process remains largely manual, making it time-consuming and labor-intensive. To address the challenges, this study introduces Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection (Hestia), which leverages reinforcement learning to learn a generalizable policy for 5-DoF next-best viewpoint prediction. Unlike prior approaches, Hestia systematically defines the next-best-view task by proposing core components such as dataset choice, observation design, action space, reward calculation, and learning schemes, forming a foundation for the planner. Hestia goes beyond prior next-best-view approaches and traditional capture systems through integration and validation in a real-world setup, where a drone serves as a mobile sensor for active scene exploration. Experimental results show that Hestia performs robustly across three datasets and translated object settings in the NVIDIA IsaacLab environment, and proves feasible for real-world deployment.
在三维重建和新视角合成方面的进展已经使得高效、逼真的渲染成为可能,但数据采集过程仍然主要是手动进行的,这既耗时又耗费人力。为了应对这一挑战,本研究提出了分层最佳视点探索方法(Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection, 简称Hestia),该方法利用强化学习来学习一种适用于5自由度下最佳视点预测的通用策略。与先前的方法不同,Hestia系统地定义了最佳视点任务,并提出了核心组件,如数据集选择、观测设计、动作空间、奖励计算和学习方案,为规划者奠定了基础。 Hestia超越了之前的最佳视点方法及传统的捕捉系统,在一个实际设置中通过无人机作为移动传感器进行主动场景探索实现了集成与验证。实验结果表明,Hestia在NVIDIA IsaacLab环境中三个数据集和转换后的物体设置上表现稳健,并证明其适合于现实世界的部署。 总结起来,该研究介绍了一种创新的自动数据采集系统(Hestia),它能够高效地预测最佳视点并进行3D重建工作,为自动化、智能化的数据采集提供了新的解决方案。
https://arxiv.org/abs/2508.01014
Generative Adversarial Networks (GANs) have achieved realistic super-resolution (SR) of images however, they lack semantic consistency and per-pixel confidence, limiting their credibility in critical remote sensing applications such as disaster response, urban planning and agriculture. This paper introduces Semantic and Uncertainty-Aware ESRGAN (SU-ESRGAN), the first SR framework designed for satellite imagery to integrate the ESRGAN, segmentation loss via DeepLabv3 for class detail preservation and Monte Carlo dropout to produce pixel-wise uncertainty maps. The SU-ESRGAN produces results (PSNR, SSIM, LPIPS) comparable to the Baseline ESRGAN on aerial imagery. This novel model is valuable in satellite systems or UAVs that use wide field-of-view (FoV) cameras, trading off spatial resolution for coverage. The modular design allows integration in UAV data pipelines for on-board or post-processing SR to enhance imagery resulting due to motion blur, compression and sensor limitations. Further, the model is fine-tuned to evaluate its performance on cross domain applications. The tests are conducted on two drone based datasets which differ in altitude and imaging perspective. Performance evaluation of the fine-tuned models show a stronger adaptation to the Aerial Maritime Drone Dataset, whose imaging characteristics align with the training data, highlighting the importance of domain-aware training in SR-applications.
生成对抗网络(GANs)在图像超分辨率(SR)方面取得了现实感极强的效果,然而它们缺乏语义一致性与像素级别的信心度,这限制了其在如灾害响应、城市规划和农业等关键遥感应用中的可信度。本文介绍了《语义及不确定性感知的ESRGAN(SU-ESRGAN)》,这是首个专为卫星图像设计的超分辨率框架,该框架融合了ESRGAN模型、通过DeepLabv3实现的分割损失以保留类别细节,并采用蒙特卡洛dropout来生成像素级别的不确定性图。SU-ESRGAN在航拍图像上产生的结果(PSNR,SSIM,LPIPS)与基线ESRGAN相当。 这种新型模型对于使用宽视场(FoV)相机的卫星系统或无人机特别有价值,在这些系统中通常会牺牲空间分辨率以换取覆盖范围。其模块化设计允许在无人机数据管道中进行集成,无论是机载还是后期处理超分辨率任务,都可增强因运动模糊、压缩和传感器限制导致的质量下降的图像。 此外,该模型经过微调来评估其跨域应用性能。测试是在两个基于无人机的数据集上进行的,这两个数据集在海拔高度和成像视角方面有所不同。对于微调后的模型性能评价显示,SU-ESRGAN对与训练数据特征更为匹配的空中海洋无人机数据集具有更强的适应性,这突显了领域感知训练在超分辨率应用中的重要性。
https://arxiv.org/abs/2508.00750
The proposal introduces an innovative drone swarm perception system that aims to solve problems related to computational limitations and low-bandwidth communication, and real-time scene reconstruction. The framework enables efficient multi-agent 3D/4D scene synthesis through federated learning of shared diffusion model and YOLOv12 lightweight semantic extraction and local NeRF updates while maintaining privacy and scalability. The framework redesigns generative diffusion models for joint scene reconstruction, and improves cooperative scene understanding, while adding semantic-aware compression protocols. The approach can be validated through simulations and potential real-world deployment on drone testbeds, positioning it as a disruptive advancement in multi-agent AI for autonomous systems.
该提案介绍了一种创新的无人机群感知系统,旨在解决计算限制、低带宽通信以及实时场景重构的问题。此框架通过联合学习共享扩散模型和轻量级语义提取(YOLOv12)及局部NeRF更新来实现高效的多智能体3D/4D场景合成,并且在保持隐私性和可扩展性的同时实现了这一点。该框架重新设计了生成式扩散模型以进行联合场景重构,改进了协同场景理解能力,并增加了具有语义感知的压缩协议。可以通过模拟和潜在的真实世界部署(如无人机测试平台)对该方法的有效性进行验证,这标志着多智能体AI在自主系统中的颠覆性进步。
https://arxiv.org/abs/2508.00967
In the context of Concentrated Solar Power (CSP) plants, aerial images captured by drones present a unique set of challenges. Unlike urban or natural landscapes commonly found in existing datasets, solar fields contain highly reflective surfaces, and domain-specific elements that are uncommon in traditional computer vision benchmarks. As a result, machine learning models trained on generic datasets struggle to generalize to this setting without extensive retraining and large volumes of annotated data. However, collecting and labeling such data is costly and time-consuming, making it impractical for rapid deployment in industrial applications. To address this issue, we propose a novel approach: the creation of AerialCSP, a virtual dataset that simulates aerial imagery of CSP plants. By generating synthetic data that closely mimic real-world conditions, our objective is to facilitate pretraining of models before deployment, significantly reducing the need for extensive manual labeling. Our main contributions are threefold: (1) we introduce AerialCSP, a high-quality synthetic dataset for aerial inspection of CSP plants, providing annotated data for object detection and image segmentation; (2) we benchmark multiple models on AerialCSP, establishing a baseline for CSP-related vision tasks; and (3) we demonstrate that pretraining on AerialCSP significantly improves real-world fault detection, particularly for rare and small defects, reducing the need for extensive manual labeling. AerialCSP is made publicly available at this https URL.
在太阳能热发电(Concentrated Solar Power,简称CSP)电站的背景下,无人机捕获的空中图像带来了一套独特的挑战。与现有数据集中常见的城市或自然景观不同,太阳能场包含高度反射的表面和传统计算机视觉基准中少见的特定领域元素。因此,在通用数据集上训练的机器学习模型在不进行大量再训练及未获取大体量标注数据的情况下难以适应这一环境。然而,收集并标记此类数据的成本高昂且耗时长,这使得其在工业应用中的快速部署变得不可行。 为了解决这个问题,我们提出了一种新颖的方法:创建AerialCSP,这是一个虚拟的数据集,用于模拟CSP电站的空中图像。通过生成与真实世界条件极为相似的人工合成数据,我们的目标是使模型能在实际部署前进行预训练,并大幅减少对大量手动标注的需求。 我们的主要贡献有三点: 1. 我们介绍了AerialCSP,这是一个高质量的合成数据集,用于CSP电站的空中检查,提供了对象检测和图像分割方面的注释数据。 2. 我们在AerialCSP上评估了多种模型,为与CSP相关的视觉任务建立了基准。 3. 我们展示了通过AerialCSP进行预训练能显著提高真实世界中的故障检测能力,尤其是对于罕见且小的缺陷,从而减少对大量手动标注的需求。 AerialCSP数据集现可在此网址公开获取。
https://arxiv.org/abs/2508.00440
We introduce a U-net model for 360° acoustic source localization formulated as a spherical semantic segmentation task. Rather than regressing discrete direction-of-arrival (DoA) angles, our model segments beamformed audio maps (azimuth and elevation) into regions of active sound presence. Using delay-and-sum (DAS) beamforming on a custom 24-microphone array, we generate signals aligned with drone GPS telemetry to create binary supervision masks. A modified U-Net, trained on frequency-domain representations of these maps, learns to identify spatially distributed source regions while addressing class imbalance via the Tversky loss. Because the network operates on beamformed energy maps, the approach is inherently array-independent and can adapt to different microphone configurations without retraining from scratch. The segmentation outputs are post-processed by computing centroids over activated regions, enabling robust DoA estimates. Our dataset includes real-world open-field recordings of a DJI Air 3 drone, synchronized with 360° video and flight logs across multiple dates and locations. Experimental results show that U-net generalizes across environments, providing improved angular precision, offering a new paradigm for dense spatial audio understanding beyond traditional Sound Source Localization (SSL).
我们介绍了一种用于360°声源定位的U-Net模型,该模型被形式化为一个球面语义分割任务。我们的方法不是回归离散的方向角(DoA),而是将波束形成的音频地图(方位和仰角)划分为有声音活动的区域。我们使用延迟与求和(DAS)波束形成技术在自定义24麦克风阵列上生成与无人机GPS数据对齐的信号,从而创建二值监督掩码。通过在这些地图的频域表示形式上训练修改后的U-Net模型,并采用Tversky损失来处理类别不平衡问题,该网络能够识别空间分布的声音源区域。由于网络操作基于波束形成的能量图,因此此方法本质上与阵列无关,可以适应不同麦克风配置而无需从头开始重新训练。 在激活区域内计算质心的后处理步骤使我们的模型能够提供稳健的方向角(DoA)估计。我们数据集包括在现实世界开放环境中录制的DJI Air 3无人机的声音样本,这些声音与多个日期和地点同步记录的360°视频和飞行日志相结合。 实验结果表明,U-Net模型能够在不同环境条件下泛化,并提供改进的角度精度,这为传统的声源定位(SSL)之外的空间音频理解提供了新的范式。
https://arxiv.org/abs/2508.00307
Road damage detection is a critical task for ensuring traffic safety and maintaining infrastructure integrity. While deep learning-based detection methods are now widely adopted, they still face two core challenges: first, the inadequate multi-scale feature extraction capabilities of existing networks for diverse targets like cracks and potholes, leading to high miss rates for small-scale damage; and second, the substantial parameter counts and computational demands of mainstream models, which hinder their deployment for efficient, real-time detection in practical applications. To address these issues, this paper proposes a high-precision and lightweight model, YOLO - Road Orthogonal Compact (YOLO-ROC). We designed a Bidirectional Multi-scale Spatial Pyramid Pooling Fast (BMS-SPPF) module to enhance multi-scale feature extraction and implemented a hierarchical channel compression strategy to reduce computational complexity. The BMS-SPPF module leverages a bidirectional spatial-channel attention mechanism to improve the detection of small targets. Concurrently, the channel compression strategy reduces the parameter count from 3.01M to 0.89M and GFLOPs from 8.1 to 2.6. Experiments on the RDD2022_China_Drone dataset demonstrate that YOLO-ROC achieves a mAP50 of 67.6%, surpassing the baseline YOLOv8n by 2.11%. Notably, the mAP50 for the small-target D40 category improved by 16.8%, and the final model size is only 2.0 MB. Furthermore, the model exhibits excellent generalization performance on the RDD2022_China_Motorbike dataset.
道路损坏检测是确保交通安全和维护基础设施完整性的关键任务。虽然基于深度学习的检测方法现在已被广泛采用,但仍面临两大核心挑战:首先,现有的网络在提取裂缝、坑洞等多样化目标的多尺度特征方面能力不足,导致小规模损害的漏检率高;其次,主流模型参数量大且计算需求高,这限制了它们在实际应用中进行高效实时检测的能力。为了解决这些问题,本文提出了一种高精度和轻量级模型——YOLO-Road Orthogonal Compact (YOLO-ROC)。 我们设计了一个双向多尺度空间金字塔池化快速(BMS-SPPF)模块来增强多尺度特征提取,并实施了层次化的通道压缩策略以减少计算复杂度。BMS-SPPF模块利用双向空间-通道注意机制,提升了对小目标的检测能力。同时,通过通道压缩策略将参数量从3.01M降低到0.89M,浮点运算次数(GFLOPs)也从8.1降至2.6。 在RDD2022_China_Drone数据集上的实验表明,YOLO-ROC达到了mAP50为67.6%,比基准模型YOLOv8n高出2.11%。值得注意的是,针对小目标D40类别的mAP50提高了16.8%,最终模型大小仅为2.0 MB。此外,在RDD2022_China_Motorbike数据集上,该模型表现出色的泛化性能。 这些改进和优化使YOLO-ROC成为在实际应用中高效进行道路损坏检测的理想选择。
https://arxiv.org/abs/2507.23225
The proliferation of drones in civilian airspace has raised urgent security concerns, necessitating robust real-time surveillance systems. In response to the 2025 VIP Cup challenge tasks - drone detection, tracking, and payload identification - we propose a dual-stream drone monitoring framework. Our approach deploys independent You Only Look Once v11-nano (YOLOv11n) object detectors on parallel infrared (thermal) and visible (RGB) data streams, deliberately avoiding early fusion. This separation allows each model to be specifically optimized for the distinct characteristics of its input modality, addressing the unique challenges posed by small aerial objects in diverse environmental conditions. We customize data preprocessing and augmentation strategies per domain - such as limiting color jitter for IR imagery - and fine-tune training hyperparameters to enhance detection performance under conditions of heavy noise, low light, and motion blur. The resulting lightweight YOLOv11n models demonstrate high accuracy in distinguishing drones from birds and in classifying payload types, all while maintaining real-time performance. This report details the rationale for a dual-modality design, the specialized training pipelines, and the architectural optimizations that collectively enable efficient and accurate drone surveillance across RGB and IR channels.
民用空域中无人机的广泛使用引发了紧急的安全问题,需要强大的实时监控系统。为应对2025年VIP杯挑战赛的任务——包括无人机检测、跟踪和载荷识别,我们提出了一种双流无人机监测框架。我们的方法部署了独立的You Only Look Once v11-nano(YOLOv11n)物体探测器,在平行的红外(热成像)和可见光(RGB)数据流上运行,并故意避免早期融合。这种分离使得每个模型能够针对其输入模式的独特特性进行专门优化,从而解决了在各种环境条件下小型空中物体带来的独特挑战。 我们根据领域定制了数据预处理和增强策略——例如限制IR图像中的颜色抖动——并调整训练超参数以在重噪音、低光和运动模糊的条件下提高检测性能。最终产生的轻量级YOLOv11n模型能够在区分无人机与鸟类及识别载荷类型方面表现出高精度,同时保持实时性能。 本报告详细阐述了双模态设计的理由、专门化的训练流程以及架构优化措施,这些共同实现了RGB和IR通道上高效且准确的无人机监测。
https://arxiv.org/abs/2507.22650
For the autonomous drone-based inspection of wind turbine (WT) blades, accurate detection of the WT and its key features is essential for safe drone positioning and collision avoidance. Existing deep learning methods typically rely on manually labeled real-world images, which limits both the quantity and the diversity of training datasets in terms of weather conditions, lighting, turbine types, and image complexity. In this paper, we propose a method to generate synthetic training data that allows controlled variation of visual and environmental factors, increasing the diversity and hence creating challenging learning scenarios. Furthermore, we train a YOLOv11 feature detection network solely on synthetic WT images with a modified loss function, to detect WTs and their key features within an image. The resulting network is evaluated both using synthetic images and a set of real-world WT images and shows promising performance across both synthetic and real-world data, achieving a Pose mAP50-95 of 0.97 on real images never seen during training.
针对基于自主无人机的风力涡轮机(WT)叶片检查,准确检测WT及其关键特征对于安全的无人机定位和碰撞避免至关重要。现有的深度学习方法通常依赖于人工标注的真实世界图像,这在天气条件、光照、涡轮类型以及图像复杂度方面限制了训练数据集的数量和多样性。本文提出了一种生成合成训练数据的方法,该方法能够控制视觉和环境因素的变化,从而增加多样性和创造具有挑战性的学习场景。 此外,我们使用修改后的损失函数,在仅由合成WT图像构成的数据集上训练YOLOv11特征检测网络,以在图像中检测WT及其关键特征。经过评估,该网络不仅利用合成图像进行了测试,还利用了一组真实世界中的WT图像,并且在这两类数据上都展示了有前景的性能表现,在从未用于训练的真实图像上达到了Pose mAP50-95为0.97的成绩。
https://arxiv.org/abs/2507.21611
A drone trajectory planner should be able to dynamically adjust the safety-efficiency trade-off according to varying mission requirements in unknown environments. Although traditional polynomial-based planners offer computational efficiency and smooth trajectory generation, they require expert knowledge to tune multiple parameters to adjust this trade-off. Moreover, even with careful tuning, the resulting adjustment may fail to achieve the desired trade-off. Similarly, although reinforcement learning-based planners are adaptable in unknown environments, they do not explicitly address the safety-efficiency trade-off. To overcome this limitation, we introduce a Decision Transformer-based trajectory planner that leverages a single parameter, Return-to-Go (RTG), as a \emph{temperature parameter} to dynamically adjust the safety-efficiency trade-off. In our framework, since RTG intuitively measures the safety and efficiency of a trajectory, RTG tuning does not require expert knowledge. We validate our approach using Gazebo simulations in both structured grid and unstructured random environments. The experimental results demonstrate that our planner can dynamically adjust the safety-efficiency trade-off by simply tuning the RTG parameter. Furthermore, our planner outperforms existing baseline methods across various RTG settings, generating safer trajectories when tuned for safety and more efficient trajectories when tuned for efficiency. Real-world experiments further confirm the reliability and practicality of our proposed planner.
一个无人机轨迹规划器应该能够在未知环境中根据不同的任务需求动态调整安全性和效率之间的权衡。虽然传统的基于多项式的规划方法提供了计算效率和生成平滑路径的能力,但它们需要专家知识来调优多个参数以调整这种权衡关系。即使经过仔细的调优,结果可能也无法实现预期的权衡效果。同样地,尽管基于强化学习的方法可以在未知环境中进行自适应调整,但他们并没有明确解决安全性和效率之间的权衡问题。 为了解决这个问题,我们引入了一种基于决策Transformer(Decision Transformer)的轨迹规划器,利用单一参数“回退收益”(Return-to-Go, RTG)作为动态调节安全和效率之间权衡关系的“温度参数”。在我们的框架中,由于RTG直观地衡量了路径的安全性和效率,因此调整RTG并不需要专家知识。 我们使用Gazebo仿真软件,在结构化网格环境和无序随机环境中验证了这一方法。实验结果表明,通过简单地调整RTG参数,我们的规划器可以动态调节安全与效率之间的权衡。此外,在各种RTG设置下,我们的规划器的表现优于现有基准方法:在调优以提高安全性时生成更安全的路径,在调优以提高效率时生成更高效的路径。现实世界的实验进一步确认了我们所提出的规划器的可靠性和实用性。
https://arxiv.org/abs/2507.21506
In the research and development (R&D) and verification and validation (V&V) phases of autonomous driving decision-making and planning systems, it is necessary to integrate human factors to achieve decision-making and evaluation that align with human cognition. However, most existing datasets primarily focus on vehicle motion states and trajectories, neglecting human-related information. In addition, current naturalistic driving datasets lack sufficient safety-critical scenarios while simulated datasets suffer from low authenticity. To address these issues, this paper constructs the Risk-Informed Subjective Evaluation and Eye-tracking (RISEE) dataset which specifically contains human subjective evaluations and eye-tracking data apart from regular naturalistic driving trajectories. By leveraging the complementary advantages of drone-based (high realism and extensive scenario coverage) and simulation-based (high safety and reproducibility) data collection methods, we first conduct drone-based traffic video recording at a highway ramp merging area. After that, the manually selected highly interactive scenarios are reconstructed in simulation software, and drivers' first-person view (FPV) videos are generated, which are then viewed and evaluated by recruited participants. During the video viewing process, participants' eye-tracking data is collected. After data processing and filtering, 3567 valid subjective risk ratings from 101 participants across 179 scenarios are retained, along with 2045 qualified eye-tracking data segments. The collected data and examples of the generated FPV videos are available in our website.
在自动驾驶决策与规划系统的研发(R&D)和验证及确认(V&V)阶段,有必要整合人类因素以实现符合人类认知的决策与评估。然而,大多数现有数据集主要关注车辆运动状态和轨迹,而忽略了相关的人类信息。此外,当前自然驾驶数据集中缺乏足够的关键安全场景,同时模拟数据集的真实性较低。为解决这些问题,本文构建了基于风险的主观评价及眼动追踪(RISEE)数据集,该数据集特别包含了除常规自然驾驶轨迹之外的人类主观评价和眼动追踪数据。 通过利用无人机采集(高现实感和广泛的场景覆盖范围)与模拟采集方法(高安全性和可重复性)的优势互补,我们首先在高速路匝道汇流区域进行无人机交通视频录制。接着,在仿真软件中重建手动选择的互动性强的场景,并生成驾驶员第一人称视角(FPV) 视频,然后由招募的参与者观看并评估这些视频。在视频观看过程中,收集参与者的视觉追踪数据。经过数据处理和过滤后,保留了来自101位参与者对179个场景的有效主观风险评分3567份及2045段合格的眼动追踪数据。所采集的数据以及生成的FPV视频示例可在我们的网站上获得。
https://arxiv.org/abs/2507.19490
Aerial wildlife tracking is critical for conservation efforts and relies on detecting small objects on the ground below the aircraft. It presents technical challenges: crewed aircraft are expensive, risky and disruptive; autonomous drones have limited computational capacity for onboard AI systems. Since the objects of interest may appear only a few pixels wide, small object detection is an inherently challenging computer vision subfield compounded by computational efficiency needs. This paper applies a patching augmentation to datasets to study model performance under various settings. A comparative study of three common yet architecturally diverse object detectors is conducted using the data, varying the patching method's hyperparameters against detection accuracy. Each model achieved at least 93\% mAP@IoU=0.5 on at least one patching configuration. Statistical analyses provide an in-depth commentary on the effects of various factors. Analysis also shows that faster, simpler models are about as effective as models that require more computational power for this task and perform well given limited patch scales, encouraging UAV deployment. Datasets and models will be made available via this https URL.
空中野生动物追踪对于保护工作至关重要,它依赖于在飞机下方地面上检测小型物体。这带来了技术挑战:有人驾驶的飞机成本高、风险大且干扰性强;而自主无人机由于计算能力有限,其机载AI系统的能力受到限制。鉴于感兴趣的物体可能仅占据图像中的几个像素宽度,因此小目标检测是一个固有的具有挑战性的计算机视觉子领域,并且需要考虑计算效率的需求。 本文通过在数据集上应用补丁增强技术来研究不同设置下的模型性能。使用这些数据进行了三种常见但架构各异的目标检测器的比较研究,通过改变补丁方法的超参数来观察其对检测准确度的影响。每个模型至少在一个补丁配置下达到了93%的mAP@IoU=0.5。 统计分析提供了关于各种因素影响的深入评论。分析还表明,更快、更简单的模型在执行此任务时效果与需要更多计算能力的模型相当,并且在有限的补丁尺度下表现出色,这鼓励了无人机系统的部署。数据集和模型将在以下网址提供:[此链接](https://example.com)(请将URL替换为实际提供的地址)。
https://arxiv.org/abs/2507.21256
This work introduces a self-supervised neuro-analytical, cost efficient, model for visual-based quadrotor control in which a small 1.7M parameters student ConvNet learns automatically from an analytical teacher, an improved image-based visual servoing (IBVS) controller. Our IBVS system solves numerical instabilities by reducing the classical visual servoing equations and enabling efficient stable image feature detection. Through knowledge distillation, the student model achieves 11x faster inference compared to the teacher IBVS pipeline, while demonstrating similar control accuracy at a significantly lower computational and memory cost. Our vision-only self-supervised neuro-analytic control, enables quadrotor orientation and movement without requiring explicit geometric models or fiducial markers. The proposed methodology leverages simulation-to-reality transfer learning and is validated on a small drone platform in GPS-denied indoor environments. Our key contributions include: (1) an analytical IBVS teacher that solves numerical instabilities inherent in classical approaches, (2) a two-stage segmentation pipeline combining YOLOv11 with a U-Net-based mask splitter for robust anterior-posterior vehicle segmentation to correctly estimate the orientation of the target, and (3) an efficient knowledge distillation dual-path system, which transfers geometric visual servoing capabilities from the analytical IBVS teacher to a compact and small student neural network that outperforms the teacher, while being suitable for real-time onboard deployment.
这项工作介绍了一种自监督神经分析、成本效益的模型,用于基于视觉的四旋翼飞行器控制。该模型中一个只有1.7M参数的小学生卷积网络(ConvNet)自动从改进后的基于图像的视觉伺服控制器(IBVS)这一分析教师那里学习。 我们的IBVS系统通过简化传统的视觉伺服方程来解决数值不稳定性,并实现了高效的稳定图像特征检测。通过知识蒸馏,学生的模型在推理速度上比老师的IBVS管道快11倍,同时表现出相似的控制精度,但在计算和内存成本方面显著降低。我们提出的仅依赖于视觉的自监督神经分析控制方法使四旋翼飞行器能够在不需要显式几何模型或标记点的情况下调整姿态与移动。 该方法利用了仿真到现实的迁移学习,并在一个小型无人机平台上通过GPS限制下的室内环境进行了验证。我们的主要贡献包括:(1)一种能够解决经典方法中固有数值不稳定的分析IBVS教师;(2)结合YOLOv11和基于U-Net的分割器的两阶段分割流水线,用于稳健地对前后车辆进行分割,从而准确估计目标的方向;(3)一个高效的双通道知识蒸馏系统,该系统能够将几何视觉伺服能力从分析IBVS教师转移到一个小而紧凑的学生神经网络上,该学生模型优于老师,并适用于实时机载部署。
https://arxiv.org/abs/2507.19878
In coverage control problems that involve time-varying density functions, the coverage control law depends on spatial integrals of the time evolution of the density function. The latter is often neglected, replaced with an upper bound or calculated as a numerical approximation of the spatial integrals involved. In this paper, we consider a special case of time-varying density functions modeled as Gaussian Mixture Models (GMMs) that evolve with time via a set of time-varying sources (with known corresponding velocities). By imposing this structure, we obtain an efficient time-varying coverage controller that fully incorporates the time evolution of the density function. We show that the induced trajectories under our control law minimise the overall coverage cost. We elicit the structure of the proposed controller and compare it with a classical time-varying coverage controller, against which we benchmark the coverage performance in simulation. Furthermore, we highlight that the computationally efficient and distributed nature of the proposed control law makes it ideal for multi-vehicle robotic applications involving time-varying coverage control problems. We employ our method in plume monitoring using a swarm of drones. In an experimental field trial we show that drones guided by the proposed controller are able to track a simulated time-varying chemical plume in a distributed manner.
在处理涉及时变密度函数的覆盖控制问题时,覆盖控制法则依赖于该密度函数随时间变化的空间积分。通常情况下,这些空间积分要么被忽略,要么用上限替换,或者计算成数值近似值。本文中,我们考虑了一种特殊情况:使用高斯混合模型(GMMs)来建模的时变密度函数,这种模型会随着时间的变化而演变,并且这一变化过程是由一组时变源驱动的(这些源对应的运动速度是已知的)。通过引入这样的结构,我们可以获得一种高效的时间可变覆盖控制器,该控制器能够全面地整合密度函数随时间的演化。我们证明了在我们的控制法则作用下诱导出的轨迹将最小化整体覆盖成本。 本文揭示所提议控制器的设计架构,并将其与经典的时变覆盖控制器进行对比,在仿真环境中用以评估覆盖性能表现。此外,我们强调提出的控制法则因其计算效率和分布式特性而特别适用于涉及时间变化覆盖控制问题的多车辆机器人应用场合。我们在使用无人机群进行气团监测的应用中采用了这种方法。通过实地试验,展示了由所提议控制器引导的无人机能够以分散的方式追踪模拟的时间变化化学烟羽。 此段落概述了论文的主要贡献及实验验证情况,说明了一种新型时间可变覆盖控制方法的设计与应用成效。
https://arxiv.org/abs/2507.18938
In recent years, Compressed Sensing (CS) has gained significant interest as a technique for acquiring high-resolution sensory data using fewer measurements than traditional Nyquist sampling requires. At the same time, autonomous robotic platforms such as drones and rovers have become increasingly popular tools for remote sensing and environmental monitoring tasks, including measurements of temperature, humidity, and air quality. Within this context, this paper presents, to the best of our knowledge, the first investigation into how the structure of CS measurement matrices can be exploited to design optimized sampling trajectories for robotic environmental data collection. We propose a novel Monte Carlo optimization framework that generates measurement matrices designed to minimize both the robot's traversal path length and the signal reconstruction error within the CS framework. Central to our approach is the application of Dictionary Learning (DL) to obtain a data-driven sparsifying transform, which enhances reconstruction accuracy while further reducing the number of samples that the robot needs to collect. We demonstrate the effectiveness of our method through experiments reconstructing $NO_2$ pollution maps over the Gulf region. The results indicate that our approach can reduce robot travel distance to less than $10\%$ of a full-coverage path, while improving reconstruction accuracy by over a factor of five compared to traditional CS methods based on DCT and polynomial dictionaries, as well as by a factor of two compared to previously-proposed Informative Path Planning (IPP) methods.
近年来,压缩感知(Compressed Sensing, CS)作为一种技术引起了广泛兴趣,它能够使用比传统Nyquist采样所需的更少的测量次数来获取高分辨率的感觉数据。与此同时,自主机器人平台如无人机和探测车已成为远程传感和环境监测任务中越来越受欢迎的工具,这些任务包括温度、湿度和空气质量等参数的测量。在此背景下,本文首次(据我们所知)探讨了如何利用压缩感知测量矩阵的结构来为机器人环保数据收集设计优化的采样轨迹。我们提出了一种新颖的蒙特卡洛优化框架,该框架生成旨在同时最小化机器人的行进路径长度和信号重建误差的测量矩阵,在压缩感知框架内进行操作。 我们的方法的核心是应用字典学习(Dictionary Learning, DL)以获得一种数据驱动的稀疏变换,这不仅增强了重构准确性,还进一步减少了机器人所需收集的数据样本数量。通过在海湾地区$NO_2$污染图的重建实验中展示了我们方法的有效性。结果显示,与传统的基于DCT和多项式字典的方法相比,我们的方法将机器人的行进距离减少到了完全覆盖路径长度的大约10%以内,并提高了五倍以上的重构精度;而相较于之前提出的知情路径规划(Informative Path Planning, IPP)方法,则提高了两倍的重构准确性。
https://arxiv.org/abs/2507.18462
Autonomous drones have gained considerable attention for applications in real-world scenarios, such as search and rescue, inspection, and delivery. As their use becomes ever more pervasive in civilian applications, failure to ensure safe operation can lead to physical damage to the system, environmental pollution, and even loss of human life. Recent work has demonstrated that motion planning techniques effectively generate a collision-free trajectory during navigation. However, these methods, while creating the motion plans, do not inherently consider the safe operational region of the system, leading to potential safety constraints violation during deployment. In this paper, we propose a method that leverages run time safety assurance in a kinodynamic motion planning scheme to satisfy the system's operational constraints. First, we use a sampling-based geometric planner to determine a high-level collision-free path within a user-defined space. Second, we design a low-level safety assurance filter to provide safety guarantees to the control input of a Linear Quadratic Regulator (LQR) designed with the purpose of trajectory tracking. We demonstrate our proposed approach in a restricted 3D simulation environment using a model of the Crazyflie 2.0 drone.
自主无人机在现实场景中的应用,如搜索救援、检查和配送等领域引起了广泛关注。随着其在民用领域的使用越来越普遍,若不能确保安全操作,则可能导致系统损坏、环境污染甚至人员伤亡。近期的研究表明,运动规划技术能有效生成导航过程中的无碰撞轨迹。然而,在创建这些运动计划时,并未固有地考虑系统的安全运行区域,这可能会导致部署过程中出现安全性约束的违规情况。 在本文中,我们提出了一种方法,该方法通过在动力学运动规划方案中利用实时安全保证来满足系统操作约束的要求。首先,我们使用基于采样的几何规划器,在用户定义的空间内确定一条高层次的无碰撞路径;其次,我们设计了一个低层次的安全保障过滤器,为线性二次调节器(LQR)提供控制输入的安全保证,该调节器旨在实现轨迹跟踪。 我们在一个受限的3D仿真环境中,使用Crazyflie 2.0无人机模型展示了我们的方法。
https://arxiv.org/abs/2507.17679
With the rise of robotics, LiDAR-based 3D object detection has garnered significant attention in both academia and industry. However, existing datasets and methods predominantly focus on vehicle-mounted platforms, leaving other autonomous platforms underexplored. To bridge this gap, we introduce Pi3DET, the first benchmark featuring LiDAR data and 3D bounding box annotations collected from multiple platforms: vehicle, quadruped, and drone, thereby facilitating research in 3D object detection for non-vehicle platforms as well as cross-platform 3D detection. Based on Pi3DET, we propose a novel cross-platform adaptation framework that transfers knowledge from the well-studied vehicle platform to other platforms. This framework achieves perspective-invariant 3D detection through robust alignment at both geometric and feature levels. Additionally, we establish a benchmark to evaluate the resilience and robustness of current 3D detectors in cross-platform scenarios, providing valuable insights for developing adaptive 3D perception systems. Extensive experiments validate the effectiveness of our approach on challenging cross-platform tasks, demonstrating substantial gains over existing adaptation methods. We hope this work paves the way for generalizable and unified 3D perception systems across diverse and complex environments. Our Pi3DET dataset, cross-platform benchmark suite, and annotation toolkit have been made publicly available.
随着机器人技术的兴起,基于LiDAR的三维物体检测在学术界和工业界都获得了广泛关注。然而,现有的数据集和方法主要集中在车载平台,而其他自主平台(如四足机器人和无人机)的研究则相对较少。为了弥补这一空白,我们推出了Pi3DET,这是第一个包含来自多个平台(车辆、四足机器人和无人机)的LiDAR数据及三维边界框标注的数据基准,从而促进非车载平台以及跨平台三维检测研究的发展。 基于Pi3DET,我们提出了一种新颖的跨平台适应框架,该框架可以从研究充分的车载平台转移知识到其他平台。通过在几何层面和特征层面上实现稳健对齐,我们的框架能够实现在不同视角下的不变性三维检测。此外,我们还建立了一个基准来评估现有三维检测器在跨平台场景中的抗干扰性和鲁棒性,为开发适应性强的三维感知系统提供了宝贵的见解。 广泛的实验验证了我们在具有挑战性的跨平台任务上的方法的有效性,并证明相比现有的适应方法,我们的方法取得了显著的进步。我们希望这项工作能够为各种复杂环境下的通用化和统一化的三维感知系统铺平道路。 Pi3DET数据集、跨平台基准测试套件以及标注工具包现已公开发布。
https://arxiv.org/abs/2507.17665
Precise modeling of microscopic vehicle trajectories is critical for traffic behavior analysis and autonomous driving systems. We propose Ctx2TrajGen, a context-aware trajectory generation framework that synthesizes realistic urban driving behaviors using GAIL. Leveraging PPO and WGAN-GP, our model addresses nonlinear interdependencies and training instability inherent in microscopic settings. By explicitly conditioning on surrounding vehicles and road geometry, Ctx2TrajGen generates interaction-aware trajectories aligned with real-world context. Experiments on the drone-captured DRIFT dataset demonstrate superior performance over existing methods in terms of realism, behavioral diversity, and contextual fidelity, offering a robust solution to data scarcity and domain shift without simulation.
微观车辆轨迹的精确建模对于交通行为分析和自主驾驶系统至关重要。我们提出了一种基于GAIL(生成对抗模仿学习)的Ctx2TrajGen框架,该框架能够利用周围环境信息合成现实的城市驾驶行为。通过结合PPO(策略梯度方法)和WGAN-GP(带梯度惩罚的生成对抗网络),我们的模型解决了微观场景中固有的非线性相互依赖关系和训练不稳定问题。Ctx2TrajGen框架明确地根据周围的车辆和道路几何形状进行条件设置,从而生成与现实世界环境相一致、具有交互意识的轨迹。 在无人机捕获的DRIFT数据集上的实验表明,相较于现有的方法,Ctx2TrajGen在真实性、行为多样性以及情境一致性方面表现出优越性能。这项工作为解决数据稀缺性和领域转移问题提供了一种无需模拟的强大解决方案。
https://arxiv.org/abs/2507.17418
The increasing demand for fast and cost effective last mile delivery solutions has catalyzed significant advancements in drone based logistics. This research describes the development of an AI integrated drone delivery system, focusing on route optimization, object detection, secure package handling, and real time tracking. The proposed system leverages YOLOv4 Tiny for object detection, the NEO 6M GPS module for navigation, and the A7670 SIM module for real time communication. A comparative analysis of lightweight AI models and hardware components is conducted to determine the optimal configuration for real time UAV based delivery. Key challenges including battery efficiency, regulatory compliance, and security considerations are addressed through the integration of machine learning techniques, IoT devices, and encryption protocols. Preliminary studies demonstrate improvement in delivery time compared to conventional ground based logistics, along with high accuracy recipient authentication through facial recognition. The study also discusses ethical implications and societal acceptance of drone deliveries, ensuring compliance with FAA, EASA and DGCA regulatory standards. Note: This paper presents the architecture, design, and preliminary simulation results of the proposed system. Experimental results, simulation benchmarks, and deployment statistics are currently being acquired. A comprehensive analysis will be included in the extended version of this work.
对于快速且低成本的最后一公里配送解决方案的需求不断增加,这已经催化了基于无人机物流技术的重大进展。这项研究描述了一种集成人工智能的无人机递送系统的开发,重点在于路线优化、物体检测、安全包裹处理和实时追踪。所提出的系统利用YOLOv4 Tiny进行物体检测,使用NEO 6M GPS模块进行导航,并采用A7670 SIM模块实现实时通信。 该研究对轻量级AI模型及硬件组件进行了比较分析,以确定基于无人机的实时递送系统的最佳配置。通过整合机器学习技术、物联网设备和加密协议解决了包括电池效率、监管合规性和安全性考虑在内的关键挑战。初步研究表明,与传统的地面物流相比,交付时间有所改善,并且通过面部识别实现高精度收件人身份验证。该研究还讨论了无人机配送的伦理影响及社会接受度,确保遵守FAA(美国联邦航空管理局)、EASA(欧洲航空安全局)和DGCA(印度民航总局)的相关法规标准。 注:本文介绍了所提系统的架构、设计及初步仿真结果。实验数据、模拟基准以及部署统计信息正在收集当中,更详尽的分析将在该研究扩展版本中包括。
https://arxiv.org/abs/2507.17253
Interacting with real-world objects in Mixed Reality (MR) often proves difficult when they are crowded, distant, or partially occluded, hindering straightforward selection and manipulation. We observe that these difficulties stem from performing interaction directly on physical objects, where input is tightly coupled to their physical constraints. Our key insight is to decouple interaction from these constraints by introducing proxies-abstract representations of real-world objects. We embody this concept in Reality Proxy, a system that seamlessly shifts interaction targets from physical objects to their proxies during selection. Beyond facilitating basic selection, Reality Proxy uses AI to enrich proxies with semantic attributes and hierarchical spatial relationships of their corresponding physical objects, enabling novel and previously cumbersome interactions in MR - such as skimming, attribute-based filtering, navigating nested groups, and complex multi object selections - all without requiring new gestures or menu systems. We demonstrate Reality Proxy's versatility across diverse scenarios, including office information retrieval, large-scale spatial navigation, and multi-drone control. An expert evaluation suggests the system's utility and usability, suggesting that proxy-based abstractions offer a powerful and generalizable interaction paradigm for future MR systems.
在混合现实(MR)中与真实世界对象进行交互常常因为这些对象过于拥挤、距离较远或部分被遮挡而变得困难,这阻碍了直接的选择和操作。我们观察到这些问题的根源在于直接在物理对象上执行交互时输入与其物理约束紧密耦合。我们的关键见解是通过引入代理——即现实世界对象的抽象表示来解除这种约束,从而使交互更加灵活。 Reality Proxy 是一个实现这一概念的系统,它可以在选择过程中将互动目标从实际物体无缝转移到它们对应的代理(proxies)。除了简化基本的选择操作之外,Reality Proxy 还使用人工智能技术为这些代理添加与相应物理对象相关的语义属性和空间层次关系。这使得在 MR 中进行诸如浏览、基于属性筛选、导航嵌套组以及复杂多对象选择等新型且以前难以实现的交互成为可能,并且无需引入新的手势或菜单系统。 我们展示了 Reality Proxy 在各种场景中的灵活性,包括办公信息检索、大规模空间导航和多无人机控制。专家评估表明该系统的实用性和易用性,并暗示基于代理(proxies)的抽象为未来的 MR 系统提供了一种强大而通用化的交互方式。
https://arxiv.org/abs/2507.17248