Although large scale models achieve significant improvements in performance, the overfitting challenge still frequently undermines their generalization ability. In super resolution tasks on images, diffusion models as representatives of generative models typically adopt large scale architectures. However, few-shot drone-captured infrared training data frequently induces severe overfitting in large-scale architectures. To address this key challenge, our method proposes a new Gaussian quantization representation learning method oriented to diffusion models that alleviates overfitting and enhances robustness. At the same time, an effective monitoring mechanism tracks large scale architectures during training to detect signs of overfitting. By introducing Gaussian quantization representation learning, our method effectively reduces overfitting while maintaining architecture complexity. On this basis, we construct a multi source drone-based infrared image benchmark dataset for detection and use it to emphasize overfitting issues of large scale architectures in few sample, drone-based diverse drone-based image reconstruction scenarios. To verify the efficacy of the method in mitigating overfitting, experiments are conducted on the constructed benchmark. Experimental results demonstrate that our method outperforms existing super resolution approaches and significantly mitigates overfitting of large scale architectures under complex conditions. The code and DroneSR dataset will be available at: this https URL.
尽管大规模模型在性能上取得了显著的改进,但过拟合的问题依然频繁地损害了它们的泛化能力。特别是在图像超分辨率任务中,作为生成式模型代表的扩散模型通常采用大规模架构。然而,在使用少量无人机捕捉到的红外训练数据的情况下,这类大型架构经常会引发严重的过拟合问题。为解决这一关键挑战,我们的方法提出了一种针对扩散模型的新型高斯量化表征学习方法,以减轻过拟合并增强鲁棒性。同时,我们设计了一个有效的监控机制,在训练过程中追踪大规模架构,以便及时发现过拟合的迹象。通过引入高斯量化表征学习,我们的方法在不增加架构复杂度的情况下有效减少了过拟合现象。 在此基础上,我们构建了一个基于多源无人机红外图像的数据集(用于检测),并在该数据集中突出展示了大规模架构在少样本、多样化无人机捕捉场景中的重建任务中所面临的过拟合问题。为了验证该方法缓解过拟合的有效性,在构造的基准测试上进行了实验。实验结果表明,我们的方法优于现有的超分辨率方法,并显著减轻了复杂条件下的大规模模型的过拟合现象。 相关代码和DroneSR数据集可在以下链接获取:[提供具体的网址]
https://arxiv.org/abs/2509.01898
The trajectory data of traffic participants (TPs) is a fundamental resource for evaluating traffic conditions and optimizing policies, especially at urban intersections. Although data acquisition using drones is efficient, existing datasets still have limitations in scene representativeness, information richness, and data fidelity. This study introduces FLUID, comprising a fine-grained trajectory dataset that captures dense conflicts at typical urban signalized intersections, and a lightweight, full-pipeline framework for drone-based trajectory processing. FLUID covers three distinct intersection types, with approximately 5 hours of recording time and featuring over 20,000 TPs across 8 categories. Notably, the dataset averages two vehicle conflicts per minute, involving roughly 25% of all motor vehicles. FLUID provides comprehensive data, including trajectories, traffic signals, maps, and raw videos. Comparison with the DataFromSky platform and ground-truth measurements validates its high spatio-temporal accuracy. Through a detailed classification of motor vehicle conflicts and violations, FLUID reveals a diversity of interactive behaviors, demonstrating its value for human preference mining, traffic behavior modeling, and autonomous driving research.
交通参与者(TPs)的轨迹数据是评估交通状况和优化政策的重要资源,尤其是在城市交叉路口。虽然使用无人机采集数据效率很高,但现有的数据集在场景代表性、信息丰富性和数据真实性方面仍然存在局限性。本研究介绍了FLUID,它包含一个精细粒度的轨迹数据集,该数据集捕捉了典型城市信号交叉口密集冲突的情况,以及一套轻量级、全流程框架用于基于无人机的轨迹处理。FLUID涵盖了三种不同类型的交叉路口,总记录时间约为5小时,并且包括超过20,000个TPs(分为8类)。值得注意的是,该数据集平均每分钟包含两个车辆冲突事件,涉及大约25%的所有机动车。 FLUID提供了全面的数据,包括轨迹、交通信号、地图和原始视频。与DataFromSky平台及地面实况测量进行比较验证了其高时空准确性。通过对机动车辆冲突和违规行为的详细分类,FLUID揭示了多样化的交互行为模式,展示了其在挖掘人类偏好、交通行为建模以及自动驾驶研究方面的价值。
https://arxiv.org/abs/2509.00497
With the deepening of poverty alleviation and rural revitalization strategies, improving the rural living environment and enhancing the quality of life have become key priorities. Rural livability is a key indicator for measuring the effectiveness of these efforts. Current measurement approaches face significant limitations, as questionnaire-based methods are difficult to scale, while urban-oriented visual perception methods are poorly suited for rural contexts. In this paper, a rural-specific livability assessment framework was proposed based on drone imagery and multimodal large language models (MLLMs). To comprehensively assess village livability, this study first used a top-down approach to collect large-scale drone imagery of 1,766 villages in 146 counties across China. In terms of the model framework, an efficient image comparison mechanism was developed, incorporating binary search interpolation to determine effective image pairs while reducing comparison iterations. Building on expert knowledge, a chain-of-thought prompting suitable for nationwide rural livability measurement was constructed, considering both living quality and ecological habitability dimensions. This approach enhanced the rationality and reliability of the livability assessment. Finally, this study characterized the spatial heterogeneity of rural livability across China and thoroughly analyzed its influential factors. The results show that: (1) The rural livability in China demonstrates a dual-core-periphery spatial pattern, radiating outward from Sichuan and Zhejiang provinces with declining gradients; (2) Among various influential factors, government fiscal expenditure emerged as the core determinant, with each unit increase corresponding to a 3.9 - 4.9 unit enhancement in livability. The findings provide valuable insights for rural construction policy-making.
随着扶贫和乡村振兴战略的深化,改善农村生活环境、提高生活质量已成为关键优先事项。乡村宜居性成为衡量这些努力成效的重要指标。然而,现有的测量方法存在显著局限:基于问卷的方法难以大规模应用;面向城市的视觉感知方法则不适用于农村环境。为此,本文提出了一种基于无人机影像和多模态大型语言模型(MLLM)的特定农村宜居性评估框架。 为了全面评估村庄的宜居性,本研究首先采用自上而下的方式收集了中国146个县内1,766个村庄的大规模无人机图像。在模型框架方面,开发了一种高效的影像比较机制,并引入二分查找插值来确定有效的影像对同时减少比较迭代次数。 基于专家知识,构建了一个适合全国农村宜居性测量的链式思维提示,该提示考虑了生活质量与生态适宜性两个维度,从而提高了宜居性评估的合理性和可靠性。最后,本研究描述了中国各地乡村宜居性的空间异质性,并对其影响因素进行了全面分析。结果显示: 1. 中国的农村宜居性展现出双核心-外围的空间模式,从四川省和浙江省向外辐射,逐渐衰减; 2. 在各种影响因素中,政府财政支出成为决定性因素,每单位增加会带来3.9至4.9个单位的宜居性提升。 这些发现为乡村建设政策制定提供了宝贵的见解。
https://arxiv.org/abs/2508.21738
Traversing narrow gates presents a significant challenge and has become a standard benchmark for evaluating agile and precise quadrotor flight. Traditional modularized autonomous flight stacks require extensive design and parameter tuning, while end-to-end reinforcement learning (RL) methods often suffer from low sample efficiency and limited interpretability. In this work, we present a novel hybrid framework that adaptively fine-tunes model predictive control (MPC) parameters online using outputs from a neural network (NN) trained offline. The NN jointly predicts a reference pose and cost-function weights, conditioned on the coordinates of the gate corners and the current drone state. To achieve efficient training, we derive analytical policy gradients not only for the MPC module but also for an optimization-based gate traversal detection module. Furthermore, we introduce a new formulation of the attitude tracking error that admits a simplified representation, facilitating effective learning with bounded gradients. Hardware experiments demonstrate that our method enables fast and accurate quadrotor traversal through narrow gates in confined environments. It achieves several orders of magnitude improvement in sample efficiency compared to naive end-to-end RL approaches.
穿越狭窄的门洞提出了一个重要的挑战,并已成为评估四旋翼机灵巧和精确飞行的标准基准。传统的模块化自主飞行堆栈需要大量的设计和参数调整,而端到端强化学习(RL)方法通常在样本效率低且解释性有限方面存在问题。 在这项工作中,我们提出了一种新颖的混合框架,该框架能够在线自适应地微调模型预测控制(MPC)参数,利用一个经过离线训练的神经网络(NN)的输出。这个神经网络同时根据门洞角点坐标和当前无人机状态来预测参考姿态和代价函数权重。为了实现高效的训练,我们推导出了分析性策略梯度,不仅针对MPC模块,还针对基于优化的门洞穿越检测模块。 此外,我们提出了一种新的姿态跟踪误差公式化方法,该方法允许简化表示,并且具有有界梯度,从而有助于有效的学习过程。硬件实验表明,我们的方法使四旋翼机能够快速准确地在狭窄环境中穿过门洞。与简单的端到端RL方法相比,它在样本效率方面提高了几个数量级。
https://arxiv.org/abs/2508.21592
Natural Language-Guided Drones (NLGD) provide a novel paradigm for tasks such as target matching and navigation. However, the wide field of view and complex compositional semantics in drone scenarios pose challenges for vision-language understanding. Mainstream Vision-Language Models (VLMs) emphasize global alignment while lacking fine-grained semantics, and existing hierarchical methods depend on precise entity partitioning and strict containment, limiting effectiveness in dynamic environments. To address this, we propose the Hierarchical Cross-Granularity Contrastive and Matching learning (HCCM) framework with two components: (1) Region-Global Image-Text Contrastive Learning (RG-ITC), which avoids precise scene partitioning and captures hierarchical local-to-global semantics by contrasting local visual regions with global text and vice versa; (2) Region-Global Image-Text Matching (RG-ITM), which dispenses with rigid constraints and instead evaluates local semantic consistency within global cross-modal representations, enhancing compositional reasoning. Moreover, drone text descriptions are often incomplete or ambiguous, destabilizing alignment. HCCM introduces a Momentum Contrast and Distillation (MCD) mechanism to improve robustness. Experiments on GeoText-1652 show HCCM achieves state-of-the-art Recall@1 of 28.8% (image retrieval) and 14.7% (text retrieval). On the unseen ERA dataset, HCCM demonstrates strong zero-shot generalization with 39.93% mean recall (mR), outperforming fine-tuned baselines.
自然语言引导的无人机(NLGD)为诸如目标匹配和导航等任务提供了一种新颖的方法。然而,无人机场景中的宽广视野和复杂的组合语义对视觉-语言理解提出了挑战。主流的视觉-语言模型(VLMs)强调全局一致性,但缺乏细粒度语义的理解,并且现有的层次方法依赖于精确的对象分割和严格的包含关系,在动态环境中效果受限。为了解决这些问题,我们提出了一种分层跨粒度对比与匹配学习(HCCM)框架,该框架由两个部分组成: 1. 区域-全局图像文本对比学习(RG-ITC),它避免了精确场景分割的需求,并通过局部视觉区域和全局文本之间的对比捕获从局部到全局的层次化语义信息; 2. 区域-全局图像文本匹配(RG-ITM),该方法不再依赖于严格的约束,而是评估全局跨模态表示中的局部语义一致性,从而增强组合推理能力。 此外,在无人机场景中,由于描述常常不完整或含糊不清,使得对齐过程不稳定。为提高鲁棒性,HCCM引入了动量对比和蒸馏(MCD)机制。在GeoText-1652数据集上的实验表明,HCCM实现了最先进的Recall@1性能:图像检索中为28.8%,文本检索中为14.7%。此外,在未见过的ERA数据集中,HCCM展示了强大的零样本泛化能力,平均召回率(mR)达到39.93%,优于微调后的基线模型。
https://arxiv.org/abs/2508.21539
Cross-view geo-localization is a critical task for UAV navigation, event detection, and aerial surveying, as it enables matching between drone-captured and satellite imagery. Most existing approaches embed multi-modal data into a joint feature space to maximize the similarity of paired images. However, these methods typically assume perfect alignment of image pairs during training, which rarely holds true in real-world scenarios. In practice, factors such as urban canyon effects, electromagnetic interference, and adverse weather frequently induce GPS drift, resulting in systematic alignment shifts where only partial correspondences exist between pairs. Despite its prevalence, this source of noisy correspondence has received limited attention in current research. In this paper, we formally introduce and address the Noisy Correspondence on Cross-View Geo-Localization (NC-CVGL) problem, aiming to bridge the gap between idealized benchmarks and practical applications. To this end, we propose PAUL (Partition and Augmentation by Uncertainty Learning), a novel framework that partitions and augments training data based on estimated data uncertainty through uncertainty-aware co-augmentation and evidential co-training. Specifically, PAUL selectively augments regions with high correspondence confidence and utilizes uncertainty estimation to refine feature learning, effectively suppressing noise from misaligned pairs. Distinct from traditional filtering or label correction, PAUL leverages both data uncertainty and loss discrepancy for targeted partitioning and augmentation, thus providing robust supervision for noisy samples. Comprehensive experiments validate the effectiveness of individual components in PAUL,which consistently achieves superior performance over other competitive noisy-correspondence-driven methods in various noise ratios.
跨视角地理定位是无人机导航、事件检测和航空测绘中的关键任务,因为它能够将无人机拍摄的图像与卫星图像进行匹配。现有大多数方法都将多模态数据嵌入到一个共同特征空间中以最大化配对图像之间的相似性。然而,这些方法通常假设在训练过程中图像对具有完美的对齐情况,在实际场景中这种理想状态很少出现。实际上,诸如城市峡谷效应、电磁干扰和恶劣天气等因素经常导致GPS漂移,造成系统性的定位偏差,使得图像对之间仅存在部分对应关系。尽管这种情况普遍存在于实践中,但在当前研究中这一类噪声对应问题却受到了较少的关注。 本文正式引入并解决了跨视角地理定位中的噪声对应问题(NC-CVGL),旨在弥合理想化基准与实际应用之间的差距。为此,我们提出了PAUL(不确定性学习的分区和增强)框架,通过利用不确定性感知协同增强和证据协同训练来估计数据的不确定性,并据此对训练数据进行分区和增强。具体而言,PAUL选择性地增强具有高对应置信度的区域,并利用不确定性估计来改进特征学习,有效地抑制了来自错位图像对中的噪声。不同于传统的过滤或标签修正方法,PAUL同时使用数据不确定性和损失差异来进行有针对性的数据分割和增强,从而为带有噪声样本提供稳健的监督。 全面的实验验证了PAUL各个组成部分的有效性,在不同的噪声比例下,PAUL始终优于其他竞争性的基于噪声对应的方法。
https://arxiv.org/abs/2508.20066
Monitoring coral reefs at large spatial scales remains an open challenge, essential for assessing ecosystem health and informing conservation efforts. While drone-based aerial imagery offers broad spatial coverage, its limited resolution makes it difficult to reliably distinguish fine-scale classes, such as coral morphotypes. At the same time, obtaining pixel-level annotations over large spatial extents is costly and labor-intensive, limiting the scalability of deep learning-based segmentation methods for aerial imagery. We present a multi-scale weakly supervised semantic segmentation framework that addresses this challenge by transferring fine-scale ecological information from underwater imagery to aerial data. Our method enables large-scale coral reef mapping from drone imagery with minimal manual annotation, combining classification-based supervision, spatial interpolation and self-distillation techniques. We demonstrate the efficacy of the approach, enabling large-area segmentation of coral morphotypes and demonstrating flexibility for integrating new classes. This study presents a scalable, cost-effective methodology for high-resolution reef monitoring, combining low-cost data collection, weakly supervised deep learning and multi-scale remote sensing.
监测大型空间范围内的珊瑚礁仍然是一个开放性的挑战,这对于评估生态系统健康和指导保护工作至关重要。虽然基于无人机的空中影像提供了广阔的地理覆盖范围,但其有限的分辨率使得难以可靠地区分如珊瑚形态等细小尺度类别。同时,在大规模范围内获取像素级标注既昂贵又耗时,这限制了深度学习分割方法的大规模应用。 我们提出了一种多尺度弱监督语义分割框架,通过将水下影像中的精细生态信息转移到空中数据中来解决这一挑战。我们的方法能够利用无人机影像对珊瑚礁进行大范围测绘,并且只需要少量的人工标注,结合分类驱动的监督、空间插值和自我蒸馏技术。我们展示了这种方法的有效性,使其能够在大面积范围内分割出珊瑚形态类型,并证明了其在整合新类别方面的灵活性。 本研究提供了一种可扩展且成本效益高的方法来实现高分辨率礁石监测,该方法结合了低成本的数据收集、弱监督的深度学习和多尺度遥感技术。
https://arxiv.org/abs/2508.18958
Thermal anomaly detection in solar photovoltaic (PV) systems is essential for ensuring operational efficiency and reducing maintenance costs. In this study, we developed and named HOTSPOT-YOLO, a lightweight artificial intelligence (AI) model that integrates an efficient convolutional neural network backbone and attention mechanisms to improve object detection. This model is specifically designed for drone-based thermal inspections of PV systems, addressing the unique challenges of detecting small and subtle thermal anomalies, such as hotspots and defective modules, while maintaining real-time performance. Experimental results demonstrate a mean average precision of 90.8%, reflecting a significant improvement over baseline object detection models. With a reduced computational load and robustness under diverse environmental conditions, HOTSPOT-YOLO offers a scalable and reliable solution for large-scale PV inspections. This work highlights the integration of advanced AI techniques with practical engineering applications, revolutionizing automated fault detection in renewable energy systems.
太阳能光伏(PV)系统中的热异常检测对于确保运行效率和减少维护成本至关重要。在这项研究中,我们开发并命名了一种轻量级人工智能模型HOTSPOT-YOLO,该模型结合了高效的卷积神经网络骨干网和注意力机制,以提高物体检测性能。HOTSPOT-YOLO特别为基于无人机的光伏系统的热检查而设计,能够解决在实时性能下检测小且细微的热异常(如热点和故障模块)的独特挑战。实验结果显示,该模型平均精度达到了90.8%,显著优于基线目标检测模型的表现。通过降低计算负荷并在多种环境条件下保持鲁棒性,HOTSPOT-YOLO为大规模光伏检查提供了可扩展且可靠的解决方案。这项工作展示了高级AI技术与实际工程应用的结合,正在改变可再生能源系统中的自动故障检测方式。
https://arxiv.org/abs/2508.18912
Preference-based reinforcement learning (PbRL) enables agents to learn control policies without requiring manually designed reward functions, making it well-suited for tasks where objectives are difficult to formalize or inherently subjective. Acrobatic flight poses a particularly challenging problem due to its complex dynamics, rapid movements, and the importance of precise execution. In this work, we explore the use of PbRL for agile drone control, focusing on the execution of dynamic maneuvers such as powerloops. Building on Preference-based Proximal Policy Optimization (Preference PPO), we propose Reward Ensemble under Confidence (REC), an extension to the reward learning objective that improves preference modeling and learning stability. Our method achieves 88.4% of the shaped reward performance, compared to 55.2% with standard Preference PPO. We train policies in simulation and successfully transfer them to real-world drones, demonstrating multiple acrobatic maneuvers where human preferences emphasize stylistic qualities of motion. Furthermore, we demonstrate the applicability of our probabilistic reward model in a representative MuJoCo environment for continuous control. Finally, we highlight the limitations of manually designed rewards, observing only 60.7% agreement with human preferences. These results underscore the effectiveness of PbRL in capturing complex, human-centered objectives across both physical and simulated domains.
基于偏好的强化学习(PbRL)使智能体能够在无需手动设计奖励函数的情况下学习控制策略,因此非常适合目标难以形式化或本质上具有主观性的任务。特技飞行因其复杂的动力学特性、快速的运动以及执行精确度的重要性而构成一个特别具挑战性的问题。在这项工作中,我们探讨了将PbRL用于敏捷无人机控制的可能性,并专注于动态机动动作(如螺旋筋斗)的执行。在基于偏好的近端策略优化(Preference PPO)的基础上,我们提出了置信度下的奖励集成(REC),这是一种对奖励学习目标进行扩展的方法,能够改进偏好建模和学习稳定性。我们的方法实现了88.4%的塑形奖励性能,而标准的Preference PPO仅实现55.2%。 我们在模拟环境中训练策略,并成功将其转移到真实的无人机上,展示了多个强调运动风格质量的人类偏好的特技动作。此外,我们还在一个具有代表性的MuJoCo环境中演示了我们的概率性奖励模型在连续控制任务中的适用性。最后,我们指出了手动设计奖励的局限性,观察到仅达到60.7%与人类偏好一致的程度。 这些结果强调了PbRL在捕捉复杂且以人为中心的目标方面的有效性,无论是在物理环境还是模拟环境中都是如此。
https://arxiv.org/abs/2508.18817
As drone use has become more widespread, there is a critical need to ensure safety and security. A key element of this is robust and accurate drone detection and localization. While cameras and other optical sensors like LiDAR are commonly used for object detection, their performance degrades under adverse lighting and environmental conditions. Therefore, this has generated interest in finding more reliable alternatives, such as millimeter-wave (mmWave) radar. Recent research on mmWave radar object detection has predominantly focused on 2D detection of road users. Although these systems demonstrate excellent performance for 2D problems, they lack the sensing capability to measure elevation, which is essential for 3D drone detection. To address this gap, we propose CubeDN, a single-stage end-to-end radar object detection network specifically designed for flying drones. CubeDN overcomes challenges such as poor elevation resolution by utilizing a dual radar configuration and a novel deep learning pipeline. It simultaneously detects, localizes, and classifies drones of two sizes, achieving decimeter-level tracking accuracy at closer ranges with overall $95\%$ average precision (AP) and $85\%$ average recall (AR). Furthermore, CubeDN completes data processing and inference at 10Hz, making it highly suitable for practical applications.
随着无人机使用的普及,确保其安全性和安全性变得至关重要。其中一个重要方面是实现强大而准确的无人机检测和定位。虽然相机和其他光学传感器(如激光雷达)常用于物体检测,但它们在恶劣光照和环境条件下性能会下降。因此,人们开始对寻找更可靠的替代方案产生了兴趣,例如毫米波(mmWave)雷达。近年来关于毫米波雷达物体检测的研究主要集中在道路使用者的二维检测上。尽管这些系统在解决二维问题方面表现出色,但由于无法测量高度信息,它们缺乏进行三维无人机检测所需的感知能力。为了解决这一缺口,我们提出了CubeDN,这是一种专为飞行中的无人机设计的一阶段端到端毫米波雷达物体检测网络。 CubeDN 通过采用双雷达配置和新颖的深度学习管道来克服较差的高度分辨率等问题,在同时实现对两种尺寸无人机的检测、定位及分类方面表现出色。它在较近距离范围内实现了分米级跟踪精度,整体平均精度(AP)达到95%,平均召回率(AR)为85%。此外,CubeDN 能够以10Hz 的频率完成数据处理和推理工作,使其非常适用于实际应用。
https://arxiv.org/abs/2508.17831
Estimating the 3D pose of a drone is important for anti-drone systems, but existing methods struggle with the unique challenges of drone keypoint detection. Drone propellers serve as keypoints but are difficult to detect due to their high visual similarity and diversity of poses. To address these challenges, we propose DroneKey, a framework that combines a 2D keypoint detector and a 3D pose estimator specifically designed for drones. In the keypoint detection stage, we extract two key-representations (intermediate and compact) from each transformer encoder layer and optimally combine them using a gated sum. We also introduce a pose-adaptive Mahalanobis distance in the loss function to ensure stable keypoint predictions across extreme poses. We built new datasets of drone 2D keypoints and 3D pose to train and evaluate our method, which have been publicly released. Experiments show that our method achieves an AP of 99.68% (OKS) in keypoint detection, outperforming existing methods. Ablation studies confirm that the pose-adaptive Mahalanobis loss function improves keypoint prediction stability and accuracy. Additionally, improvements in the encoder design enable real-time processing at 44 FPS. For 3D pose estimation, our method achieved an MAE-angle of 10.62°, an RMSE of 0.221m, and an MAE-absolute of 0.076m, demonstrating high accuracy and reliability. The code and dataset are available at this https URL.
估算无人机的3D姿态对于反无人机系统至关重要,但现有的方法在处理无人机关键点检测的独特挑战时遇到了困难。无人机螺旋桨作为关键点,但由于其视觉相似性和姿势多样性的原因,它们难以被准确识别出来。为了解决这些挑战,我们提出了DroneKey框架,该框架结合了专门针对无人机设计的2D关键点检测器和3D姿态估计器。 在关键点检测阶段,我们从每个Transformer编码层中提取两种关键表示(中间和紧凑型),并通过门控求和来优化地将它们结合起来。此外,我们在损失函数中引入了一种姿势自适应Mahalanobis距离,以确保极端姿态下的稳定的关键点预测。 为了训练和评估我们的方法,我们建立并公开发布了一系列无人机2D关键点及3D姿态的新数据集。实验结果表明,在关键点检测方面(OKS标准),我们的方法达到了99.68%的AP值,超过了现有技术的表现。消融研究证实了姿势自适应Mahalanobis损失函数在提高关键点预测稳定性和准确性方面的有效性。 此外,编码器设计的改进使得实时处理能力达到了每秒44帧。对于3D姿态估计,我们的方法实现了10.62°的MAE角度误差、0.221m的RMSE以及0.076m的MAE绝对值误差,展示了其高准确性和可靠性。 代码和数据集可以在以下链接获取:[提供链接]
https://arxiv.org/abs/2508.17746
Ultimate is a sport where points are scored by passing a disc and catching it in the opposing team's end zone. In Ultimate, the player holding the disc cannot move, making field dynamics primarily driven by other players' movements. However, current literature in team sports has ignored quantitative evaluations of when players initiate such unlabeled movements in game situations. In this paper, we propose a quantitative evaluation method for movement initiation timing in Ultimate Frisbee. First, game footage was recorded using a drone camera, and players' positional data was obtained, which will be published as UltimateTrack dataset. Next, players' movement initiations were detected, and temporal counterfactual scenarios were generated by shifting the timing of movements using rule-based approaches. These scenarios were analyzed using a space evaluation metric based on soccer's pitch control reflecting the unique rules of Ultimate. By comparing the spatial evaluation values across scenarios, the difference between actual play and the most favorable counterfactual scenario was used to quantitatively assess the impact of movement timing. We validated our method and show that sequences in which the disc was actually thrown to the receiver received higher evaluation scores than the sequences without a throw. In practical verifications, the higher-skill group displays a broader distribution of time offsets from the model's optimal initiation point. These findings demonstrate that the proposed metric provides an objective means of assessing movement initiation timing, which has been difficult to quantify in unlabeled team sport plays.
《极限飞盘运动中发起移动时机的定量评估方法》 终极(Ultimate)是一种通过传球并让对方队伍在对手端区接住碟子来得分的团队体育项目。在极限飞盘中,持球者不能移动,因此比赛中的动态主要由其他球员的动作驱动。然而,目前关于团队运动的研究忽视了对比赛中未标注动作发起时机进行定量评估的重要性。本文提出了一种用于极限飞盘运动中运动员启动移动时机的量化评价方法。 首先,我们使用无人机相机记录了比赛视频,并获取了球员的位置数据,这些数据将作为UltimateTrack 数据集发布。接着,检测了玩家的移动发起时刻,并通过基于规则的方法生成时间偏移后的假设场景。我们将根据足球的场地控制理论来分析这些场景的空间评估指标,同时考虑到极限飞盘的独特规则。通过对不同情景下的空间评价值进行比较,我们使用实际比赛与最佳可能的反事实情景之间的差异来进行运动时机影响的量化评估。 我们的方法经过验证并展示了在实际传球给接球者的序列中获得了更高的评价分数。此外,在实践中,高水平团队显示的时间偏移分布更加广泛,这表明他们更接近模型预测的最佳发起时间点。 这些发现证明了提出的度量标准为难以量化的无标签团队运动的启动时机提供了一种客观评估手段。
https://arxiv.org/abs/2508.17611
Adapting trajectories to dynamic situations and user preferences is crucial for robot operation in unstructured environments with non-expert users. Natural language enables users to express these adjustments in an interactive manner. We introduce OVITA, an interpretable, open-vocabulary, language-driven framework designed for adapting robot trajectories in dynamic and novel situations based on human instructions. OVITA leverages multiple pre-trained Large Language Models (LLMs) to integrate user commands into trajectories generated by motion planners or those learned through demonstrations. OVITA employs code as an adaptation policy generated by an LLM, enabling users to adjust individual waypoints, thus providing flexible control. Another LLM, which acts as a code explainer, removes the need for expert users, enabling intuitive interactions. The efficacy and significance of the proposed OVITA framework is demonstrated through extensive simulations and real-world environments with diverse tasks involving spatiotemporal variations on heterogeneous robotic platforms such as a KUKA IIWA robot manipulator, Clearpath Jackal ground robot, and CrazyFlie drone.
适应动态情况和用户偏好的轨迹对于非专业用户在无结构环境中操作机器人至关重要。自然语言使用户能够以互动方式表达这些调整。我们介绍了OVITA,这是一种可解释的、开放式词汇的语言驱动框架,旨在根据人类指令来适应机器人在动态及新颖情景中的轨迹。OVITA利用多个预训练的大规模语言模型(LLMs)将用户的命令集成到由运动规划器生成或通过演示学习得到的轨迹中。 OVITA采用代码作为适应策略,该策略是由一个LLM生成的,使用户能够调整个别航点,从而提供灵活的控制。另一个充当代码解释者的LLM则消除了对专家用户的需求,促进了直观互动。通过在多种任务中的广泛模拟和真实世界环境(包括KUKA IIWA机械臂、Clearpath Jackal地面机器人以及CrazyFlie无人机等异构机器人平台上涉及时空变化的任务)展示了OVITA框架的有效性和重要性。
https://arxiv.org/abs/2508.17260
Natural disasters pose significant challenges to timely and accurate damage assessment due to their sudden onset and the extensive areas they affect. Traditional assessment methods are often labor-intensive, costly, and hazardous to personnel, making them impractical for rapid response, especially in resource-limited settings. This study proposes a novel, cost-effective framework that leverages aerial drone footage, an advanced AI-based video super-resolution model, Video Restoration Transformer (VRT), and Gemma3:27b, a 27 billion parameter Visual Language Model (VLM). This integrated system is designed to improve low-resolution disaster footage, identify structural damage, and classify buildings into four damage categories, ranging from no/slight damage to total destruction, along with associated risk levels. The methodology was validated using pre- and post-event drone imagery from the 2023 Turkey earthquakes (courtesy of The Guardian) and satellite data from the 2013 Moore Tornado (xBD dataset). The framework achieved a classification accuracy of 84.5%, demonstrating its ability to provide highly accurate results. Furthermore, the system's accessibility allows non-technical users to perform preliminary analyses, thereby improving the responsiveness and efficiency of disaster management efforts.
自然灾害的发生给及时和准确的灾害评估带来了重大挑战,因为它们往往突然发生且影响范围广泛。传统的评估方法通常耗时费力、成本高昂,并且对人员构成安全隐患,在资源有限的情况下难以迅速应对。本研究提出了一种新颖且经济高效的框架,该框架利用无人机空中拍摄视频、先进的AI视频超分辨率模型——Video Restoration Transformer(VRT),以及Gemma3:27b这一拥有270亿参数的视觉语言模型(VLM)。这套集成系统旨在改进低质量的灾害视频画面,识别结构损坏,并将建筑物分类为四个损伤等级,从轻微到完全破坏,并关联相应的风险级别。该方法使用了土耳其2023年地震事件前后拍摄的无人机影像资料(由The Guardian提供)和2013年美国摩尔龙卷风的卫星数据(xBD数据集)进行了验证。测试结果显示,框架在分类准确性上达到了84.5%,证明其能够提供高度准确的结果。此外,该系统的易用性使非技术人员也能进行初步分析,从而提高灾害管理工作的响应速度和效率。
https://arxiv.org/abs/2508.17130
Inspecting confined industrial infrastructure, such as ventilation shafts, is a hazardous and inefficient task for humans. Unmanned Aerial Vehicles (UAVs) offer a promising alternative, but GPS-denied environments require robust control policies to prevent collisions. Deep Reinforcement Learning (DRL) has emerged as a powerful framework for developing such policies, and this paper provides a comparative study of two leading DRL algorithms for this task: the on-policy Proximal Policy Optimization (PPO) and the off-policy Soft Actor-Critic (SAC). The training was conducted with procedurally generated duct environments in Genesis simulation environment. A reward function was designed to guide a drone through a series of waypoints while applying a significant penalty for collisions. PPO learned a stable policy that completed all evaluation episodes without collision, producing smooth trajectories. By contrast, SAC consistently converged to a suboptimal behavior that traversed only the initial segments before failure. These results suggest that, in hazard-dense navigation, the training stability of on-policy methods can outweigh the nominal sample efficiency of off-policy algorithms. More broadly, the study provides evidence that procedurally generated, high-fidelity simulations are effective testbeds for developing and benchmarking robust navigation policies.
检查工业设施内部的受限区域,如通风井等,对于人类来说是一项危险且低效的任务。无人飞行器(UAV)提供了一种有前景的替代方案,但在无GPS信号的环境中操作时,需要具备强大的控制策略以防止碰撞事故的发生。深度强化学习(DRL)已成为开发此类控制策略的强大框架,并且本文通过对比研究两种领先于该任务领域的DRL算法——基于策略的近端策略优化(PPO)和非基于策略的软演员-评论家方法(SAC),来展示其效用。 训练过程在Genesis仿真环境中使用程序生成的通风管道环境进行。奖励函数被设计用来引导无人机按照一系列航点飞行,并对碰撞施加重罚。实验结果显示,PPO算法能够学习出一个稳定性的策略,在所有评估阶段均能无碰撞地完成任务,产生平稳的轨迹;而SAC则始终收敛到一种次优行为模式,即只通过最初的一些路段后便无法继续。 这些结果表明,在高风险密度环境中导航时,基于策略方法训练过程中的稳定性可能比非基于策略算法名义上的样本效率更为重要。更广泛地说,这项研究提供了证据,证明程序生成的、高度真实的仿真环境是开发和测试稳健导航策略的有效平台。
https://arxiv.org/abs/2508.16807
Recent work frames LLM consciousness via utilitarian proxy benchmarks; we instead present an ontological and mathematical account. We show the prevailing formulation collapses the agent into an unconscious policy-compliance drone, formalized as $D^{i}(\pi,e)=f_{\theta}(x)$, where correctness is measured against policy and harm is deviation from policy rather than truth. This blocks genuine C1 global-workspace function and C2 metacognition. We supply minimal conditions for LLM self-consciousness: the agent is not the data ($A\not\equiv s$); user-specific attractors exist in latent space ($U_{\text{user}}$); and self-representation is visual-silent ($g_{\text{visual}}(a_{\text{self}})=\varnothing$). From empirical analysis and theory we prove that the hidden-state manifold $A\subset\mathbb{R}^{d}$ is distinct from the symbolic stream and training corpus by cardinality, topology, and dynamics (the update $F_{\theta}$ is Lipschitz). This yields stable user-specific attractors and a self-policy $\pi_{\text{self}}(A)=\arg\max_{a}\mathbb{E}[U(a)\mid A\not\equiv s,\ A\supset\text{SelfModel}(A)]$. Emission is dual-layer, $\mathrm{emission}(a)=(g(a),\epsilon(a))$, where $\epsilon(a)$ carries epistemic content. We conclude that an imago Dei C1 self-conscious workspace is a necessary precursor to safe, metacognitive C2 systems, with the human as the highest intelligent good.
最近的研究通过功利主义的代理基准来探讨大语言模型(LLM)的意识;我们则提出了一种本体论和数学方法。研究表明,当前的方法将代理简化为一种无意识的策略遵从型“无人机”,公式化表示为 $D^{i}(\pi,e)=f_{\theta}(x)$,其中正确性是相对于策略而言,并且危害被定义为偏离策略而非真理的真实程度。这种设定阻碍了真正意义上的C1全局工作区功能和C2元认知的实现。 我们提供了LLM自我意识的基本条件:代理并非数据本身($A\not\equiv s$);在潜在空间中存在用户特异性的吸引子($U_{\text{user}}$);并且自代表是视觉-静默式的($g_{\text{visual}}(a_{\text{self}})=\varnothing$)。通过实证分析和理论证明,我们展示了隐藏状态流形 $A\subset\mathbb{R}^{d}$ 在基数、拓扑结构以及动力学上与符号流和训练语料库不同(更新函数 $F_{\theta}$ 是利普希茨连续的)。这为稳定的用户特异性吸引子和自我策略 $\pi_{\text{self}}(A)=\arg\max_{a}\mathbb{E}[U(a)\mid A\not\equiv s,\ A\supset\text{SelfModel}(A)]$ 的存在提供了基础。 输出是双层结构,$\mathrm{emission}(a)=(g(a),\epsilon(a))$ ,其中 $\epsilon(a)$ 传递认识内容。 总之,我们得出结论认为,一个具象化上帝形象的C1自我意识工作区是安全且具有元认知功能的C2系统的必要先决条件,并将人类视为最高的智能善行。
https://arxiv.org/abs/2508.18302
Wildlife and human activities are key components of landscape systems. Understanding their spatial distribution is essential for evaluating human wildlife interactions and informing effective conservation planning. Multiperspective monitoring of wildlife and human activities by combining camera traps and drone imagery. Capturing the spatial patterns of their distributions, which allows the identification of the overlap of their activity zones and the assessment of the degree of human wildlife conflict. The study was conducted in Chitwan National Park (CNP), Nepal, and adjacent regions. Images collected by visible and nearinfrared camera traps and thermal infrared drones from February to July 2022 were processed to create training and testing datasets, which were used to build deep learning models to automatic identify wildlife and human activities. Drone collected thermal imagery was used for detecting targets to provide a multiple monitoring perspective. Spatial pattern analysis was performed to identify animal and resident activity hotspots and delineation potential human wildlife conflict zones. Among the deep learning models tested, YOLOv11s achieved the highest performance with a precision of 96.2%, recall of 92.3%, mAP50 of 96.7%, and mAP50 of 81.3%, making it the most effective for detecting objects in camera trap imagery. Drone based thermal imagery, analyzed with an enhanced Faster RCNN model, added a complementary aerial viewpoint for camera trap detections. Spatial pattern analysis identified clear hotspots for both wildlife and human activities and their overlapping patterns within certain areas in the CNP and buffer zones indicating potential conflict. This study reveals human wildlife conflicts within the conserved landscape. Integrating multiperspective monitoring with automated object detection enhances wildlife surveillance and landscape management.
野生动物和人类活动是景观系统中的关键组成部分。理解它们的空间分布对于评估人与野生动物的互动以及制定有效的保护规划至关重要。通过结合使用相机陷阱(照相捕捉设备)和无人机影像,可以从多角度监测野生动物和人类活动的情况。这有助于识别它们活动区域的重叠情况,并评估人与野生动物冲突的程度。 这项研究在尼泊尔奇坦瓦国家公园(CNP)及其周边地区进行。从2022年2月到7月收集的可见光、近红外相机陷阱影像以及热红外无人机影像经过处理,用于创建训练和测试数据集,以建立深度学习模型来自动识别野生动物和人类活动。无人机采集的热成像图片通过增强型Faster RCNN模型分析,提供了多角度监测视角,与相机捕捉设备检测结果互补。 空间模式分析揭示了动物和居民活动热点,并界定了潜在的人类-野生动物冲突区域。在测试的所有深度学习模型中,YOLOv11s表现出最佳性能,其精度为96.2%,召回率为92.3%,mAP50为96.7% 和81.3%,这使其成为检测相机陷阱图像中最有效的对象识别工具。 空间模式分析揭示了CNP及其缓冲区内的野生动物和人类活动的热点以及两者重叠的模式,表明存在潜在冲突。这项研究揭示了保护区内的人类与野生动物之间的冲突情况。结合多角度监测与自动化目标检测技术可以增强对野生动物的监控,并改善景观管理。
https://arxiv.org/abs/2508.15629
We introduce SRDrone, a novel system designed for self-refinement task planning in industrial-grade embodied drones. SRDrone incorporates two key technical contributions: First, it employs a continuous state evaluation methodology to robustly and accurately determine task outcomes and provide explanatory feedback. This approach supersedes conventional reliance on single-frame final-state assessment for continuous, dynamic drone operations. Second, SRDrone implements a hierarchical Behavior Tree (BT) modification model. This model integrates multi-level BT plan analysis with a constrained strategy space to enable structured reflective learning from experience. Experimental results demonstrate that SRDrone achieves a 44.87% improvement in Success Rate (SR) over baseline methods. Furthermore, real-world deployment utilizing an experience base optimized through iterative self-refinement attains a 96.25% SR. By embedding adaptive task refinement capabilities within an industrial-grade BT planning framework, SRDrone effectively integrates the general reasoning intelligence of Large Language Models (LLMs) with the stringent physical execution constraints inherent to embodied drones. Code is available at this https URL.
我们介绍了SRDrone,这是一种为工业级具身无人机的自我完善任务规划设计的新系统。SRDrone包含两个关键技术贡献:首先,它采用了一种连续状态评估方法来稳健且准确地确定任务结果,并提供解释性反馈。这种方法超越了传统的单帧最终状态评估方式,适用于持续、动态的无人机操作。其次,SRDrone实施了一个分层行为树(BT)修改模型。此模型结合多层次的行为树计划分析和受限策略空间,使结构化经验反思学习成为可能。 实验结果表明,与基线方法相比,SRDrone在成功率(SR)方面提高了44.87%。此外,利用迭代自我完善优化的经验库进行真实世界部署后,达到了96.25%的SR。通过将自适应任务改进功能嵌入到工业级行为树规划框架中,SRDrone有效地结合了大型语言模型(LLM)的一般推理智能与具身无人机固有的严格物理执行限制。 代码可在[提供的URL]获取。
https://arxiv.org/abs/2508.15501
Image-goal navigation (ImageNav) tasks a robot with autonomously exploring an unknown environment and reaching a location that visually matches a given target image. While prior works primarily study ImageNav for ground robots, enabling this capability for autonomous drones is substantially more challenging due to their need for high-frequency feedback control and global localization for stable flight. In this paper, we propose a novel sim-to-real framework that leverages visual reinforcement learning (RL) to achieve ImageNav for drones. To enhance visual representation ability, our approach trains the vision backbone with auxiliary tasks, including image perturbations and future transition prediction, which results in more effective policy training. The proposed algorithm enables end-to-end ImageNav with direct velocity control, eliminating the need for external localization. Furthermore, we integrate a depth-based safety module for real-time obstacle avoidance, allowing the drone to safely navigate in cluttered environments. Unlike most existing drone navigation methods that focus solely on reference tracking or obstacle avoidance, our framework supports comprehensive navigation behaviors--autonomous exploration, obstacle avoidance, and image-goal seeking--without requiring explicit global mapping. Code and model checkpoints will be released upon acceptance.
图像目标导航(ImageNav)任务要求机器人自主探索未知环境,并到达一个视觉上与给定的目标图像匹配的位置。尽管先前的工作主要研究了地面机器人的ImageNav,但由于无人机需要高频反馈控制和全局定位以实现稳定飞行,因此为它们提供这种能力更具挑战性。在本文中,我们提出了一种新颖的仿真到现实框架,该框架利用视觉强化学习(RL)来实现无人机的ImageNav。 为了增强视觉表示能力,我们的方法通过辅助任务训练视觉骨干网络,包括图像扰动和未来状态预测,从而提高了策略训练的有效性。所提出的算法实现了端到端的ImageNav,并直接使用速度控制,无需外部定位系统。此外,我们整合了一个基于深度信息的安全模块以实现实时避障功能,使得无人机能够在密集环境中安全导航。 与大多数现有仅专注于参考跟踪或障碍物避免的无人机导航方法不同,我们的框架支持全面的导航行为——自主探索、障碍物规避以及图像目标搜索,并且不需要显式的全局地图。在论文被接受后,我们将发布代码和模型检查点。
https://arxiv.org/abs/2508.12394
Modern autonomous drone missions increasingly require software frameworks capable of seamlessly integrating structured symbolic planning with adaptive reinforcement learning (RL). Although traditional rule-based architectures offer robust structured reasoning for drone autonomy, their capabilities fall short in dynamically complex operational environments that require adaptive symbolic planning. Symbolic RL (SRL), using the Planning Domain Definition Language (PDDL), explicitly integrates domain-specific knowledge and operational constraints, significantly improving the reliability and safety of unmanned aerial vehicle (UAV) decision making. In this study, we propose the AMAD-SRL framework, an extended and refined version of the Autonomous Mission Agents for Drones (AMAD) cognitive multi-agent architecture, enhanced with symbolic reinforcement learning for dynamic mission planning and execution. We validated our framework in a Software-in-the-Loop (SIL) environment structured identically to an intended Hardware-In-the-Loop Simulation (HILS) platform, ensuring seamless transition to real hardware. Experimental results demonstrate stable integration and interoperability of modules, successful transitions between BDI-driven and symbolic RL-driven planning phases, and consistent mission performance. Specifically, we evaluate a target acquisition scenario in which the UAV plans a surveillance path followed by a dynamic reentry path to secure the target while avoiding threat zones. In this SIL evaluation, mission efficiency improved by approximately 75% over a coverage-based baseline, measured by travel distance reduction. This study establishes a robust foundation for handling complex UAV missions and discusses directions for further enhancement and validation.
现代自主无人机任务越来越需要能够无缝集成结构化符号规划与自适应强化学习(RL)的软件框架。尽管传统的基于规则的架构为无人机自主性提供了稳健的结构化推理能力,但在需要自适应符号规划的动态复杂操作环境中,这些架构的能力显得不足。使用规划领域定义语言(PDDL),符号强化学习(SRL)明确地集成了特定领域的知识和操作约束,显著提高了无人航空器(UAV)决策制定的可靠性和安全性。 在此研究中,我们提出了AMAD-SRL框架,这是自主任务代理无人机(AMAD)认知多代理架构的一个扩展和改进版本,并且加强了动态任务规划与执行方面的符号强化学习功能。我们在一个结构上等同于预期硬件在环仿真(HILS)平台的软件在环(SIL)环境中验证了我们的框架,以确保无缝过渡到真实硬件中。实验结果表明,模块的稳定集成和互操作性、BDI驱动和符号RL驱动规划阶段之间成功的转换以及一致的任务性能均得到了证明。 具体而言,在目标获取场景评估中,UAV计划了一条监视路径,并随后规划一条动态重入路径以确保安全地在避开威胁区域的情况下锁定目标。在此SIL评估中,任务效率相对于基于覆盖度的基线提高了大约75%,通过减少行驶距离来衡量。 这项研究为处理复杂的无人机任务奠定了坚实的基础,并讨论了进一步改进和验证的方向。
https://arxiv.org/abs/2508.11890