Information on trees at the individual level is crucial for monitoring forest ecosystems and planning forest management. Current monitoring methods involve ground measurements, requiring extensive cost, time and labor. Advances in drone remote sensing and computer vision offer great potential for mapping individual trees from aerial imagery at broad-scale. Large pre-trained vision models, such as the Segment Anything Model (SAM), represent a particularly compelling choice given limited labeled data. In this work, we compare methods leveraging SAM for the task of automatic tree crown instance segmentation in high resolution drone imagery in three use cases: 1) boreal plantations, 2) temperate forests and 3) tropical forests. We also study the integration of elevation data into models, in the form of Digital Surface Model (DSM) information, which can readily be obtained at no additional cost from RGB drone imagery. We present BalSAM, a model leveraging SAM and DSM information, which shows potential over other methods, particularly in the context of plantations. We find that methods using SAM out-of-the-box do not outperform a custom Mask R-CNN, even with well-designed prompts. However, efficiently tuning SAM end-to-end and integrating DSM information are both promising avenues for tree crown instance segmentation models.
树木的个体信息对于监测森林生态系统和规划林业管理至关重要。目前,地面测量方法耗时、费力且成本高昂。无人机遥感技术和计算机视觉的进步为从高分辨率航拍图像中绘制单个树木提供了巨大潜力。特别地,在标注数据有限的情况下,大型预训练视觉模型(如Segment Anything Model (SAM))成为了一个极具吸引力的选择。在本研究中,我们对比了三种应用场景下使用SAM进行自动树冠实例分割的方法:1)针叶林人工林;2)温带森林;3)热带森林。此外,我们还探讨了如何将高程数据(以数字表面模型(DSM)的形式获取,可以从RGB无人机图像免费获得)集成到模型中。我们提出了一种名为BalSAM的模型,该模型结合了SAM和DSM信息,并在人工林场景中表现出比其他方法更优的潜力。然而,我们也发现直接使用未经调整的SAM并不能优于定制化的Mask R-CNN模型,即使使用精心设计的提示也是如此。不过,对SAM进行端到端的有效微调以及整合DSM数据均为树冠实例分割模型提供了有前景的发展方向。
https://arxiv.org/abs/2506.04970
Monitoring aerial objects is crucial for security, wildlife conservation, and environmental studies. Traditional RGB-based approaches struggle with challenges such as scale variations, motion blur, and high-speed object movements, especially for small flying entities like insects and drones. In this work, we explore the potential of event-based vision for detecting and recognizing flying objects, in particular animals that may not follow short and long-term predictable patters. Event cameras offer high temporal resolution, low latency, and robustness to motion blur, making them well-suited for this task. We introduce EV-Flying, an event-based dataset of flying objects, comprising manually annotated birds, insects and drones with spatio-temporal bounding boxes and track identities. To effectively process the asynchronous event streams, we employ a point-based approach leveraging lightweight architectures inspired by PointNet. Our study investigates the classification of flying objects using point cloud-based event representations. The proposed dataset and methodology pave the way for more efficient and reliable aerial object recognition in real-world scenarios.
监测空中物体对于安全、野生动物保护和环境研究至关重要。传统基于RGB的方法在面对尺度变化、运动模糊以及高速移动目标(特别是像昆虫和无人机这样的小型飞行实体)时,面临诸多挑战。在这项工作中,我们探索了事件驱动视觉在检测和识别飞行物体方面的潜力,尤其是那些不遵循短期或长期可预测模式的动物。事件相机提供高时间分辨率、低延迟及对运动模糊的强大抵抗力,非常适合此类任务。我们介绍了EV-Flying,这是一个基于事件的飞行物数据集,包含手动标注的鸟类、昆虫和无人机,并带有空间-时间边界框和轨迹标识。为了有效处理异步事件流,我们采用了一种点基方法,借鉴轻量级架构如PointNet的设计思想。我们的研究探讨了使用点云表示法进行飞行物体分类的方法。所提出的数据集和方法为在现实场景中实现更高效、可靠的空中目标识别铺平了道路。
https://arxiv.org/abs/2506.04048
Open-vocabulary semantic segmentation (OVSS) involves assigning labels to each pixel in an image based on textual descriptions, leveraging world models like CLIP. However, they encounter significant challenges in cross-domain generalization, hindering their practical efficacy in real-world applications. Embodied AI systems are transforming autonomous navigation for ground vehicles and drones by enhancing their perception abilities, and in this study, we present AetherVision-Bench, a benchmark for multi-angle segmentation across aerial, and ground perspectives, which facilitates an extensive evaluation of performance across different viewing angles and sensor modalities. We assess state-of-the-art OVSS models on the proposed benchmark and investigate the key factors that impact the performance of zero-shot transfer models. Our work pioneers the creation of a robustness benchmark, offering valuable insights and establishing a foundation for future research.
开放词汇语义分割(OVSS)涉及根据文本描述将标签分配给图像中的每个像素,利用像CLIP这样的世界模型。然而,这类方法在跨域泛化方面面临重大挑战,这限制了它们在实际应用中的有效性。嵌入式AI系统通过增强自主导航的感知能力,正在改变地面车辆和无人机的应用方式。在这项研究中,我们提出了AetherVision-Bench,这是一个针对多角度分割的基准测试平台,涵盖了空中和地面视角,有助于对不同观察角度和传感器模态下的性能进行全面评估。我们在提出的基准上评估了最新的OVSS模型,并探讨了影响零样本迁移模型性能的关键因素。我们的工作开创了一个鲁棒性基准的先河,提供了有价值的见解并为未来的研究奠定了基础。
https://arxiv.org/abs/2506.03709
The increasing proliferation of small UAVs in civilian and military airspace has raised critical safety and security concerns, especially when unauthorized or malicious drones enter restricted zones. In this work, we present a reinforcement learning (RL) framework for agile 1v1 quadrotor pursuit-evasion. We train neural network policies to command body rates and collective thrust, enabling high-speed pursuit and evasive maneuvers that fully exploit the quadrotor's nonlinear dynamics. To mitigate nonstationarity and catastrophic forgetting during adversarial co-training, we introduce an Asynchronous Multi-Stage Population-Based (AMSPB) algorithm where, at each stage, either the pursuer or evader learns against a sampled opponent drawn from a growing population of past and current policies. This continual learning setup ensures monotonic performance improvement and retention of earlier strategies. Our results show that (i) rate-based policies achieve significantly higher capture rates and peak speeds than velocity-level baselines, and (ii) AMSPB yields stable, monotonic gains against a suite of benchmark opponents.
民用和军事空域中小型无人机的日益普及引发了重要的安全和安保问题,尤其是当未经授权或恶意操作的无人机进入受限区域时。在这项工作中,我们提出了一种用于敏捷1v1四旋翼追逐与逃避的强化学习(RL)框架。我们训练神经网络策略来指挥机体速率和总推力,使高速追踪和规避动作能够充分利用四旋翼的非线性动态特性。 为了在对抗性协同训练中缓解非平稳性和灾难性遗忘问题,我们引入了一种异步多阶段群体基础算法(AMSPB),其中,在每个阶段,追击者或逃避者都会针对从过去和当前策略组成的不断扩大的群体中随机抽取的对手进行学习。这种持续学习设置确保了性能的单调改进,并保留了早期策略。 我们的实验结果表明:(i)基于速率的政策实现了比速度级基准显著更高的捕捉率和峰值速度;(ii)AMSPB算法对一系列基准对手产生了稳定且单调的收益。
https://arxiv.org/abs/2506.02849
Unmanned Aerial Vehicles (UAVs) are one of the most revolutionary inventions of 21st century. At the core of a UAV lies the central processing system that uses wireless signals to control their movement. The most popular UAVs are quadcopters that use a set of four motors, arranged as two on either side with opposite spin. An autonomous UAV is called a drone. Drones have been in service in the US army since the 90's for covert missions critical to national security. It would not be wrong to claim that drones make up an integral part of the national security and provide the most valuable service during surveillance operations. While UAVs are controlled using wireless signals, there reside some challenges that disrupt the operation of such vehicles such as signal quality and range, real time processing, human expertise, robust hardware and data security. These challenges can be solved by programming UAVs to be autonomous, using object detection and tracking, through Computer Vision algorithms. Computer Vision is an interdisciplinary field that seeks the use of deep learning to gain a high-level understanding of digital images and videos for the purpose of automating the task of human visual system. Using computer vision, algorithms for detecting and tracking various objects can be developed suitable to the hardware so as to allow real time processing for immediate judgement. This paper attempts to review the various approaches several authors have proposed for the purpose of autonomous navigation of UAVs by through various algorithms of object detection and tracking in real time, for the purpose of applications in various fields such as disaster management, dense area exploration, traffic vehicle surveillance etc.
无人驾驶飞行器(UAV,即无人机)是21世纪最具革命性的发明之一。在UAV的核心位置有一个中央处理系统,该系统使用无线信号来控制其运动。目前最受欢迎的UAV是四旋翼飞机,它由四个电机组成,两个位于一侧且旋转方向相反。自主运行的UAV被称为无人机。自90年代以来,美国军队一直在为国家安全的关键秘密任务中使用无人机服务。毫不夸张地说,无人机在国家安全中占据了重要地位,并在监视行动期间提供了最有价值的服务。尽管UAV可以通过无线信号进行控制,但诸如信号质量和范围、实时处理能力、人力专业技能、硬件的坚固性以及数据安全等挑战仍可能影响其操作性能。为了解决这些问题,可以编程使UAV自主运行,利用物体检测和跟踪技术,并通过计算机视觉算法实现。 计算机视觉是一个跨学科领域,旨在运用深度学习来理解数字图像和视频中的高层次内容,以便自动化人类视觉系统的工作任务。借助于计算机视觉,可以开发适用于特定硬件的实时处理对象检测与追踪算法,从而实现实时判断。 本文试图回顾多位作者为UAV自主导航目的通过各种实时物体检测和跟踪算法提出的多种方法,并探讨这些技术在灾害管理、密集区域探索、交通车辆监控等各个领域的应用。
https://arxiv.org/abs/2506.05378
Quadrotors have demonstrated remarkable versatility, yet their full aerobatic potential remains largely untapped due to inherent underactuation and the complexity of aggressive maneuvers. Traditional approaches, separating trajectory optimization and tracking control, suffer from tracking inaccuracies, computational latency, and sensitivity to initial conditions, limiting their effectiveness in dynamic, high-agility scenarios. Inspired by recent breakthroughs in data-driven methods, we propose a reinforcement learning-based framework that directly maps drone states and aerobatic intentions to control commands, eliminating modular separation to enable quadrotors to perform end-to-end policy optimization for extreme aerobatic maneuvers. To ensure efficient and stable training, we introduce an automated curriculum learning strategy that dynamically adjusts aerobatic task difficulty. Enabled by domain randomization for robust zero-shot sim-to-real transfer, our approach is validated in demanding real-world experiments, including the first demonstration of a drone autonomously performing continuous inverted flight while reactively navigating a moving gate, showcasing unprecedented agility.
四旋翼飞行器展示了非凡的灵活性,但由于固有的欠驱动特性和复杂剧烈机动操作的难度,其全面的特技飞行潜力尚未充分发挥。传统的分离轨迹优化和跟踪控制的方法受到跟踪不准确、计算延迟以及对初始条件敏感性的影响,在动态和高敏捷度场景中的效果受到了限制。受最近数据驱动方法突破的启发,我们提出了一种基于强化学习框架的方法,该方法直接将无人机状态与特技意图映射到控制命令上,消除了模块化分离,使四旋翼能够进行端到端策略优化以执行极端特技飞行动作。 为了确保高效和稳定的训练过程,我们引入了自动化的课程学习策略,可以动态调整特技任务的难度。借助领域随机化技术实现鲁棒的零样本仿真至现实转换能力,我们的方法在严格的实际环境中进行了验证,包括首次演示无人机自主完成连续倒飞并主动导航通过移动门的任务,展现了前所未有的敏捷性。
https://arxiv.org/abs/2505.24396
This study investigates the application of Genetic Fuzzy Systems (GFS) to model the self-noise generated by airfoils, a key issue in aeroaccoustics with significant implications for aerospace, automotive and drone applications. Using the publicly available Airfoil Self Noise dataset, various Fuzzy regression strategies are explored and compared. The paper evaluates a brute force Takagi Sugeno Kang (TSK) fuzzy system with high rule density, a cascading Geneti Fuzzy Tree (GFT) architecture and a novel clustered approach based on Fuzzy C-means (FCM) to reduce the model's complexity. This highlights the viability of clustering assisted fuzzy inference as an effective regression tool for complex aero accoustic phenomena. Keywords : Fuzzy logic, Regression, Cascading systems, Clustering and AI.
这项研究探讨了遗传模糊系统(GFS)在建模由机翼自噪声生成中的应用,这是一个航空声学领域的重要问题,在航空航天、汽车和无人机应用中具有重大影响。通过使用公开的Airfoil Self Noise数据集,对各种模糊回归策略进行了探索与比较。论文评估了一种基于高规则密度的强力Takagi Sugeno Kang(TSK)模糊系统、一种级联遗传模糊树(GFT)架构以及一种新的基于Fuzzy C-means (FCM)聚类方法来降低模型复杂度的新颖方法。这突显了在复杂的航空声学现象中,通过聚类辅助的模糊推理作为有效的回归工具的应用潜力。 关键词:模糊逻辑、回归、级联系统、聚类和人工智能
https://arxiv.org/abs/2505.23746
The rapid growth of e-commerce and the increasing demand for timely, cost-effective last-mile delivery have increased interest in collaborative logistics. This research introduces a novel collaborative synchronized multi-platform vehicle routing problem with drones and robots (VRP-DR), where a fleet of $\mathcal{M}$ trucks, $\mathcal{N}$ drones and $\mathcal{K}$ robots, cooperatively delivers parcels. Trucks serve as mobile platforms, enabling the launching, retrieving, and en-route charging of drones and robots, thereby addressing critical limitations such as restricted payload capacities, limited range, and battery constraints. The VRP-DR incorporates five realistic features: (1) multi-visit service per trip, (2) multi-trip operations, (3) flexible docking, allowing returns to the same or different trucks (4) cyclic and acyclic operations, enabling return to the same or different nodes; and (5) en-route charging, enabling drones and robots to recharge while being transported on the truck, maximizing operational efficiency by utilizing idle transit time. The VRP-DR is formulated as a mixed-integer linear program (MILP) to minimize both operational costs and makespan. To overcome the computational challenges of solving large-scale instances, a scalable heuristic algorithm, FINDER (Flexible INtegrated Delivery with Energy Recharge), is developed, to provide efficient, near-optimal solutions. Numerical experiments across various instance sizes evaluate the performance of the MILP and heuristic approaches in terms of solution quality and computation time. The results demonstrate significant time savings of the combined delivery mode over the truck-only mode and substantial cost reductions from enabling multi-visits. The study also provides insights into the effects of en-route charging, docking flexibility, drone count, speed, and payload capacity on system performance.
电子商务的快速增长和对及时、经济高效的最后一公里配送需求的增加,引发了人们对于协同物流的兴趣。这项研究提出了一种新颖的合作同步多平台车辆路径问题(VRP-DR),其中由$\mathcal{M}$辆卡车、$\mathcal{N}$架无人机以及$\mathcal{K}$个机器人组成的车队共同完成包裹递送任务。卡车作为移动平台,能够支持无人机和机器人的启动、回收及途中充电,从而解决载重有限、航程受限和电池约束等关键限制问题。VRP-DR涵盖了五个现实特征:(1)单次旅程中的多次服务;(2)多趟作业操作;(3)灵活对接,允许返回同一或不同卡车;(4)循环与非循环操作,支持回到相同或不同的节点;以及(5)途中充电,在运输过程中为无人机和机器人提供充电能力,最大化利用闲置的运输时间以提高运营效率。VRP-DR被建模成一个混合整数线性规划问题(MILP),旨在最小化运营成本与总耗时。为了克服解决大规模实例计算挑战的问题,研究开发了一种可扩展的启发式算法FINDER(灵活集成配送及能量充电),以提供高效且接近最优的解决方案。通过不同规模实例的数值实验评估了MILP和启发式方法在解的质量和计算时间方面的性能。结果表明,综合递送模式相比仅使用卡车模式显著节约了时间和成本,并从支持多次服务中实现了可观的成本削减。该研究还提供了有关途中充电、对接灵活性、无人机数量、速度及载重能力对系统表现影响的洞见。
https://arxiv.org/abs/2505.23584
Small Uncrewed Aerial Systems (sUAS) are increasingly deployed as autonomous swarms in search-and-rescue and other disaster-response scenarios. In these settings, they use computer vision (CV) to detect objects of interest and autonomously adapt their missions. However, traditional CV systems often struggle to recognize unfamiliar objects in open-world environments or to infer their relevance for mission planning. To address this, we incorporate large language models (LLMs) to reason about detected objects and their implications. While LLMs can offer valuable insights, they are also prone to hallucinations and may produce incorrect, misleading, or unsafe recommendations. To ensure safe and sensible decision-making under uncertainty, high-level decisions must be governed by cognitive guardrails. This article presents the design, simulation, and real-world integration of these guardrails for sUAS swarms in search-and-rescue missions.
小型无人飞行系统(sUAS)在搜索和救援以及其他灾害应对场景中越来越多地被部署为自主集群。在这种情况下,它们使用计算机视觉(CV)来检测感兴趣的物体并自动调整其任务计划。然而,传统的CV系统往往难以识别开放环境中的不熟悉的物体或推断这些物体对任务规划的相关性。为此,我们引入了大型语言模型(LLMs),以针对检测到的物体进行推理并评估它们的意义。尽管LLMs可以提供宝贵的见解,但它们也容易产生幻觉,并可能生成错误、误导性的甚至危险的建议。为了在不确定性条件下确保安全和合理的决策制定,必须通过认知护栏来规范高层决策。本文介绍了为搜索和救援任务中的sUAS集群设计、模拟和实际集成这些认知护栏的方法。
https://arxiv.org/abs/2505.23576
We aim to use virtual reality (VR) to improve the spatial awareness of pilots by real-time scanning of the environment around the drone using onboard sensors, live streaming of this environment to a VR headset, and rendering a virtual representation of the drone and its environment for the pilot. This way, the pilot can see the immediate environment of the drone up close from a third-person perspective, as opposed to the first-person perspective that most drone cameras provide. This provides much more information about the drone surroundings for the pilot while operating the drone than existing teleoperation solutions. Previous solutions using VR have relied upon pre-made designs of the environment, which makes it difficult to adapt to changing environments. Our solution, in contrast, scans the environment as you fly, making it much more flexible for use in unknown environments.
我们的目标是利用虚拟现实(VR)技术,通过无人机上的传感器实时扫描周围环境,并将这些信息流传输到VR头显中,从而提高飞行员的空间感知能力。此外,我们还会渲染一个虚拟的无人机及其环境的表示图给飞行员。这样一来,飞行员可以从第三人称视角近距离观察无人机周围的即时环境,而不仅仅是像大多数无人机摄像头提供的第一人称视角那样。这种方式为操作无人机时提供了比现有远程控制解决方案更多的关于无人机周围环境的信息。 以往采用VR技术的方法依赖于预先设计好的环境模型,这使得适应不断变化的环境变得困难。相比之下,我们的解决方案在飞行过程中实时扫描环境,使其更加灵活地适用于未知环境。
https://arxiv.org/abs/2505.22599
While unmanned aerial vehicles (UAVs) offer wide-area, high-altitude coverage for anomaly detection, they face challenges such as dynamic viewpoints, scale variations, and complex scenes. Existing datasets and methods, mainly designed for fixed ground-level views, struggle to adapt to these conditions, leading to significant performance drops in drone-view scenarios. To bridge this gap, we introduce A2Seek (Aerial Anomaly Seek), a large-scale, reasoning-centric benchmark dataset for aerial anomaly understanding. This dataset covers various scenarios and environmental conditions, providing high-resolution real-world aerial videos with detailed annotations, including anomaly categories, frame-level timestamps, region-level bounding boxes, and natural language explanations for causal reasoning. Building on this dataset, we propose A2Seek-R1, a novel reasoning framework that generalizes R1-style strategies to aerial anomaly understanding, enabling a deeper understanding of "Where" anomalies occur and "Why" they happen in aerial frames. To this end, A2Seek-R1 first employs a graph-of-thought (GoT)-guided supervised fine-tuning approach to activate the model's latent reasoning capabilities on A2Seek. Then, we introduce Aerial Group Relative Policy Optimization (A-GRPO) to design rule-based reward functions tailored to aerial scenarios. Furthermore, we propose a novel "seeking" mechanism that simulates UAV flight behavior by directing the model's attention to informative regions. Extensive experiments demonstrate that A2Seek-R1 achieves up to a 22.04% improvement in AP for prediction accuracy and a 13.9% gain in mIoU for anomaly localization, exhibiting strong generalization across complex environments and out-of-distribution scenarios. Our dataset and code will be released at this https URL.
尽管无人驾驶飞行器(无人机)能够提供广泛区域和高空的异常检测覆盖,但它们面临诸如动态视角、尺度变化及复杂场景等挑战。现有的数据集和方法主要针对固定的地面级视图设计,难以适应这些条件,在无人机视角的应用场景中导致性能大幅下降。为解决这一差距,我们推出了A2Seek(Aerial Anomaly Seek),这是一个大规模、以推理为中心的基准数据集,旨在用于空中异常理解。该数据集涵盖了多种场景和环境条件,并提供了高分辨率的真实世界空域视频以及详细的注释,包括异常类别、帧级别的时间戳、区域级别的边界框和自然语言解释,用于因果关系推理。 基于此数据集,我们提出了A2Seek-R1,这是一种新颖的推理框架,它将R1风格策略推广到空中异常理解领域,使模型能够更深入地理解“在哪里”发生了异常以及“为什么”会发生这些异常。为此,A2Seek-R1首先采用了一种图思维(GoT)引导的监督微调方法来激活模型在A2Seek上的潜在推理能力。接着,我们引入了空中群体相对策略优化(A-GRPO),以设计适用于空中场景的规则化奖励函数。此外,我们还提出了一种新颖的“寻找”机制,通过将模型注意力集中到信息丰富的区域上来模拟无人机飞行行为。 广泛的实验表明,A2Seek-R1在预测准确率上达到了高达22.04%的AP(平均精度)改进,在异常定位上的mIoU(平均交并比)提升了13.9%,展示了其在复杂环境和分布外场景中的强大泛化能力。我们的数据集和代码将在[提供的URL]处发布。
https://arxiv.org/abs/2505.21962
Small Object Detection (SOD) poses significant challenges due to limited information and the model's low class prediction score. While Transformer-based detectors have shown promising performance, their potential for SOD remains largely unexplored. In typical DETR-like frameworks, the CNN backbone network, specialized in aggregating local information, struggles to capture the necessary contextual information for SOD. The multiple attention layers in the Transformer Encoder face difficulties in effectively attending to small objects and can also lead to blurring of features. Furthermore, the model's lower class prediction score of small objects compared to large objects further increases the difficulty of SOD. To address these challenges, we introduce a novel approach called Cross-DINO. This approach incorporates the deep MLP network to aggregate initial feature representations with both short and long range information for SOD. Then, a new Cross Coding Twice Module (CCTM) is applied to integrate these initial representations to the Transformer Encoder feature, enhancing the details of small objects. Additionally, we introduce a new kind of soft label named Category-Size (CS), integrating the Category and Size of objects. By treating CS as new ground truth, we propose a new loss function called Boost Loss to improve the class prediction score of the model. Extensive experimental results on COCO, WiderPerson, VisDrone, AI-TOD, and SODA-D datasets demonstrate that Cross-DINO efficiently improves the performance of DETR-like models on SOD. Specifically, our model achieves 36.4% APs on COCO for SOD with only 45M parameters, outperforming the DINO by +4.4% APS (36.4% vs. 32.0%) with fewer parameters and FLOPs, under 12 epochs training setting. The source codes will be available at this https URL.
小目标检测(SOD)面临诸多挑战,主要是由于信息有限和模型的类别预测得分较低。尽管基于Transformer的探测器已显示出良好的性能,但它们在小目标检测方面的潜力尚未得到充分探索。在典型的DETR框架中,CNN骨干网络专门用于汇聚局部信息,在捕捉必要的上下文信息以进行小目标检测时显得力不从心。Transformer编码器中的多层注意力机制难以有效关注到小物体,并且还可能导致特征模糊。此外,模型对小对象的类别预测得分低于大对象进一步增加了SOD的难度。 为解决这些挑战,我们提出了一种新的方法叫作Cross-DINO(跨DINO)。这种方法通过使用深度MLP网络来汇聚初始特征表示与短程和长程信息,以应对SOD。随后应用一个新的交叉编码两次模块(CCTM),将这些初始表示集成到Transformer编码器的特性中,从而增强对小目标细节的捕捉能力。 此外,我们还引入了一种新的软标签称为类别-大小(CS),这种标签整合了对象的类别和大小信息。通过将CS作为新的真实值对待,我们提出了一种新的损失函数——Boost Loss,以提高模型的类别预测得分。 在COCO、WiderPerson、VisDrone、AI-TOD和SODA-D等数据集上的大量实验结果表明,Cross-DINO能够有效地提升DETR类似模型在小目标检测中的性能。具体而言,在仅有45M参数的情况下,我们的模型在COCO上实现了36.4%的APs(平均精度)用于SOD,比DINO高出+4.4% APS(36.4% vs 32.0%),且参数和FLOPs更少。这在仅训练12个epoch的情况下得以实现。 源代码将在[此处](https://this-url.com)提供。
https://arxiv.org/abs/2505.21868
The safety of autonomous systems in dynamic and hazardous environments poses significant challenges. This paper presents a testing approach named SCALOFT for systematically assessing the safety of an autonomous aerial drone in a mine. SCALOFT provides a framework for developing diverse test cases, real-time monitoring of system behaviour, and detection of safety violations. Detected violations are then logged with unique identifiers for detailed analysis and future improvement. SCALOFT helps build a safety argument by monitoring situation coverage and calculating a final coverage measure. We have evaluated the performance of this approach by deliberately introducing seeded faults into the system and assessing whether SCALOFT is able to detect those faults. For a small set of plausible faults, we show that SCALOFT is successful in this.
在动态和危险环境中自主系统的安全问题带来了重大挑战。本文提出了一种名为SCALOFT的测试方法,用于系统性地评估自主空中无人机在矿山环境中的安全性。SCALOFT提供了一个框架,用于开发多样化的测试案例、实时监控系统行为以及检测安全违规情况。检测到的违规情况将被记录并附上独特的标识符,以便进行详细分析和未来改进。通过监测场景覆盖率和计算最终覆盖度量,SCALOFT有助于构建安全性论证。我们通过对系统故意引入已知错误来评估该方法的表现,并检查SCALOFT是否能够检测出这些错误。对于一组可能的故障情况,结果显示SCALOFT在这方面取得了成功。
https://arxiv.org/abs/2505.20969
Path planning is a critical component in autonomous drone operations, enabling safe and efficient navigation through complex environments. Recent advances in foundation models, particularly large language models (LLMs) and vision-language models (VLMs), have opened new opportunities for enhanced perception and intelligent decision-making in robotics. However, their practical applicability and effectiveness in global path planning remain relatively unexplored. This paper proposes foundation model-guided path planners (FM-Planner) and presents a comprehensive benchmarking study and practical validation for drone path planning. Specifically, we first systematically evaluate eight representative LLM and VLM approaches using standardized simulation scenarios. To enable effective real-time navigation, we then design an integrated LLM-Vision planner that combines semantic reasoning with visual perception. Furthermore, we deploy and validate the proposed path planner through real-world experiments under multiple configurations. Our findings provide valuable insights into the strengths, limitations, and feasibility of deploying foundation models in real-world drone applications and providing practical implementations in autonomous flight. Project site: this https URL.
路径规划是自主无人机操作中的关键组成部分,它能够使无人机在复杂环境中实现安全和高效的导航。近年来,在基础模型领域,尤其是大规模语言模型(LLMs)和视觉-语言模型(VLMs)方面取得了显著进展,这些进步为机器人技术提供了增强感知能力和智能决策的新机会。然而,这些模型在全球路径规划中的实际适用性和有效性仍然相对未被充分探索。 本文提出了一种由基础模型引导的路径规划器(FM-Planner),并进行了全面的基准测试研究和实践验证以应用于无人机路径规划中。具体而言,我们首先使用标准化仿真场景对八种代表性的LLM和VLM方法进行全面评估。为了实现有效的实时导航,我们设计了一个集成的语言模型-视觉规划器,该规划器结合了语义推理与视觉感知能力。此外,我们在多种配置下通过实际实验部署并验证了所提出的路径规划方案。 我们的研究结果为将基础模型应用于现实世界中的无人机应用提供了宝贵见解,并展示了其在自主飞行中实现的实用实施方案的有效性、局限性和可行性。项目网站:[此URL](https://this-url.com)。
https://arxiv.org/abs/2505.20783
Landing safely in crowded urban environments remains an essential yet challenging endeavor for Unmanned Aerial Vehicles (UAVs), especially in emergency situations. In this work, we propose a risk-aware approach that harnesses semantic segmentation to continuously evaluate potential hazards in the drone's field of view. By using a specialized deep neural network to assign pixel-level risk values and applying an algorithm based on risk maps, our method adaptively identifies a stable Safe Landing Zone (SLZ) despite moving critical obstacles such as vehicles, people, etc., and other visual challenges like shifting illumination. A control system then guides the UAV toward this low-risk region, employing altitude-dependent safety thresholds and temporal landing point stabilization to ensure robust descent trajectories. Experimental validation in diverse urban environments demonstrates the effectiveness of our approach, achieving over 90% landing success rates in very challenging real scenarios, showing significant improvements in various risk metrics. Our findings suggest that risk-oriented vision methods can effectively help reduce the risk of accidents in emergency landing situations, particularly in complex, unstructured, urban scenarios, densely populated with moving risky obstacles, while potentiating the true capabilities of UAVs in complex urban operations.
安全着陆在拥挤的城市环境中仍然是无人驾驶航空器(UAV)的一项重要且具有挑战性的任务,特别是在紧急情况下。本文提出了一种基于语义分割的风险感知方法,该方法能够连续评估无人机视场内的潜在危险。通过使用专门的深度神经网络为每个像素分配风险值,并结合基于风险图的算法,我们的方法能够在存在移动障碍物(如车辆、行人等)及其他视觉挑战(如光照变化)的情况下自适应地识别稳定的着陆区域(SLZ)。随后,控制系统会引导UAV前往这一低风险区域,在此过程中采用高度依赖的安全阈值和时间上的着陆点稳定化措施以确保稳健的下降轨迹。实验验证在各种城市环境中进行,并展示了我们方法的有效性,即使在非常具有挑战性的现实场景中也实现了超过90%的成功着陆率,同时在多种风险指标上表现出显著改进。 我们的研究结果表明,基于视觉的风险导向方法可以有效地帮助减少紧急降落情况下的事故风险,尤其是在复杂且结构化较差的城市环境中,这种环境充满了移动的危险障碍物。此外,这也有助于发挥UAV在复杂城市操作中的真正能力。
https://arxiv.org/abs/2505.20423
In cluttered spaces, such as forests, drone picking up a payload via an abseil claw is an open challenge, as the cable is likely tangled and blocked by the branches and obstacles. To address such a challenge, in this work, a cooperative aerial system is proposed, which consists of a payload drone and a dexterous rappelling end droid. The two ends are linked via a Kevlar tether cable. The end droid is actuated by four propellers, which enable mid-air dexterous adjustment of clawing angle and guidance of cable movement. To avoid tanglement and rappelling obstacles, a trajectory optimization method that integrates cable length constraints and dynamic feasibility is developed, which guarantees safe pickup. A tether cable dynamic model is established to evaluate real-time cable status, considering both taut and sagging conditions. Simulation and real-world experiments are conducted to demonstrate that the proposed system is capable of picking up payload in cluttered spaces. As a result, the end droid can reach the target point successfully under cable constraints and achieve passive retrieval during the lifting phase without propulsion, which enables effective and efficient aerial manipulation.
在杂乱的空间环境中,如森林中,通过悬垂抓取装置拾起负载对于无人机来说是一个开放性的挑战,因为电缆可能会被缠绕并受到树枝和其他障碍物的阻挡。为了解决这一难题,在这项工作中提出了一种协作式空中系统方案,该系统由一个载荷无人机和一个灵巧攀爬末端机器人组成,并通过凯夫拉尔绳索相连。末端机器人配备有四个推进器,这使得它能够在空中灵活地调整抓取角度并引导电缆运动。 为了避开缠绕及攀爬障碍物的问题,本研究开发了一种集成电缆长度限制和动态可行性的轨迹优化方法,以确保安全拾起负载。此外还建立了一个绳索动力学模型来实时评估缆线状态,并考虑到绷紧和松弛两种情况。 通过模拟实验与实际应用的测试证明了该系统能够在杂乱的空间中成功拾取载荷。由此可知,在满足电缆约束条件下,末端机器人能够到达目标点并实现提升阶段中的被动回收(无需推进力),这使得空中操作既有效又高效。
https://arxiv.org/abs/2505.19980
Early identification of weeds is essential for effective management and control, and there is growing interest in automating the process using computer vision techniques coupled with AI methods. However, challenges associated with training AI-based weed identification models, such as limited expert-verified data and complexity and variability in morphological features, have hindered progress. To address these issues, we present WeedNet, the first global-scale weed identification model capable of recognizing an extensive set of weed species, including noxious and invasive plant species. WeedNet is an end-to-end real-time weed identification pipeline and uses self-supervised learning, fine-tuning, and enhanced trustworthiness strategies. WeedNet achieved 91.02% accuracy across 1,593 weed species, with 41% species achieving 100% accuracy. Using a fine-tuning strategy and a Global-to-Local approach, the local Iowa WeedNet model achieved an overall accuracy of 97.38% for 85 Iowa weeds, most classes exceeded a 90% mean accuracy per class. Testing across intra-species dissimilarity (developmental stages) and inter-species similarity (look-alike species) suggests that diversity in the images collected, spanning all the growth stages and distinguishable plant characteristics, is crucial in driving model performance. The generalizability and adaptability of the Global WeedNet model enable it to function as a foundational model, with the Global-to-Local strategy allowing fine-tuning for region-specific weed communities. Additional validation of drone- and ground-rover-based images highlights the potential of WeedNet for integration into robotic platforms. Furthermore, integration with AI for conversational use provides intelligent agricultural and ecological conservation consulting tools for farmers, agronomists, researchers, land managers, and government agencies across diverse landscapes.
早期识别杂草对于有效的管理和控制至关重要,利用计算机视觉技术结合人工智能方法来自动化这一过程的兴趣日益增长。然而,训练基于AI的杂草识别模型面临的挑战,如缺乏专家验证的数据以及形态特征的复杂性和多变性,阻碍了进展。为了解决这些问题,我们提出了WeedNet,这是第一个能够识别大量杂草种类(包括有害和入侵植物)的全球规模杂草识别模型。WeedNet是一个端到端的实时杂草识别流水线,并使用自监督学习、微调和增强信任度策略。 WeedNet在1593种杂草中达到了91.02%的整体准确率,其中41%的种类实现了100%的准确率。采用了一种微调策略以及全球到局部的方法后,针对爱荷华州85种杂草的本地版本Iowa WeedNet取得了97.38%的整体准确率,并且大部分类别达到了每类平均超过90%的准确性。 跨物种内部差异(生长阶段)和跨物种相似性的测试表明,收集图像中的多样性,在所有生长阶段和可识别植物特征上展开,对于推动模型性能至关重要。全球WeedNet模型的一般化和适应性使它能够作为基础模型发挥作用,并且通过从全球到局部的方法可以针对特定地区的杂草群落进行微调。 此外,使用无人机和地面探测器收集图像的进一步验证突显了将WeedNet集成到机器人平台中的潜力。另外,与人工智能结合用于对话式的应用,为农民、农学家、研究人员、土地管理者以及政府机构提供了智能农业和生态保育咨询工具,在多种景观中具有广泛的应用价值。
https://arxiv.org/abs/2505.18930
Multi-view multi-object tracking (MVMOT) has found widespread applications in intelligent transportation, surveillance systems, and urban management. However, existing studies rarely address genuinely free-viewpoint MVMOT systems, which could significantly enhance the flexibility and scalability of cooperative tracking systems. To bridge this gap, we first construct the Multi-Drone Multi-Object Tracking (MDMOT) dataset, captured by mobile drone swarms across diverse real-world scenarios, initially establishing the first benchmark for multi-object tracking in arbitrary multi-view environment. Building upon this foundation, we propose \textbf{FusionTrack}, an end-to-end framework that reasonably integrates tracking and re-identification to leverage multi-view information for robust trajectory association. Extensive experiments on our MDMOT and other benchmark datasets demonstrate that FusionTrack achieves state-of-the-art performance in both single-view and multi-view tracking.
多视角多目标跟踪(MVMOT)在智能交通、监控系统和城市管理系统中得到了广泛应用。然而,现有研究很少涉及真正的自由视点MVMOT系统,这可以显著提高合作跟踪系统的灵活性和可扩展性。为填补这一空白,我们首先构建了由移动无人机群在各种现实场景下捕捉的Multi-Drone Multi-Object Tracking(MDMOT)数据集,初步建立了任意多视角环境下多目标跟踪的第一个基准。在此基础上,我们提出了\textbf{FusionTrack},这是一个端到端框架,合理地整合了跟踪和重识别功能,以利用多视图信息进行稳健的轨迹关联。在我们的MDMOT和其他基准数据集上进行了广泛的实验表明,FusionTrack在单视图和多视图跟踪方面均达到了最先进的性能。
https://arxiv.org/abs/2505.18727
Omnidirectional aerial vehicles (OMAVs) have opened up a wide range of possibilities for inspection, navigation, and manipulation applications using drones. In this paper, we introduce MorphEUS, a morphable co-axial quadrotor that can control position and orientation independently with high efficiency. It uses a paired servo motor mechanism for each rotor arm, capable of pointing the vectored-thrust in any arbitrary direction. As compared to the \textit{state-of-the-art} OMAVs, we achieve higher and more uniform force/torque reachability with a smaller footprint and minimum thrust cancellations. The overactuated nature of the system also results in resiliency to rotor or servo-motor failures. The capabilities of this quadrotor are particularly well-suited for contact-based infrastructure inspection and close-proximity imaging of complex geometries. In the accompanying control pipeline, we present theoretical results for full controllability, almost-everywhere exponential stability, and thrust-energy optimality. We evaluate our design and controller on high-fidelity simulations showcasing the trajectory-tracking capabilities of the vehicle during various tasks. Supplementary details and experimental videos are available on the project webpage.
全向空中飞行器(OMAVs)为无人机在检查、导航和操作应用方面开辟了广泛的可能性。在这篇论文中,我们介绍了MorphEUS,这是一种可变形的共轴四旋翼机,能够高效地独立控制位置和方向。它采用了一种配对伺服电机机制,每个旋翼臂都有这种机制,可以将矢量推力指向任意方向。与最先进的OMAV相比,我们在较小的占地面积、最小化推力抵消的情况下实现了更高的力量/扭矩可达性和更加均匀的力量/扭矩分布。系统的过驱动特性还使其在遇到旋转或伺服电机故障时具有更强的韧性。这种四旋翼机的功能特别适合基于接触的基础设施检查和复杂几何形状的近距离成像任务。 我们在伴随控制管道中提出了关于完全可控性、几乎处处指数稳定性和推力能量最优性的理论结果。我们通过高保真模拟对我们的设计和控制器进行了评估,展示了车辆在执行各种任务时的轨迹跟踪能力。项目的更多详细信息和实验视频可在项目网页上找到。
https://arxiv.org/abs/2505.18270
Carbon dioxide (CO$_2$) emissions are critical indicators of both environmental impact and various industrial processes, including livestock management. We introduce CarboNeXT, a semantic segmentation framework for Optical Gas Imaging (OGI), designed to detect and quantify CO$_2$ emissions across diverse applications. Our approach integrates a multi-scale context aggregation network with UPerHead and auxiliary FCN components to effectively model both local details and global relationships in gas plume imagery. We contribute two novel datasets: (1) the Controlled Carbon Dioxide Release (CCR) dataset, which simulates gas leaks with systematically varied flow rates (10-100 SCCM), and (2) the Real Time Ankom (RTA) dataset, focusing on emissions from dairy cow rumen fluid in vitro experiments. Extensive evaluations demonstrate that CarboNeXT outperforms state-of-the-art methods, achieving 88.46% mIoU on CCR and 92.95% mIoU on RTA, with particular effectiveness in challenging low-flow scenarios. The model operates at 60.95 FPS, enabling real-time monitoring applications. Additionally, we propose CarboFormer, a lightweight variant with only 5.07M parameters that achieves 84.68 FPS, with competitive performance of 84.88% mIoU on CCR and 92.98% on RTA, making it suitable for resource-constrained platforms such as programmable drones. Our work advances both environmental sensing and precision livestock management by providing robust tools for CO$_2$ emission analysis, with a specific focus on livestock applications.
二氧化碳(CO₂)排放是环境影响和各种工业过程的重要指标,包括畜牧业管理。我们介绍了CarboNeXT,这是一个面向光学气体成像(OGI)的语义分割框架,旨在检测并量化跨多种应用场合下的二氧化碳排放情况。我们的方法结合了多尺度上下文聚合网络、UPerHead组件以及辅助FCN部件,以有效建模气柱图像中的局部细节和全局关系。 我们贡献了两个新颖的数据集:(1)控制性二氧化碳释放(CCR)数据集,该数据集模拟不同流量范围(10-100 SCCM)下的气体泄漏情况;(2)实时安夸姆(RTA)数据集,专注于乳牛瘤胃体外实验中的排放研究。 详尽的评估表明,CarboNeXT在最先进的方法中表现突出,在CCR和RTA数据集上分别取得了88.46% mIoU和92.95% mIoU的成绩。特别是在低流量等挑战性场景下尤为有效。该模型以每秒60.95帧的速度运行,支持实时监测应用。 此外,我们提出了CarboFormer,这是一种参数仅为5.07M的轻量级变体,在CCR和RTA数据集上分别达到了84.68FPS和84.88% mIoU以及92.98%,适用于资源受限平台如可编程无人机等场景。 我们的工作通过提供用于二氧化碳排放分析的强大工具,推进了环境传感及精准畜牧业管理的发展,并特别关注于畜牧业应用。
https://arxiv.org/abs/2506.05360