Flapping-wing drones have attracted significant attention due to their biomimetic flight. They are considered more human-friendly due to their characteristics such as low noise and flexible wings, making them suitable for human-drone interactions. However, few studies have explored the practical interaction between humans and flapping-wing drones. On establishing a physical interaction system with flapping-wing drones, we can acquire inspirations from falconers who guide birds of prey to land on their arms. This interaction interprets the human body as a dynamic landing platform, which can be utilized in various scenarios such as crowded or spatially constrained environments. Thus, in this study, we propose a falconry-like interaction system in which a flapping-wing drone performs a palm landing motion on a human hand. To achieve a safe approach toward humans, we design a trajectory planning method that considers both physical and psychological factors of the human safety such as the drone's velocity and distance from the user. We use a commercial flapping platform with our implemented motion planning and conduct experiments to evaluate the palm landing performance and safety. The results demonstrate that our approach enables safe and smooth hand landing interactions. To the best of our knowledge, it is the first time to achieve a contact-based interaction between flapping-wing drones and humans.
扑翼无人机因其仿生飞行特性而吸引了广泛的关注。由于其低噪音和灵活翅膀等特点,被认为更有利于与人类友好互动,适合进行人机交互。然而,很少有研究探讨人类与扑翼无人机之间实际的互动方式。建立一个物理互动系统时,我们可以借鉴猎鹰驯养者引导猛禽停落在手臂上的方法。这种方法将人体视为动态着陆平台,在拥挤或空间受限环境中具有多种应用场景。因此,本研究提出了一种类似猎人与鹰互动的系统,使扑翼无人机能够在人的手掌上进行安全着陆。 为了实现安全接近人类的目标,我们设计了一个轨迹规划方法,考虑了物理和心理因素的安全性,如无人机的速度和距离等。我们使用一个商业化的扑翼平台并实施了该运动计划,在实验中评估了手部着陆性能与安全性。结果表明,我们的方法能够实现安全、流畅的手部接触互动。 据我们所知,这是首次实现了扑翼无人机与人类之间的接触式交互。
https://arxiv.org/abs/2507.17144
The autonomous formation flight of fixed-wing drones is hard when the coordination requires the actuation over their speeds since they are critically bounded and aircraft are mostly designed to fly at a nominal airspeed. This paper proposes an algorithm to achieve formation flights of fixed-wing drones without requiring any actuation over their speed. In particular, we guide all the drones to travel over specific paths, e.g., parallel straight lines, and we superpose an oscillatory behavior onto the guiding vector field that drives the drones to the paths. This oscillation enables control over the average velocity along the path, thereby facilitating inter-drone coordination. Each drone adjusts its oscillation amplitude distributively in a closed-loop manner by communicating with neighboring agents in an undirected and connected graph. A novel consensus algorithm is introduced, leveraging a non-negative, asymmetric saturation function. This unconventional saturation is justified since negative amplitudes do not make drones travel backward or have a negative velocity along the path. Rigorous theoretical analysis of the algorithm is complemented by validation through numerical simulations and a real-world formation flight.
固定翼无人机的自主编队飞行在需要通过调节其速度来进行协调时会变得非常困难,因为它们的速度受到了关键限制,并且大多数飞机设计为以名义空速飞行。本文提出了一种算法,在不需调整任何无人机速度的情况下实现固定翼无人机的编队飞行。具体来说,我们指导所有无人机沿着特定路径飞行,例如平行直线,并在驱动无人机到达该路径的引导矢量场中叠加振荡行为。这种振荡使沿路径的平均速度受到控制,从而促进了无人机之间的协调。每个无人机通过与无向且连通图中的邻近代理进行通信,在闭环模式下分布式地调整其振幅。本文还引入了一种新的共识算法,利用非负、不对称饱和函数。这种非常规的饱和操作是合理的,因为负振幅不会使无人机倒退或沿路径产生负速度。对算法进行了严格的理论分析,并通过数值模拟和实际编队飞行验证了该方法的有效性。
https://arxiv.org/abs/2507.16458
Combining an energy-efficient drone with a high-capacity truck for last-mile package delivery can benefit operators and customers by reducing delivery times and environmental impact. However, directly integrating drone flight dynamics into the combinatorially hard truck route planning problem is challenging. Simplified models that ignore drone flight physics can lead to suboptimal delivery plans. We propose an integrated formulation for the joint problem of truck route and drone trajectory planning and a new end-to-end solution approach that combines optimization and machine learning to generate high-quality solutions in practical online runtimes. Our solution method trains neural network predictors based on offline solutions to the drone trajectory optimization problem instances to approximate drone flight times, and uses these approximations to optimize the overall truck-and-drone delivery plan by augmenting an existing order-first-split-second heuristic. Our method explicitly incorporates key kinematics and energy equations in drone trajectory optimization, and thereby outperforms state-of-the-art benchmarks that ignore drone flight physics. Extensive experimentation using synthetic datasets and real-world case studies shows that the integration of drone trajectories into package delivery planning substantially improves system performance in terms of tour duration and drone energy consumption. Our modeling and computational framework can help delivery planners achieve annual savings worth millions of dollars while also benefiting the environment.
将节能型无人机与高载货量卡车结合用于最后一公里包裹递送,可以同时为运营商和客户减少配送时间并降低环境影响。然而,直接将无人机飞行动力学整合到组合难题的卡车路线规划问题中颇具挑战性。忽略无人机飞行物理学的简化模型会导致次优的交付计划。我们提出了一种针对卡车路线与无人机轨迹规划联合问题的整体建模方法,并引入一种新的端到端解决方案方法,该方法结合了优化和机器学习技术,能够在实际在线运行时间内生成高质量方案。 我们的解决策略基于离线解算出的无人机轨迹优化实例训练神经网络预测器来近似无人机飞行时间,并利用这些近似值通过增强现有的“先排序后分割”启发式算法来优化整体卡车与无人机递送计划。我们在无人机轨迹优化中显式地纳入了关键运动学和能量方程,从而超越了忽略无人机飞行物理学的最新基准方法。 通过对合成数据集及现实世界案例研究进行广泛的实验验证,我们发现将无人机路径整合到包裹配送规划之中显著提升了系统的性能指标,包括行程时间和无人机能耗。我们的建模与计算框架可以帮助递送计划者实现每年数百万美元的成本节省,并且对环境也有积极影响。
https://arxiv.org/abs/2507.16259
The increasing demand for autonomous systems in complex and dynamic environments has driven significant research into intelligent path planning methodologies. For decades, graph-based search algorithms, linear programming techniques, and evolutionary computation methods have served as foundational approaches in this domain. Recently, deep reinforcement learning (DRL) has emerged as a powerful method for enabling autonomous agents to learn optimal navigation strategies through interaction with their environments. This survey provides a comprehensive overview of traditional approaches as well as the recent advancements in DRL applied to path planning tasks, focusing on autonomous vehicles, drones, and robotic platforms. Key algorithms across both conventional and learning-based paradigms are categorized, with their innovations and practical implementations highlighted. This is followed by a thorough discussion of their respective strengths and limitations in terms of computational efficiency, scalability, adaptability, and robustness. The survey concludes by identifying key open challenges and outlining promising avenues for future research. Special attention is given to hybrid approaches that integrate DRL with classical planning techniques to leverage the benefits of both learning-based adaptability and deterministic reliability, offering promising directions for robust and resilient autonomous navigation.
在复杂和动态环境中对自主系统的日益增长的需求,推动了智能路径规划方法的研究。几十年来,基于图的搜索算法、线性编程技术和进化计算方法一直是该领域的基础方法。最近,深度强化学习(DRL)作为一种强大的技术手段出现,使自主代理能够通过与环境互动来学习最优导航策略。这篇综述提供了传统方法以及近期在路径规划任务中应用于深度强化学习的最新进展的全面概述,重点关注自动驾驶汽车、无人机和机器人平台。文中对传统的和基于学习的方法的关键算法进行了分类,并强调了它们各自的创新点及实际应用情况。然后详细讨论了这些技术在计算效率、可扩展性、适应性和鲁棒性方面的各自优缺点。综述最后指出了关键的开放挑战,并概述了未来研究的有前途的方向,特别关注将深度强化学习与经典规划技术相结合的混合方法,以充分利用基于学习的灵活性和确定性的可靠性,为自主导航提供稳健且可持续的发展方向。
https://arxiv.org/abs/2507.15469
Unmanned Aerial Vehicles (UAVs) are increasingly used in surveillance, logistics, agriculture, disaster management, and military operations. Accurate detection and classification of UAV flight states, such as hovering, cruising, ascending, or transitioning, which are essential for safe and effective operations. However, conventional time series classification (TSC) methods often lack robustness and generalization for dynamic UAV environments, while state of the art(SOTA) models like Transformers and LSTM based architectures typically require large datasets and entail high computational costs, especially with high-dimensional data streams. This paper proposes a novel framework that integrates a Transformer-based Generative Adversarial Network (GAN) with Multiple Instance Locally Explainable Learning (MILET) to address these challenges in UAV flight state classification. The Transformer encoder captures long-range temporal dependencies and complex telemetry dynamics, while the GAN module augments limited datasets with realistic synthetic samples. MIL is incorporated to focus attention on the most discriminative input segments, reducing noise and computational overhead. Experimental results show that the proposed method achieves superior accuracy 96.5% on the DroneDetect dataset and 98.6% on the DroneRF dataset that outperforming other SOTA approaches. The framework also demonstrates strong computational efficiency and robust generalization across diverse UAV platforms and flight states, highlighting its potential for real-time deployment in resource constrained environments.
无人飞行器(UAV,即无人机)在监视、物流、农业、灾害管理和军事行动中使用日益增多。准确检测和分类无人机的飞行状态(如悬停、巡航、上升或过渡),对于安全有效的操作至关重要。然而,传统的时序分类(TSC)方法通常缺乏对动态无人机环境的鲁棒性和泛化能力,而最先进的模型(SOTA)如Transformer和基于LSTM架构的方法往往需要大型数据集,并且处理高维数据流时计算成本高昂。 本文提出了一种新颖框架,该框架结合了基于Transformer的生成对抗网络(GAN)与多实例局部可解释学习(MILET),以解决无人机飞行状态分类中的上述挑战。Transformer编码器能够捕捉长期的时间依赖关系和复杂的遥测动态,而GAN模块则通过生成现实的合成样本扩充有限的数据集。引入MI(多实例学习)关注最具有判别性的输入片段,从而减少噪声并降低计算开销。 实验结果显示,在DroneDetect数据集上提出的模型达到96.5%的准确率,在DroneRF数据集上的准确率为98.6%,均优于其他SOTA方法。该框架还展示了强大的计算效率和跨不同无人机平台及飞行状态的良好泛化能力,这突显了其在资源受限环境中的实时部署潜力。
https://arxiv.org/abs/2507.14592
Vision transformer (ViT) models, when coupled with interpretation models, are regarded as secure and challenging to deceive, making them well-suited for security-critical domains such as medical applications, autonomous vehicles, drones, and robotics. However, successful attacks on these systems can lead to severe consequences. Recent research on threats targeting ViT models primarily focuses on generating the smallest adversarial perturbations that can deceive the models with high confidence, without considering their impact on model interpretations. Nevertheless, the use of interpretation models can effectively assist in detecting adversarial examples. This study investigates the vulnerability of transformer models to adversarial attacks, even when combined with interpretation models. We propose an attack called "AdViT" that generates adversarial examples capable of misleading both a given transformer model and its coupled interpretation model. Through extensive experiments on various transformer models and two transformer-based interpreters, we demonstrate that AdViT achieves a 100% attack success rate in both white-box and black-box scenarios. In white-box scenarios, it reaches up to 98% misclassification confidence, while in black-box scenarios, it reaches up to 76% misclassification confidence. Remarkably, AdViT consistently generates accurate interpretations in both scenarios, making the adversarial examples more difficult to detect.
当视觉变换器(Vision Transformer,简称 ViT)模型与解释模型结合使用时,这些组合系统被认为具有很高的安全性,并且难以被欺骗。因此,它们非常适合需要高度安全性的领域,例如医疗应用、自动驾驶汽车、无人机和机器人技术。然而,成功攻击这些系统的威胁可能导致严重后果。 最近针对ViT模型的威胁研究主要集中在生成最小的对抗性扰动上,这种扰动能够以很高的置信度欺骗模型,但没有考虑对模型解释的影响。不过,使用解释模型可以有效帮助检测对抗样本。这项研究表明了即使与解释模型结合后,Transformer模型仍然容易受到对抗攻击的影响。 我们提出了一种名为“AdViT”的攻击方法,该方法生成能够同时误导给定变换器模型及其耦合的解释模型的对抗性样例。通过在多种变压器模型和两种基于变压器的解释器上进行广泛的实验,我们展示了AdViT在白盒和黑盒场景下均能实现100%的攻击成功率。在白盒场景中,它能够达到高达98%的误分类置信度,而在黑盒场景中,则能达到76%的误分类置信度。 特别值得注意的是,在两种情况下,AdViT都能持续生成准确的解释,使对抗样本更难以被检测出来。
https://arxiv.org/abs/2507.14248
Real-time trajectory planning for unmanned aerial vehicles (UAVs) in dynamic environments remains a key challenge due to high computational demands and the need for fast, adaptive responses. Traditional Particle Swarm Optimization (PSO) methods, while effective for offline planning, often struggle with premature convergence and latency in real-time scenarios. To overcome these limitations, we propose PE-PSO, an enhanced PSO-based online trajectory planner. The method introduces a persistent exploration mechanism to preserve swarm diversity and an entropy-based parameter adjustment strategy to dynamically adapt optimization behavior. UAV trajectories are modeled using B-spline curves, which ensure path smoothness while reducing optimization complexity. To extend this capability to UAV swarms, we develop a multi-agent framework that combines genetic algorithm (GA)-based task allocation with distributed PE-PSO, supporting scalable and coordinated trajectory generation. The distributed architecture allows for parallel computation and decentralized control, enabling effective cooperation among agents while maintaining real-time performance. Comprehensive simulations demonstrate that the proposed framework outperforms conventional PSO and other swarm-based planners across several metrics, including trajectory quality, energy efficiency, obstacle avoidance, and computation time. These results confirm the effectiveness and applicability of PE-PSO in real-time multi-UAV operations under complex environmental conditions.
在动态环境中,无人飞行器(UAV)的实时轨迹规划由于高计算需求和对快速、适应性响应的需求而仍然是一个关键挑战。传统粒子群优化(PSO)方法虽然适用于离线规划,但在实时场景中往往面临过早收敛和延迟的问题。为克服这些限制,我们提出了一种基于增强PSO的方法——PE-PSO,这是一种用于在线轨迹规划的改进型PSO方法。该方法引入了持久探索机制以保持群多样性,并采用基于熵参数调整策略来动态适应优化行为。UAV轨迹使用B样条曲线进行建模,在确保路径平滑性的同时降低了优化复杂度。 为了扩展这种能力到UAV集群,我们开发了一个多智能体框架,结合遗传算法(GA)任务分配与分布式PE-PSO,支持可伸缩和协调的轨迹生成。该分布架构允许并行计算和分散控制,从而在保持实时性能的同时促进各代理之间的有效协作。 全面的模拟测试表明,所提出的框架在包括路径质量、能源效率、障碍物规避及计算时间在内的多个指标上优于传统的PSO和其他群集规划器。这些结果确认了PE-PSO方法在复杂环境条件下进行实时多UAV操作的有效性和适用性。
https://arxiv.org/abs/2507.13647
Prior work has demonstrated that incorporating well-known quantum tunnelling (QT) probability into neural network models effectively captures important nuances of human perception, particularly in the recognition of ambiguous objects and sentiment analysis. In this paper, we employ novel QT-based neural networks and assess their effectiveness in distinguishing customised CIFAR-format images of military and civilian vehicles, as well as sentiment, using a proprietary military-specific vocabulary. We suggest that QT-based models can enhance multimodal AI applications in battlefield scenarios, particularly within human-operated drone warfare contexts, imbuing AI with certain traits of human reasoning.
先前的研究已经证明,将众所周知的量子隧穿(QT)概率融入神经网络模型中,能够有效地捕捉到人类感知的重要细微差别,特别是在识别模棱两可的对象和情感分析方面。在这篇论文中,我们采用了基于新型QT的神经网络,并评估了它们在使用专有的军事特定词汇表来区分定制的CIFAR格式的军用和民用车辆图像以及情感方面的有效性。我们认为,基于QT的模型可以增强多模式人工智能应用在战场场景中的效果,特别是在有人操作的无人机战争环境中,使AI具备某些人类推理的特点。
https://arxiv.org/abs/2507.18645
Scene understanding enables intelligent agents to interpret and comprehend their environment. While existing large vision-language models (LVLMs) for scene understanding have primarily focused on indoor household tasks, they face two significant limitations when applied to outdoor large-scale scene understanding. First, outdoor scenarios typically encompass larger-scale environments observed through various sensors from multiple viewpoints (e.g., bird view and terrestrial view), while existing indoor LVLMs mainly analyze single visual modalities within building-scale contexts from humanoid viewpoints. Second, existing LVLMs suffer from missing multidomain perception outdoor data and struggle to effectively integrate 2D and 3D visual information. To address the aforementioned limitations, we build the first multidomain perception outdoor scene understanding dataset, named \textbf{\underline{SVM-City}}, deriving from multi\textbf{\underline{S}}cale scenarios with multi\textbf{\underline{V}}iew and multi\textbf{\underline{M}}odal instruction tuning data. It contains $420$k images and $4, 811$M point clouds with $567$k question-answering pairs from vehicles, low-altitude drones, high-altitude aerial planes, and satellite. To effectively fuse the multimodal data in the absence of one modality, we introduce incomplete multimodal learning to model outdoor scene understanding and design the LVLM named \textbf{\underline{City-VLM}}. Multimodal fusion is realized by constructing a joint probabilistic distribution space rather than implementing directly explicit fusion operations (e.g., concatenation). Experimental results on three typical outdoor scene understanding tasks show City-VLM achieves $18.14 \%$ performance surpassing existing LVLMs in question-answering tasks averagely. Our method demonstrates pragmatic and generalization performance across multiple outdoor scenes.
场景理解使智能代理能够解释和理解其环境。尽管现有的大型视觉语言模型(LVLM)在室内家居任务的场景理解方面已经取得了进展,但当应用于户外大规模场景理解时,它们面临着两个重要的局限性。首先,室外情景通常包括通过多种传感器从多个视角观察到的大规模环境(如鸟瞰图和地面视图),而现有的室内LVLM主要分析单个视觉模态,在人类视角下的建筑物尺度背景下进行研究。其次,现有的LVLM缺乏多领域感知户外数据,并且难以有效整合2D和3D视觉信息。 为了克服上述限制,我们建立了第一个多领域感知的户外场景理解数据集,名为**SVM-City**,源自多种规模的情景、多视角和多模态指令调优数据。该数据集中含有420,000张图像和4,811M点云,并从车辆、低空无人机、高空飞机及卫星采集了567,000个问题-答案对。 为了有效融合缺失单一模态情况下的多模式数据,我们引入了一种不完整的多模态学习方法来建模户外场景理解,并设计了一个名为**City-VLM**的LVLM。通过构建联合概率分布空间而非直接进行显式融合操作(如串联),实现了多模式融合。 在三个典型的户外场景理解任务上的实验结果显示,City-VLM在问答任务上平均超越现有LVLM18.14%的性能表现。我们的方法在多个户外场景中展示了实用性和泛化能力。
https://arxiv.org/abs/2507.12795
This paper introduces an autonomous UAV vision system for continuous, real-time tracking of marine animals, specifically sharks, in dynamic marine environments. The system integrates an onboard computer with a stabilised RGB-D camera and a custom-trained OSTrack pipeline, enabling visual identification under challenging lighting, occlusion, and sea-state conditions. A key innovation is the inter-UAV handoff protocol, which enables seamless transfer of tracking responsibilities between drones, extending operational coverage beyond single-drone battery limitations. Performance is evaluated on a curated shark dataset of 5,200 frames, achieving a tracking success rate of 81.9\% during real-time flight control at 100 Hz, and robustness to occlusion, illumination variation, and background clutter. We present a seamless UAV handoff framework, where target transfer is attempted via high-confidence feature matching, achieving 82.9\% target coverage. These results confirm the viability of coordinated UAV operations for extended marine tracking and lay the groundwork for scalable, autonomous monitoring.
本文介绍了一种自主无人飞行器(UAV)视觉系统,用于在动态海洋环境中连续、实时追踪海洋动物,特别是鲨鱼。该系统将机载计算机与稳定化的RGB-D相机和经过定制训练的OSTrack管道集成在一起,能够在具有挑战性的光照条件、遮挡以及海况下进行视觉识别。一个关键创新是无人机之间的交接协议,使跟踪责任可以在不同无人机之间无缝转移,从而在单个无人机电池限制之外延长操作范围。 该系统的性能通过一组精心挑选的包含5,200帧的鲨鱼数据集进行了评估,在实时飞行控制中以100 Hz频率运行时,追踪成功率为81.9%,并且对遮挡、光照变化以及背景杂乱具有良好的鲁棒性。我们提出了一种无缝无人机交接框架,在此框架下通过高置信度特征匹配尝试目标转移,实现82.9%的目标覆盖范围。 这些结果确认了协调的UAV操作在扩展海洋追踪中的可行性,并为可扩展、自主监测系统奠定了基础。
https://arxiv.org/abs/2507.12763
Small object detection remains a challenging problem in the field of object detection. To address this challenge, we propose an enhanced YOLOv8-based model, SOD-YOLO. This model integrates an ASF mechanism in the neck to enhance multi-scale feature fusion, adds a Small Object Detection Layer (named P2) to provide higher-resolution feature maps for better small object detection, and employs Soft-NMS to refine confidence scores and retain true positives. Experimental results demonstrate that SOD-YOLO significantly improves detection performance, achieving a 36.1% increase in mAP$_{50:95}$ and 20.6% increase in mAP$_{50}$ on the VisDrone2019-DET dataset compared to the baseline model. These enhancements make SOD-YOLO a practical and efficient solution for small object detection in UAV imagery. Our source code, hyper-parameters, and model weights are available at this https URL.
小目标检测在物体检测领域仍是一个具有挑战性的问题。为了解决这个问题,我们提出了一种基于YOLOv8增强的模型SOD-YOLO。该模型在颈部集成了ASF机制以增强多尺度特征融合,并添加了一个用于提供更高分辨率特征图的小目标检测层(命名为P2),从而提高了小物体的检测能力。此外,还采用了Soft-NMS来优化置信度评分并保留真阳性结果。实验结果显示,与基线模型相比,SOD-YOLO在VisDrone2019-DET数据集上显著提升了检测性能,mAP$_{50:95}$提高了36.1%,mAP$_{50}$提高了20.6%。这些改进使SOD-YOLO成为无人机图像中小目标检测的实用和高效解决方案。我们的源代码、超参数以及模型权重可以在[此处](https://此URL)获取(请将"this https URL"替换为实际链接)。
https://arxiv.org/abs/2507.12727
Bounding box regression (BBR) is fundamental to object detection, where the regression loss is crucial for accurate localization. Existing IoU-based losses often incorporate handcrafted geometric penalties to address IoU's non-differentiability in non-overlapping cases and enhance BBR performance. However, these penalties are sensitive to box shape, size, and distribution, often leading to suboptimal optimization for small objects and undesired behaviors such as bounding box enlargement due to misalignment with the IoU objective. To address these limitations, we propose InterpIoU, a novel loss function that replaces handcrafted geometric penalties with a term based on the IoU between interpolated boxes and the target. By using interpolated boxes to bridge the gap between predictions and ground truth, InterpIoU provides meaningful gradients in non-overlapping cases and inherently avoids the box enlargement issue caused by misaligned penalties. Simulation results further show that IoU itself serves as an ideal regression target, while existing geometric penalties are both unnecessary and suboptimal. Building on InterpIoU, we introduce Dynamic InterpIoU, which dynamically adjusts interpolation coefficients based on IoU values, enhancing adaptability to scenarios with diverse object distributions. Experiments on COCO, VisDrone, and PASCAL VOC show that our methods consistently outperform state-of-the-art IoU-based losses across various detection frameworks, with particularly notable improvements in small object detection, confirming their effectiveness.
边界框回归(BBR)在目标检测中至关重要,其中回归损失对于精确定位极为关键。现有的基于IoU的损失通常会采用手工设计的几何惩罚项来解决非重叠情况下的IoU不可微问题,并提高BBR性能。然而,这些惩罚项对边界框形状、大小和分布非常敏感,常常导致小目标优化不佳以及由于与IoU目标不一致而导致的边界框扩大的不良行为。 为了解决这些问题,我们提出了InterpIoU,这是一种新的损失函数,它用基于插值边界框与目标之间IoU的新项来替代手工设计的几何惩罚项。通过使用插值边界框作为预测和真实值之间的桥梁,InterpIoU能够在非重叠情况下提供有意义的梯度,并且从根本上避免了由于不匹配的惩罚而导致的边界框扩大的问题。仿真结果进一步表明,IoU本身就是一个理想的回归目标,现有的几何惩罚不仅没有必要,而且效果不佳。 基于InterpIoU,我们还引入了动态插值IoU(Dynamic InterpIoU),该方法能够根据IoU值动态调整插值系数,从而提高对具有不同对象分布场景的适应能力。在COCO、VisDrone和PASCAL VOC上的实验显示,我们的方法在各种检测框架中始终优于现有的基于IoU的损失函数,并且特别是在小目标检测方面取得了显著改进,证实了其有效性。
https://arxiv.org/abs/2507.12420
Sidewalk delivery robots are a promising solution for urban freight distribution, reducing congestion compared to trucks and providing a safer, higher-capacity alternative to drones. However, unreliable travel times on sidewalks due to pedestrian density, obstacles, and varying infrastructure conditions can significantly affect their efficiency. This study addresses the robust route planning problem for sidewalk robots, explicitly accounting for travel time uncertainty due to varying sidewalk conditions. Optimization is integrated with simulation to reproduce the effect of obstacles and pedestrian flows and generate realistic travel times. The study investigates three different approaches to derive uncertainty sets, including budgeted, ellipsoidal, and support vector clustering (SVC)-based methods, along with a distributionally robust method to solve the shortest path (SP) problem. A realistic case study reproducing pedestrian patterns in Stockholm's city center is used to evaluate the efficiency of robust routing across various robot designs and environmental conditions. The results show that, when compared to a conventional SP, robust routing significantly enhances operational reliability under variable sidewalk conditions. The Ellipsoidal and DRSP approaches outperform the other methods, yielding the most efficient paths in terms of average and worst-case delay. Sensitivity analyses reveal that robust approaches consistently outperform the conventional SP, particularly for sidewalk delivery robots that are wider, slower, and have more conservative navigation behaviors. These benefits are even more pronounced in adverse weather conditions and high pedestrian congestion scenarios.
人行道送货机器人是城市货运分配的一个有前景的解决方案,与卡车相比可以减少拥堵,并提供比无人机更安全、运载能力更高的替代方案。然而,由于行人密度、障碍物和不同的人行道条件导致不可靠的行驶时间,会对它们的效率产生显著影响。这项研究解决了人行道机器人的鲁棒路线规划问题,明确考虑了因人行道状况变化而导致的旅行时间不确定性。该方法结合了优化与仿真技术来重现障碍物和行人流量的影响,并生成现实中的旅行时间数据。 本研究探讨了三种不同的方法来推导不确定集,包括预算、椭圆体以及基于支持向量聚类(SVC)的方法,还包括了解决最短路径问题的分布鲁棒性方法。通过再现斯德哥尔摩市中心行人的模式进行了一项现实案例研究,以评估在不同机器人设计和环境条件下各种稳健路由方案的有效性。 结果表明,与传统的最短路径相比,在可变的人行道条件下采用鲁棒路线规划可以显著提高操作的可靠性。椭圆体方法和分布鲁棒性最短路径(DRSP)方法优于其他方法,从平均延迟和最坏情况下的延迟来看,它们产生了最为高效的路径。 敏感性分析表明,稳健的方法始终优于传统的最短路径,尤其是对于更宽、速度较慢且导航行为更加保守的人行道送货机器人而言。在恶劣天气条件和行人拥堵严重的场景下,这些优势更为明显。
https://arxiv.org/abs/2507.12067
Vision is an essential part of attitude control for many flying animals, some of which have no dedicated sense of gravity. Flying robots, on the other hand, typically depend heavily on accelerometers and gyroscopes for attitude stabilization. In this work, we present the first vision-only approach to flight control for use in generic environments. We show that a quadrotor drone equipped with a downward-facing event camera can estimate its attitude and rotation rate from just the event stream, enabling flight control without inertial sensors. Our approach uses a small recurrent convolutional neural network trained through supervised learning. Real-world flight tests demonstrate that our combination of event camera and low-latency neural network is capable of replacing the inertial measurement unit in a traditional flight control loop. Furthermore, we investigate the network's generalization across different environments, and the impact of memory and different fields of view. While networks with memory and access to horizon-like visual cues achieve best performance, variants with a narrower field of view achieve better relative generalization. Our work showcases vision-only flight control as a promising candidate for enabling autonomous, insect-scale flying robots.
视觉在许多飞行动物的姿态控制中扮演着至关重要的角色,其中一些动物甚至没有专门的重力感知器官。相比之下,飞行机器人通常高度依赖于加速度计和陀螺仪来进行姿态稳定。在这项工作中,我们提出了首个仅依靠视觉信息进行飞行控制的方法,并适用于通用环境。我们展示了装备有向下朝向事件相机的四旋翼无人机能够仅通过事件流来估计其姿态和旋转速率,从而实现无需惯性传感器的飞行控制。我们的方法采用了一种小型循环卷积神经网络(RNN-CNN),并通过监督学习进行训练。 实际飞行测试表明,我们组合使用的事件相机和低延迟神经网络能够在传统飞行控制回路中替代惯性测量单元(IMU)。此外,我们还研究了该网络在不同环境中的泛化能力以及记忆和视野范围变化的影响。虽然具有记忆功能并能访问类似地平线视觉线索的网络表现最佳,但视野较窄的变体则表现出更好的相对泛化能力。 这项工作展示了仅依赖视觉信息进行飞行控制作为一种有前景的技术,能够为实现自主、昆虫规模级别的飞行机器人提供支持。
https://arxiv.org/abs/2507.11302
This work presents a vision-based underwater exploration and inspection autonomy solution integrated into Ariel, a custom vision-driven underwater robot. Ariel carries a $5$ camera and IMU based sensing suite, enabling a refraction-aware multi-camera visual-inertial state estimation method aided by a learning-based proprioceptive robot velocity prediction method that enhances robustness against visual degradation. Furthermore, our previously developed and extensively field-verified autonomous exploration and general visual inspection solution is integrated on Ariel, providing aerial drone-level autonomy underwater. The proposed system is field-tested in a submarine dry dock in Trondheim under challenging visual conditions. The field demonstration shows the robustness of the state estimation solution and the generalizability of the path planning techniques across robot embodiments.
这项工作提出了一种基于视觉的水下探索和检测自主解决方案,该方案集成到了Ariel中——一款定制的、以视觉驱动的水下机器人。Ariel搭载了一个包括$5$个摄像头和IMU(惯性测量单元)传感器在内的感知套件,能够实现一种具有折射意识的多摄像头视觉-惯性状态估计方法,并借助基于学习的方法来预测机器人的本体感觉速度,从而增强了在视觉降级情况下系统的鲁棒性。此外,我们之前开发并经过广泛实地验证的自主探索和通用视觉检测解决方案也被集成到了Ariel上,在水下实现了类似空中无人机级别的自主能力。所提出的系统在特隆赫姆的一个潜艇干船坞中进行了实地测试,面对的是极具挑战性的视觉条件。现场演示表明了状态估计解决方案的鲁棒性以及路径规划技术跨机器人实现的一般适用性。
https://arxiv.org/abs/2507.10003
Using neuromorphic computing for robotics applications has gained much attention in recent year due to the remarkable ability of Spiking Neural Networks (SNNs) for high-precision yet low memory and compute complexity inference when implemented in neuromorphic hardware. This ability makes SNNs well-suited for autonomous robot applications (such as in drones and rovers) where battery resources and payload are typically limited. Within this context, this paper studies the use of SNNs for performing direct robot navigation and obstacle avoidance from LIDAR data. A custom robot platform equipped with a LIDAR is set up for collecting a labeled dataset of LIDAR sensing data together with the human-operated robot control commands used for obstacle avoidance. Crucially, this paper provides what is, to the best of our knowledge, a first focused study about the importance of neuron membrane leakage on the SNN precision when processing LIDAR data for obstacle avoidance. It is shown that by carefully tuning the membrane potential leakage constant of the spiking Leaky Integrate-and-Fire (LIF) neurons used within our SNN, it is possible to achieve on-par robot control precision compared to the use of a non-spiking Convolutional Neural Network (CNN). Finally, the LIDAR dataset collected during this work is released as open-source with the hope of benefiting future research.
https://arxiv.org/abs/2507.09538
For precise, efficient, and safe drone landings, ground platforms should real-time, accurately locate descending drones and guide them to designated spots. While mmWave sensing combined with cameras improves localization accuracy, lower sampling frequency of traditional frame cameras compared to mmWave radar creates bottlenecks in system throughput. In this work, we upgrade traditional frame camera with event camera, a novel sensor that harmonizes in sampling frequency with mmWave radar within ground platform setup, and introduce mmE-Loc, a high-precision, low-latency ground localization system designed for precise drone landings. To fully exploit the \textit{temporal consistency} and \textit{spatial complementarity} between these two modalities, we propose two innovative modules: \textit{(i)} the Consistency-instructed Collaborative Tracking module, which further leverages the drone's physical knowledge of periodic micro-motions and structure for accurate measurements extraction, and \textit{(ii)} the Graph-informed Adaptive Joint Optimization module, which integrates drone motion information for efficient sensor fusion and drone localization. Real-world experiments conducted in landing scenarios with a drone delivery company demonstrate that mmE-Loc significantly outperforms state-of-the-art methods in both accuracy and latency.
为了实现精确、高效和安全的无人机着陆,地面平台需要实时且准确地定位下降中的无人机,并引导其降落到指定位置。毫米波(mmWave)感应技术与摄像头结合虽然提高了定位精度,但传统帧相机相比毫米波雷达较低的采样频率成为系统吞吐量瓶颈。为此,我们在传统的帧相机基础上引入了事件相机(一种新型传感器),该传感器在地面平台设置中能够实现与毫米波雷达同步的采样频率,并推出mmE-Loc——一款专为精确无人机着陆设计的高精度、低延迟地面对准定位系统。 为了充分利用这两种模式之间的时间一致性与时空互补性,我们提出了两个创新模块:(i)协同追踪模块,该模块进一步利用了无人机周期微动和结构方面的物理知识以实现准确测量;(ii)基于图的信息自适应联合优化模块,该模块整合无人机运动信息,实现了高效的传感器融合及无人机定位。 在与一家无人机配送公司合作的实地实验中,我们发现mmE-Loc在精度和延迟方面均显著优于现有技术。
https://arxiv.org/abs/2507.09469
Path smoothness is often overlooked in path imitation learning from expert demonstrations. In this paper, we introduce a novel learning method, termed deep angular A* (DAA*), by incorporating the proposed path angular freedom (PAF) into A* to improve path similarity through adaptive path smoothness. The PAF aims to explore the effect of move angles on path node expansion by finding the trade-off between their minimum and maximum values, allowing for high adaptiveness for imitation learning. DAA* improves path optimality by closely aligning with the reference path through joint optimization of path shortening and smoothing, which correspond to heuristic distance and PAF, respectively. Throughout comprehensive evaluations on 7 datasets, including 4 maze datasets, 2 video-game datasets, and a real-world drone-view dataset containing 2 scenarios, we demonstrate remarkable improvements of our DAA* over neural A* in path similarity between the predicted and reference paths with a shorter path length when the shortest path is plausible, improving by 9.0% SPR, 6.9% ASIM, and 3.9% PSIM. Furthermore, when jointly learning pathfinding with both path loss and path probability map loss, DAA* significantly outperforms the state-of-the-art TransPath by 6.7% SPR, 6.5% PSIM, and 3.7% ASIM. We also discuss the minor trade-off between path optimality and search efficiency where applicable.
路径平滑性在从专家演示中进行路径模仿学习时常常被忽视。在这篇论文中,我们引入了一种新的学习方法——深度角度A*(DAA*),通过将提出的路径角自由度(PAF)融入A*算法来提高路径相似性,并通过自适应路径平滑性实现这一目标。PAF旨在探索移动角度对路径节点扩展的影响,通过寻找其最小值和最大值之间的权衡,从而为模仿学习提供高度的适应性。DAA*通过联合优化路径缩短和平滑(分别对应启发式距离和PAF)来提高路径最优性,并且与参考路径紧密匹配。 我们在7个数据集上进行了全面评估,包括4个迷宫数据集、2个视频游戏数据集以及一个包含两个场景的真实世界无人机视图数据集。我们展示了DAA*在预测路径与参考路径之间的相似度方面显著优于神经A*,尤其是在最短路径是可能的情况下,路径长度更短。具体而言,在标准化路径评分(SPR)、角度相似性指标(ASIM)和路径相似性指标(PSIM)上分别提高了9.0%、6.9% 和3.9%。 此外,当同时学习路径搜索与路径损失和路径概率地图损失时,DAA*显著优于最先进的TransPath,在上述三个度量标准中分别提高了6.7%、3.7% 和 6.5%。我们还讨论了在适用情况下的路径最优性和搜索效率之间的微小权衡。 通过这种方式,DAA*不仅改善了路径模仿学习的平滑性,还在提高路径相似性和减少预测误差方面超越了现有方法。
https://arxiv.org/abs/2507.09305
Accurate position estimation is essential for modern navigation systems deployed in autonomous platforms, including ground vehicles, marine vessels, and aerial drones. In this context, Visual Simultaneous Localisation and Mapping (VSLAM) - which includes Visual Odometry - relies heavily on the reliable extraction of salient feature points from the visual input data. In this work, we propose an embedded implementation of an unsupervised architecture capable of detecting and describing feature points. It is based on a quantised SuperPoint convolutional neural network. Our objective is to minimise the computational demands of the model while preserving high detection quality, thus facilitating efficient deployment on platforms with limited resources, such as mobile or embedded systems. We implemented the solution on an FPGA System-on-Chip (SoC) platform, specifically the AMD/Xilinx Zynq UltraScale+, where we evaluated the performance of Deep Learning Processing Units (DPUs) and we also used the Brevitas library and the FINN framework to perform model quantisation and hardware-aware optimisation. This allowed us to process 640 x 480 pixel images at up to 54 fps on an FPGA platform, outperforming state-of-the-art solutions in the field. We conducted experiments on the TUM dataset to demonstrate and discuss the impact of different quantisation techniques on the accuracy and performance of the model in a visual odometry task.
准确的位置估计对于部署在自主平台(包括地面车辆、海洋船只和无人机)上的现代导航系统至关重要。在此背景下,视觉同时定位与地图构建 (VSLAM) 依赖于从视觉输入数据中可靠地提取显著特征点,其中包含视觉里程计。本文提出了一种嵌入式实现方案,该方案基于未监督架构,并能够检测和描述特征点。我们的方法采用量化后的 SuperPoint 卷积神经网络。 我们的目标是通过保持高检测质量的同时最小化模型的计算需求,从而在资源有限(如移动或嵌入式系统)平台上实现高效部署。我们在 AMD/Xilinx Zynq UltraScale+ FPGA 系统级芯片 (SoC) 平台上实现了这一解决方案,并评估了深度学习处理单元(DPUs) 的性能。此外,我们使用 Brevitas 库和 FINN 框架进行模型量化及硬件感知优化。这使我们能够在FPGA平台上以高达54 fps的帧率处理640x480像素图像,优于该领域的现有解决方案。 我们在 TUM 数据集上进行了实验,展示了不同量化技术对视觉里程计任务中模型精度和性能的影响,并讨论了这些影响。
https://arxiv.org/abs/2507.07903
How to enable robots to predict the outcomes of their own motion intentions in three-dimensional space has been a fundamental problem in embodied intelligence. To explore more general spatial imagination capabilities, here we present AirScape, the first world model designed for six-degree-of-freedom aerial agents. AirScape predicts future observation sequences based on current visual inputs and motion intentions. Specifically, we construct an dataset for aerial world model training and testing, which consists of 11k video-intention pairs. This dataset includes first-person-view videos capturing diverse drone actions across a wide range of scenarios, with over 1,000 hours spent annotating the corresponding motion intentions. Then we develop a two-phase training schedule to train a foundation model -- initially devoid of embodied spatial knowledge -- into a world model that is controllable by motion intentions and adheres to physical spatio-temporal constraints.
如何使机器人能够在三维空间中预测自己运动意图的结果,一直是具身智能中的一个基本问题。为了探索更广泛的时空想象能力,我们在此介绍AirScape,这是首个为具备六自由度的空中代理设计的世界模型。AirScape基于当前视觉输入和运动意图来预测未来的观察序列。具体而言,我们构建了一个用于空中国际象棋世界模型训练与测试的数据集,其中包括了11,000对视频-意图组合。该数据集包含了第一人称视角的视频,捕捉到了无人机在各种场景中的多样动作,并且花费超过1,000小时标注对应的运动意图。然后,我们开发了一种两阶段的训练计划,将一个初始不包含具身时空知识的基础模型训练成可以根据运动意图控制、并遵守物理时空约束的世界模型。
https://arxiv.org/abs/2507.08885