Learning control policies for complex, long-horizon tasks is a central challenge in robotics and autonomous systems. Signal Temporal Logic (STL) offers a powerful and expressive language for specifying such tasks, but its non-Markovian nature and inherent sparse reward make it difficult to be solved via standard Reinforcement Learning (RL) algorithms. Prior RL approaches focus only on limited STL fragments or use STL robustness scores as sparse terminal rewards. In this paper, we propose TGPO, Temporal Grounded Policy Optimization, to solve general STL tasks. TGPO decomposes STL into timed subgoals and invariant constraints and provides a hierarchical framework to tackle the problem. The high-level component of TGPO proposes concrete time allocations for these subgoals, and the low-level time-conditioned policy learns to achieve the sequenced subgoals using a dense, stage-wise reward signal. During inference, we sample various time allocations and select the most promising assignment for the policy network to rollout the solution trajectory. To foster efficient policy learning for complex STL with multiple subgoals, we leverage the learned critic to guide the high-level temporal search via Metropolis-Hastings sampling, focusing exploration on temporally feasible solutions. We conduct experiments on five environments, ranging from low-dimensional navigation to manipulation, drone, and quadrupedal locomotion. Under a wide range of STL tasks, TGPO significantly outperforms state-of-the-art baselines (especially for high-dimensional and long-horizon cases), with an average of 31.6% improvement in task success rate compared to the best baseline. The code will be available at this https URL
https://arxiv.org/abs/2510.00225
Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning grounded in perception and action to generalize beyond training distributions and adapt to novel real-world situations. These capabilities, however, also create new security risks. In this paper, we introduce CHAI (Command Hijacking against embodied AI), a new class of prompt-based attacks that exploit the multimodal language interpretation abilities of Large Visual-Language Models (LVLMs). CHAI embeds deceptive natural language instructions, such as misleading signs, in visual input, systematically searches the token space, builds a dictionary of prompts, and guides an attacker model to generate Visual Attack Prompts. We evaluate CHAI on four LVLM agents; drone emergency landing, autonomous driving, and aerial object tracking, and on a real robotic vehicle. Our experiments show that CHAI consistently outperforms state-of-the-art attacks. By exploiting the semantic and multimodal reasoning strengths of next-generation embodied AI systems, CHAI underscores the urgent need for defenses that extend beyond traditional adversarial robustness.
嵌入式人工智能(AI)承诺通过使用基于感知和行动的常识推理来处理机器人车辆系统中数据稀缺的情况,从而能够超越训练分布并在新的现实世界情境下进行适应。然而,这些能力也带来了新的安全风险。本文介绍了 CHAI (针对嵌入式 AI 的命令劫持攻击),这是一种新型的基于提示的攻击方法,利用大型视觉-语言模型(LVLM)的多模态语言解释能力。CHAI 将欺骗性的自然语言指令,如误导性标志,嵌入到视觉输入中,并系统地搜索词元空间、构建提示字典,引导攻击者模型生成视觉攻击提示。 我们在四种 LVLM 代理上评估了 CHAI:无人机紧急着陆、自动驾驶和空中物体跟踪,以及在实际的机器人车辆上。我们的实验表明,CHAI 在面对最先进的攻击时始终表现出色。通过利用下一代嵌入式 AI 系统的语义和多模态推理能力,CHAI 强调了需要超越传统对抗鲁棒性的防御措施以应对安全挑战。
https://arxiv.org/abs/2510.00181
Autonomous drones must often respond to sudden events, such as alarms, faults, or unexpected changes in their environment, that require immediate and adaptive decision-making. Traditional approaches rely on safety engineers hand-coding large sets of recovery rules, but this strategy cannot anticipate the vast range of real-world contingencies and quickly becomes incomplete. Recent advances in embodied AI, powered by large visual language models, provide commonsense reasoning to assess context and generate appropriate actions in real time. We demonstrate this capability in a simulated urban benchmark in the Unreal Engine, where drones dynamically interpret their surroundings and decide on sudden maneuvers for safe landings. Our results show that embodied AI makes possible a new class of adaptive recovery and decision-making pipelines that were previously infeasible to design by hand, advancing resilience and safety in autonomous aerial systems.
自主无人机常常需要应对突然事件,如警报、故障或环境中的意外变化,这些情况要求即时和适应性的决策。传统方法依赖于安全工程师手动编写大量的恢复规则,但这种方法无法预测现实世界中广泛多样的突发状况,并且很快就会变得不完整。近年来,基于大型视觉语言模型的具身人工智能(Embodied AI)的进步提供了常识推理能力,能够评估环境并实时生成适当的行动。我们在虚幻引擎中的模拟城市基准测试中展示了这种能力,无人机可以动态地解释周围的环境并在需要时做出突然的机动以确保安全着陆。我们的研究结果表明,具身AI使得设计以前难以手动实现的新一代适应性恢复和决策流程成为可能,从而提升了自主空中系统的弹性和安全性。
https://arxiv.org/abs/2510.00167
Drones, or unmanned aerial vehicles (UAVs), have become powerful tools across domains-from industry to the arts. In documentary filmmaking, they offer dynamic, otherwise unreachable perspectives, transforming how stories are told. Wildlife documentaries especially benefit, yet drones also raise ethical concerns: the risk of disturbing the animals they aim to capture. This paper introduces CineWild, an autonomous UAV framework that combines robotics, cinematography, and ethics. Built on model predictive control, CineWild dynamically adjusts flight paths and camera settings to balance cinematic quality with animal welfare. Key features include adaptive zoom for filming from acoustic and visual safe distances, path-planning that avoids an animal's field of view, and smooth, low-noise maneuvers. CineWild exemplifies interdisciplinary innovation-bridging engineering, visual storytelling, and environmental ethics. We validate the system through simulation studies and will release the code upon acceptance.
无人机,或者说是无人驾驶飞行器(UAV),已经成为了从工业到艺术等各个领域中的强大工具。在纪录片制作中,它们提供了动态且难以企及的视角,从而改变了故事讲述的方式。野生动物纪录片尤其受益于此技术,然而,无人机也引发了伦理方面的担忧:存在干扰拍摄对象动物的风险。本文介绍了CineWild——一个结合了机器人学、电影摄影和伦理考量的自主飞行器框架。该系统基于模型预测控制技术,能够动态调整飞行路径和相机设置,在保证影像质量的同时兼顾动物福利。其关键特性包括采用自适应变焦功能,确保在声学和视觉安全距离内拍摄;通过避开动物视线范围内的路线规划减少干扰,并执行平稳且低噪音的操控动作。 CineWild 体现了跨学科创新——它跨越了工程、影视叙事与环境伦理之间的界限。我们将通过模拟研究验证该系统的有效性,在论文接受后会公开发布代码。
https://arxiv.org/abs/2509.24921
Autonomous technology, which has become widespread today, appears in many different configurations such as mobile robots, manipulators, and drones. One of the most important tasks of these vehicles during autonomous operations is path planning. In the literature, path planners are generally divided into two categories: probabilistic and deterministic methods. In the analysis of probabilistic methods, the common problem of almost all methods is observed in narrow passage environments. In this paper, a novel sampler is proposed that deterministically identifies narrow passage environments using occupancy grid maps and accordingly increases the amount of sampling in these regions. The codes of the algorithm is provided as open source. To evaluate the performance of the algorithm, benchmark studies are conducted in three distinct categories: specific and random simulation environments, and a real-world environment. As a result, it is observed that our algorithm provides higher performance in planning time and number of milestones compared to the baseline samplers.
今天广泛使用的自主技术以多种形式存在,如移动机器人、机械臂和无人机。在这些车辆的自主操作过程中,路径规划是最重要的任务之一。文献中通常将路径规划方法分为概率方法和确定性方法两大类。在对概率方法进行分析时,几乎所有方法在狭窄通道环境中都遇到了共同的问题。本文提出了一种新颖的采样器,该采样器使用占用栅格地图以确定性方式识别狭窄通道环境,并相应地增加这些区域内的采样量。算法代码作为开源提供。为了评估算法性能,我们在三类不同的基准测试中进行了研究:特定和随机仿真环境以及真实世界环境。结果表明,与基线采样器相比,我们的算法在规划时间和里程碑数量方面表现更优。
https://arxiv.org/abs/2509.23288
Aerial-view object detection is a critical technology for real-world applications such as natural resource monitoring, traffic management, and UAV-based search and rescue. Detecting tiny objects in high-resolution aerial imagery presents a long-standing challenge due to their limited visual cues and the difficulty of modeling global context in complex scenes. Existing methods are often hampered by delayed contextual fusion and inadequate non-linear modeling, failing to effectively use global information to refine shallow features and thus encountering a performance bottleneck. To address these challenges, we propose FMC-DETR, a novel framework with frequency-decoupled fusion for aerial-view object detection. First, we introduce the Wavelet Kolmogorov-Arnold Transformer (WeKat) backbone, which applies cascaded wavelet transforms to enhance global low-frequency context perception in shallow features while preserving fine-grained details, and employs Kolmogorov-Arnold networks to achieve adaptive non-linear modeling of multi-scale dependencies. Next, a lightweight Cross-stage Partial Fusion (CPF) module reduces redundancy and improves multi-scale feature interaction. Finally, we introduce the Multi-Domain Feature Coordination (MDFC) module, which unifies spatial, frequency, and structural priors to to balance detail preservation and global enhancement. Extensive experiments on benchmark aerial-view datasets demonstrate that FMC-DETR achieves state-of-the-art performance with fewer parameters. On the challenging VisDrone dataset, our model achieves improvements of 6.5% AP and 8.2% AP50 over the baseline, highlighting its effectiveness in tiny object detection. The code can be accessed at this https URL.
高空视角下的目标检测是一项关键技术,广泛应用于自然资源监测、交通管理和基于无人机的搜索救援等实际应用中。在高分辨率航拍图像中识别微小目标一直是一个长期存在的挑战,原因是这些目标提供的视觉线索有限,并且难以在一个复杂场景中建模全局上下文信息。现有方法往往受到延迟性上下文融合和非线性建模不足的影响,在有效利用全局信息来改进浅层特征方面遇到了性能瓶颈。 为了解决这些问题,我们提出了FMC-DETR框架,该框架采用频率解耦的融合技术来进行高空视角下的目标检测。首先,我们引入了小波Kolmogorov-Arnold变换器(WeKat)作为骨干网络,它应用级联的小波变换来增强浅层特征中的全局低频上下文感知能力,同时保留精细细节,并使用Kolmogorov-Arnold网络实现多尺度依赖关系的自适应非线性建模。接下来,我们采用轻量化的跨阶段部分融合(CPF)模块减少冗余并提高多尺度特征交互作用。最后,我们引入了多域特征协调(MDFC)模块,该模块整合空间、频率和结构先验知识,以平衡细节保留与全局增强之间的关系。 在基准高空视角数据集上的广泛实验表明,FMC-DETR能够在参数更少的情况下达到最先进的性能水平。特别是在具有挑战性的VisDrone数据集中,我们的模型相比基线模型分别提高了6.5% AP和8.2% AP50的精度(AP表示平均精度),这突显了其在微小目标检测上的有效性。 相关代码可在此网址访问:[https URL]
https://arxiv.org/abs/2509.23056
The real-time detection of small objects in complex scenes, such as the unmanned aerial vehicle (UAV) photography captured by drones, has dual challenges of detecting small targets (<32 pixels) and maintaining real-time efficiency on resource-constrained platforms. While YOLO-series detectors have achieved remarkable success in real-time large object detection, they suffer from significantly higher false negative rates for drone-based detection where small objects dominate, compared to large object scenarios. This paper proposes HierLight-YOLO, a hierarchical feature fusion and lightweight model that enhances the real-time detection of small objects, based on the YOLOv8 architecture. We propose the Hierarchical Extended Path Aggregation Network (HEPAN), a multi-scale feature fusion method through hierarchical cross-level connections, enhancing the small object detection accuracy. HierLight-YOLO includes two innovative lightweight modules: Inverted Residual Depthwise Convolution Block (IRDCB) and Lightweight Downsample (LDown) module, which significantly reduce the model's parameters and computational complexity without sacrificing detection capabilities. Small object detection head is designed to further enhance spatial resolution and feature fusion to tackle the tiny object (4 pixels) detection. Comparison experiments and ablation studies on the VisDrone2019 benchmark demonstrate state-of-the-art performance of HierLight-YOLO.
在复杂场景中实时检测小型物体(如无人机拍摄的照片)面临两个挑战:一是要检测小于32像素的小目标,二是要在资源受限平台上保持实时效率。尽管YOLO系列的探测器在大型物体的实时检测方面取得了显著的成功,但在以小物体为主导的无人机图像检测任务中,它们会出现更高的漏检率(即假阴性率),相比于处理大型物体的情况。 本文提出了一种基于YOLOv8架构的新方法——HierLight-YOLO。这是一种通过层级特征融合来增强小型目标实时检测能力的轻量级模型。我们引入了层次化扩展路径聚合网络(HEPAN),一种通过多尺度特征融合和层级跨层连接的方法,旨在提高小物体的检测精度。 HierLight-YOLO包括两个创新性的轻量化模块:倒残差深度卷积块(IRDCB)和轻量级下采样(LDown)模块。这些模块显著减少了模型参数数量和计算复杂度,同时不牺牲检测性能。此外,还设计了一个专门用于提升小物体空间分辨率及特征融合能力的检测头,以解决微小物体(4像素大小)的检测问题。 在VisDrone2019基准测试中的对比实验和消融研究表明,HierLight-YOLO达到了最先进的性能水平。
https://arxiv.org/abs/2509.22365
Aside from common challenges in remote sensing like small, sparse targets and computation cost limitations, detecting vehicles from UAV images in the Nordic regions faces strong visibility challenges and domain shifts caused by diverse levels of snow coverage. Although annotated data are expensive, unannotated data is cheaper to obtain by simply flying the drones. In this work, we proposed a sideload-CL-adaptation framework that enables the use of unannotated data to improve vehicle detection using lightweight models. Specifically, we propose to train a CNN-based representation extractor through contrastive learning on the unannotated data in the pretraining stage, and then sideload it to a frozen YOLO11n backbone in the fine-tuning stage. To find a robust sideload-CL-adaptation, we conducted extensive experiments to compare various fusion methods and granularity. Our proposed sideload-CL-adaptation model improves the detection performance by 3.8% to 9.5% in terms of mAP50 on the NVD dataset.
除了遥感中的常见挑战,如小而稀疏的目标和计算成本限制之外,在北欧地区从无人机(UAV)图像中检测车辆还面临着强烈的能见度问题以及由于雪覆盖程度不同而导致的领域变化。虽然标注数据的成本很高,但未标注的数据可以通过简单地飞行无人机来更便宜地获取。在这项工作中,我们提出了一种sideload-CL-adaptation框架,该框架能够利用未标注的数据通过轻量级模型提高车辆检测性能。具体来说,我们在预训练阶段通过对比学习在未标注数据上训练基于CNN的表示提取器,然后在微调阶段将其sideload到冻结的YOLO11n骨干网络中。为了找到一种稳健的sideload-CL-adaptation方法,我们进行了大量的实验来比较各种融合方法和粒度。我们的提出的sideload-CL-adaptation模型在NVD数据集上以mAP50为指标将检测性能提高了3.8%到9.5%。
https://arxiv.org/abs/2509.21916
Immediate damage assessment is essential after natural catastrophes; yet, conventional hand evaluation techniques are sluggish and perilous. Although satellite and unmanned aerial vehicle (UAV) photos offer extensive perspectives of impacted regions, current computer vision methodologies generally yield just classification labels or segmentation masks, so constraining their capacity to deliver a thorough situational comprehension. We introduce the Vision Language Caption Enhancer (VLCE), a multimodal system designed to produce comprehensive, contextually-informed explanations of disaster imagery. VLCE employs a dual-architecture approach: a CNN-LSTM model with a ResNet50 backbone pretrained on EuroSat satellite imagery for the xBD dataset, and a Vision Transformer (ViT) model pretrained on UAV pictures for the RescueNet dataset. Both systems utilize external semantic knowledge from ConceptNet and WordNet to expand vocabulary coverage and improve description accuracy. We assess VLCE in comparison to leading vision-language models (LLaVA and QwenVL) utilizing CLIPScore for semantic alignment and InfoMetIC for caption informativeness. Experimental findings indicate that VLCE markedly surpasses baseline models, attaining a maximum of 95.33% on InfoMetIC while preserving competitive semantic alignment. Our dual-architecture system demonstrates significant potential for improving disaster damage assessment by automating the production of actionable, information-dense descriptions from satellite and drone photos.
自然灾害发生后,立即进行损害评估至关重要;然而,传统的手工评估技术既缓慢又危险。虽然卫星和无人飞行器(UAV)拍摄的照片可以提供受影响区域的广泛视角,但当前的计算机视觉方法通常只能生成分类标签或分割掩模,这限制了它们提供全面情况理解的能力。我们引入了视觉语言描述增强器(VLCE),这是一个多模式系统,旨在产生对灾难图像的详尽、语境相关的解释。 VLCE采用双架构设计:对于xBD数据集,使用在EuroSat卫星图像上预训练的CNN-LSTM模型与ResNet50骨干网络;对于RescueNet数据集,则使用了在UAV图片上预训练的Vision Transformer(ViT)模型。这两个系统都利用ConceptNet和WordNet等外部语义知识来扩展词汇覆盖率并提高描述准确性。 我们通过CLIPScore评估VLCE与其他领先的视觉-语言模型(LLaVA和QwenVL)之间的语义对齐,并使用InfoMetIC评估标题信息的丰富程度。实验结果表明,VLCE显著超越了基准模型,在InfoMetIC上达到了最高95.33%的同时保持了竞争性的语义对齐度。 我们的双架构系统在从卫星和无人机照片中自动生成具有行动力的信息密集型描述方面展示了巨大的潜力,从而提高灾害损害评估的效率。
https://arxiv.org/abs/2509.21609
Multi-robot target tracking is a fundamental problem that requires coordinated monitoring of dynamic entities in applications such as precision agriculture, environmental monitoring, disaster response, and security surveillance. While Federated Learning (FL) has the potential to enhance learning across multiple robots without centralized data aggregation, its use in multi-Unmanned Aerial Vehicle (UAV) target tracking remains largely underexplored. Key challenges include limited onboard computational resources, significant data heterogeneity in FL due to varying targets and the fields of view, and the need for tight coupling between trajectory prediction and multi-robot planning. In this paper, we introduce DroneFL, the first federated learning framework specifically designed for efficient multi-UAV target tracking. We design a lightweight local model to predict target trajectories from sensor inputs, using a frozen YOLO backbone and a shallow transformer for efficient onboard training. The updated models are periodically aggregated in the cloud for global knowledge sharing. To alleviate the data heterogeneity that hinders FL convergence, DroneFL introduces a position-invariant model architecture with altitude-based adaptive instance normalization. Finally, we fuse predictions from multiple UAVs in the cloud and generate optimal trajectories that balance target prediction accuracy and overall tracking performance. Our results show that DroneFL reduces prediction error by 6%-83% and tracking distance by 0.4%-4.6% compared to a distributed non-FL framework. In terms of efficiency, DroneFL runs in real time on a Raspberry Pi 5 and has on average just 1.56 KBps data rate to the cloud.
多机器人目标跟踪是需要协调监控动态实体的基本问题,这在精准农业、环境监测、灾害响应和安全监控等应用中至关重要。虽然联邦学习(FL)具有通过分散式而非集中化的数据聚合来增强多个机器人之间学习的潜力,但在多无人机目标跟踪中的应用仍鲜有探索。这一领域面临的挑战包括有限的机载计算资源、由于不同目标及视野范围导致的数据异质性显著增加,以及轨迹预测与多机器人规划间紧密耦合的需求。本文中,我们提出了DroneFL,这是首个专为高效多无人机目标跟踪设计的联邦学习框架。 为了在每架无人机上进行高效的本地模型训练,我们设计了一种轻量级的局部模型,用于从传感器输入中预测目标轨迹。该模型采用冻结的YOLO主干和浅层Transformer结构,并通过基于高度适应的实例归一化来缓解位置不变性带来的数据异质性问题。 在云端定期聚合更新后的模型以实现全局知识共享后,DroneFL会在云服务器上融合来自多架无人机的预测结果,并生成综合了目标预测精度与整体跟踪性能的最佳轨迹。实验结果显示,相较于分布式的非联邦学习框架,DroneFL能够将预测误差降低6%-83%,并将追踪距离减少0.4%-4.6%。 此外,在效率方面,DroneFL可以在Raspberry Pi 5上实时运行,并且平均数据传输速率仅为1.56KBps。
https://arxiv.org/abs/2509.21523
We propose $\text{GMP}^{3}$, a multiphase global path planning framework that generates dynamically feasible three-dimensional trajectories for unmanned aerial vehicles (UAVs) operating in cluttered environments. The framework extends traditional path planning from Euclidean position spaces to the Lie group $\mathrm{SE}(3)$, allowing joint learning of translational motion and rotational dynamics. A modified Bellman-based operator is introduced to support reinforcement learning (RL) policy updates while leveraging prior trajectory information for improved convergence. $\text{GMP}^{3}$ is designed as a distributed framework in which agents influence each other and share policy information along the trajectory: each agent refines its assigned segment and shares with its neighbors via a consensus-based scheme, enabling cooperative policy updates and convergence toward a path shaped globally even under kinematic constraints. We also propose DroneManager, a modular ground control software that interfaces the planner with real UAV platforms via the MAVLink protocol, supporting real-time deployment and feedback. Simulation studies and indoor flight experiments validate the effectiveness of the proposed method in constrained 3D environments, demonstrating reliable obstacle avoidance and smooth, feasible trajectories across both position and orientation. The open-source implementation is available at this https URL
我们提出了一种多阶段全局路径规划框架 $\text{GMP}^{3}$,该框架能够为在复杂环境中运行的无人飞行器(UAV)生成动态可行的三维轨迹。此框架将传统的路径规划从欧几里得位置空间扩展到李群 $\mathrm{SE}(3)$,从而实现了平移运动和旋转动力学的联合学习。引入了一种基于贝尔曼原理修改的操作符以支持强化学习(RL)策略更新,并利用先前的轨迹信息提高收敛性。$\text{GMP}^{3}$ 被设计为一个分布式框架,在该框架中,各个代理相互影响并共享沿路径上的策略信息:每个代理细化其分配的部分并通过基于共识的方式与邻居分享,从而实现合作式的策略更新和在运动学约束条件下全局路径的收敛。此外,我们还提出了 DroneManager——一种模块化的地面控制软件,它通过 MAVLink 协议将规划器连接到真实的 UAV 平台,支持实时部署和反馈。仿真研究和室内飞行实验验证了所提方法在受限三维环境中的有效性,展示了可靠障碍物回避能力和位置与方向上的平滑可行轨迹。该开源实现可在以下网址获得:[此链接]
https://arxiv.org/abs/2509.21264
Conventional multi-rotors are under-actuated systems, hindering them from independently controlling attitude from position. In this study, we present several distinct configurations that incorporate additional control inputs for manipulating the angles of the propeller axes. This addresses the mentioned limitations, making the systems "omniorientational". We comprehensively derived detailed dynamic models for all introduced configurations and validated by a methodology using Simscape Multibody simulations. Two controllers are designed: a sliding mode controller for robust handling of disturbances and a novel PID-based controller with gravity compensation integrating linear and non-linear allocators, designed for computational efficiency. A custom control allocation strategy is implemented to manage the input-non-affine nature of these systems, seeking to maximize battery life by minimizing the "Power Consumption Factor" defined in this study. Moreover, the controllers effectively managed harsh disturbances and uncertainties. Simulations compare and analyze the proposed configurations and controllers, majorly considering their power consumption. Furthermore, we conduct a qualitative comparison to evaluate the impact of different types of uncertainties on the control system, highlighting areas for potential model or hardware improvements. The analysis in this study provides a roadmap for future researchers to design omniorientational drones based on their design objectives, offering practical insights into configuration selection and controller design. This research aligns with the project SAC-1, one of the objectives of Sharif AgRoLab.
传统多旋翼飞行器是欠驱动系统,这限制了它们独立控制姿态与位置的能力。在这项研究中,我们提出了一些具有额外控制输入的配置方案,这些控制输入可以调节螺旋桨轴的角度。这一设计克服了上述局限性,使系统成为“全方位定向”的。我们对所有提出的配置详细推导出了动态模型,并通过Simscape Multibody仿真方法进行了验证。此外,还设计了两个控制器:一个用于处理干扰的滑模控制器和一个基于PID的新颖控制器(包括重力补偿功能以及线性和非线性分配器),后者旨在提高计算效率。我们实施了一种自定义控制分配策略来管理这些系统的输入非仿射性质,并通过最小化本文研究中定义的“功耗因子”以延长电池寿命为目标。此外,所设计的控制器能够有效应对严苛干扰和不确定性。仿真比较并分析了提出的配置方案和控制器,主要考虑其功率消耗情况。我们还进行了一项定性比较,评估不同类型的不确定性对控制系统的影响,并指出了潜在改进模型或硬件的方向。本研究中的分析为未来的研究人员提供了一个设计全方位定向无人机的路线图,提供了关于配置选择和控制器设计的实际见解。这项研究与Sharif AgRoLab项目SAC-1的目标之一相一致。
https://arxiv.org/abs/2509.21210
This paper addresses a drone ball-balancing task, in which a drone stabilizes a ball atop a movable beam through cable-based interaction. We propose a hierarchical control framework that decouples high-level balancing policy from low-level drone control, and train a reinforcement learning (RL) policy to handle the high-level decision-making. Simulation results show that the RL policy achieves superior performance compared to carefully tuned PID controllers within the same hierarchical structure. Through systematic comparative analysis, we demonstrate that RL's advantage stems not from improved parameter tuning or inherent nonlinear mapping capabilities, but from its ability to effectively utilize richer state observations. These findings underscore the critical role of comprehensive state representation in learning-based systems and suggest that enhanced sensing could be instrumental in improving controller performance.
本文研究了一个无人机平衡球的任务,其中通过基于电缆的交互方式让无人机保持在一个可移动梁上的球体稳定。我们提出了一种分层控制框架,该框架将高层次的平衡策略与低层次的无人机控制系统解耦,并训练一种强化学习(RL)策略来处理高层次的决策制定。仿真结果显示,在相同的分层结构中,RL策略相较于精心调整过的PID控制器表现出更优性能。通过系统的比较分析,我们证明了RL的优势并非来自于改进参数调谐或固有的非线性映射能力,而是来源于其有效利用更丰富的状态观测的能力。这些发现强调了基于学习的系统中全面的状态表示的关键作用,并暗示增强传感技术可能是提升控制性能的重要手段。
https://arxiv.org/abs/2509.21122
Drone-based rapid and accurate environmental edge detection is highly advantageous for tasks such as disaster relief and autonomous navigation. Current methods, using radars or cameras, raise deployment costs and burden lightweight drones with high computational demands. In this paper, we propose AirTouch, a system that transforms the ground effect from a stability "foe" in traditional flight control views, into a "friend" for accurate and efficient edge detection. Our key insight is that analyzing drone basic attitude sensor readings and flight commands allows us to detect ground effect changes. Such changes typically indicate the drone flying over a boundary of two materials, making this information valuable for edge detection. We approach this insight through theoretical analysis, algorithm design, and implementation, fully leveraging the ground effect as a new sensing modality without compromising drone flight stability, thereby achieving accurate and efficient scene edge detection. We also compare this new sensing modality with vision-based methods to clarify its exclusive advantages in resource efficiency and detection capability. Extensive evaluations demonstrate that our system achieves a high detection accuracy with mean detection distance errors of 0.051m, outperforming the baseline method performance by 86%. With such detection performance, our system requires only 43 mW power consumption, contributing to this new sensing modality for low-cost and highly efficient edge detection.
基于无人机的快速且准确的环境边缘检测对于诸如灾害救援和自主导航等任务非常有利。目前使用雷达或相机的方法会增加部署成本,并给轻型无人机带来高计算需求负担。在本文中,我们提出了AirTouch系统,该系统将地面效应从传统飞行控制视角中的稳定性“敌人”转变为精确高效边缘检测的“朋友”。我们的关键洞察是分析无人机基本姿态传感器读数和飞行指令可以让我们检测到地面效应的变化。这种变化通常表明无人机飞越了两种材料之间的边界,因此这些信息对于边缘检测非常有价值。我们通过理论分析、算法设计和实施来实现这一见解,并充分利用地面效应作为新的感测模式,同时不损害无人机的飞行稳定性,从而实现了精确高效的场景边缘检测。 此外,我们将这种方法与基于视觉的方法进行比较,以阐明其在资源效率和检测能力方面的独特优势。广泛的评估表明,我们的系统能够实现高精度的检测,平均检测距离误差仅为0.051米,优于基准方法86%。凭借这种检测性能,我们的系统仅需消耗43毫瓦功率,为低成本且高效的边缘检测提供了新的感测模式。
https://arxiv.org/abs/2509.21085
3D object localisation based on a sequence of camera measurements is essential for safety-critical surveillance tasks, such as drone-based wildfire monitoring. Localisation of objects detected with a camera can typically be solved with dense depth estimation or 3D scene reconstruction. However, in the context of distant objects or tasks limited by the amount of available computational resources, neither solution is feasible. In this paper, we show that the task can be solved using particle filters for both single and multiple target scenarios. The method was studied using a 3D simulation and a drone-based image segmentation sequence with global navigation satellite system (GNSS)-based camera pose estimates. The results showed that a particle filter can be used to solve practical localisation tasks based on camera poses and image segments in these situations where other solutions fail. The particle filter is independent of the detection method, making it flexible for new tasks. The study also demonstrates that drone-based wildfire monitoring can be conducted using the proposed method paired with a pre-existing image segmentation model.
基于相机测量序列的3D物体定位对于安全关键性的监控任务(如无人机森林火灾监测)至关重要。通常,可以通过密集深度估计或三维场景重建来解决通过摄像头检测到的对象的定位问题。然而,在远距离对象或受可用计算资源限制的情况下,这两种方法都不适用。在本文中,我们展示了如何使用粒子滤波器来解决单个和多个目标场景下的任务。该方法是在3D模拟环境中以及结合基于全球导航卫星系统(GNSS)的摄像机姿态估计的无人机图像分割序列上进行研究的。结果表明,在其他解决方案失效的情况下,可以使用粒子过滤器根据摄像机姿势和图像片段执行实际定位任务。此外,粒子滤波器与检测方法无关,使其能够灵活应用于新任务。该研究表明,通过结合现有的图像分割模型,可以利用所提出的方法来进行无人机森林火灾监测。
https://arxiv.org/abs/2509.20906
Referring Image Segmentation (RIS), which aims to segment specific objects based on natural language descriptions, plays an essential role in vision-language understanding. Despite its progress in remote sensing applications, RIS in Low-Altitude Drone (LAD) scenarios remains underexplored. Existing datasets and methods are typically designed for high-altitude and static-view imagery. They struggle to handle the unique characteristics of LAD views, such as diverse viewpoints and high object density. To fill this gap, we present RIS-LAD, the first fine-grained RIS benchmark tailored for LAD scenarios. This dataset comprises 13,871 carefully annotated image-text-mask triplets collected from realistic drone footage, with a focus on small, cluttered, and multi-viewpoint scenes. It highlights new challenges absent in previous benchmarks, such as category drift caused by tiny objects and object drift under crowded same-class objects. To tackle these issues, we propose the Semantic-Aware Adaptive Reasoning Network (SAARN). Rather than uniformly injecting all linguistic features, SAARN decomposes and routes semantic information to different stages of the network. Specifically, the Category-Dominated Linguistic Enhancement (CDLE) aligns visual features with object categories during early encoding, while the Adaptive Reasoning Fusion Module (ARFM) dynamically selects semantic cues across scales to improve reasoning in complex scenes. The experimental evaluation reveals that RIS-LAD presents substantial challenges to state-of-the-art RIS algorithms, and also demonstrates the effectiveness of our proposed model in addressing these challenges. The dataset and code will be publicly released soon at: this https URL.
基于自然语言描述对特定对象进行分割的指代图像分割(RIS)在视觉-语言理解中扮演着重要角色。尽管其在遥感应用方面已取得进展,但在低空无人机(LAD)场景中的RIS研究仍然相对不足。现有的数据集和方法通常针对高空静态视图影像设计,难以处理LAD视角的特殊特性,如多样化的视角和高物体密度。为填补这一空白,我们推出了RIS-LAD,这是首个专为LAD场景量身定制的精细粒度RIS基准测试数据集。该数据集包含13,871组精心标注的图像-文本-掩码三元组,这些数据均来自真实的无人机视频,重点关注小尺寸、杂乱和多视角的场景。它强调了先前基准中未出现的新挑战,例如由微小物体引起的类别漂移以及在拥挤的同类物体背景下产生的对象漂移。 为解决这些问题,我们提出了语义感知自适应推理网络(SAARN)。不同于均匀注入所有语言特征,SAARN将语义信息分解并路由到网络的不同阶段。具体来说,“以类别为主导的语言增强”(CDLE)在早期编码中使视觉特征与物体类别对齐,“自适应推理融合模块”(ARFM)则跨尺度动态选择语义线索来改善复杂场景中的推理能力。 实验评估显示,RIS-LAD为最先进的RIS算法带来了重大挑战,并且证明了我们提出的模型在应对这些挑战方面的有效性。该数据集和代码将很快公开发布:[此链接](this https URL)。
https://arxiv.org/abs/2507.20920
As autonomous systems move from prototypes to real deployments, the ability of multiple agents to make decentralized, cooperative decisions becomes a core requirement. This paper examines how agentic artificial intelligence, agents that act independently, adaptively and proactively can improve task allocation and coordination in multi-agent systems, with primary emphasis on drone delivery and secondary relevance to warehouse automation. We formulate the problem in a cooperative multi-agent reinforcement learning setting and implement a lightweight multi-agent Proximal Policy Optimization, called IPPO, approach in PyTorch under a centralized-training, decentralized-execution paradigm. Experiments are conducted in PettingZoo environment, where multiple homogeneous drones or agents must self-organize to cover distinct targets without explicit communication.
随着自主系统从原型阶段发展到实际部署,多个代理能够做出去中心化、协作决策的能力成为一项核心需求。本文探讨了具有独立性、适应性和主动性的人工智能代理如何能改善多代理系统的任务分配和协调,主要关注无人机配送,同时对仓库自动化也有一定的参考价值。我们在合作型多代理强化学习环境中建立了问题模型,并在PyTorch环境下采用集中训练、分散执行的范式实现了一种轻量级的多代理近端策略优化(IPPO)方法。实验在PettingZoo环境内进行,在该环境中,多个同质无人机或代理必须自我组织以覆盖不同的目标,且无需显式的通信机制。
https://arxiv.org/abs/2510.00022
We propose a physics-informed machine-learned framework for sensor-based flow estimation for drone trajectories in complex urban terrain. The input is a rich set of flow simulations at many wind conditions. The outputs are velocity and uncertainty estimates for a target domain and subsequent sensor optimization for minimal uncertainty. The framework has three innovations compared to traditional flow estimators. First, the algorithm scales proportionally to the domain complexity, making it suitable for flows that are too complex for any monolithic reduced-order representation. Second, the framework extrapolates beyond the training data, e.g., smaller and larger wind velocities. Last, and perhaps most importantly, the sensor location is a free input, significantly extending the vast majority of the literature. The key enablers are (1) a Reynolds number-based scaling of the flow variables, (2) a physics-based domain decomposition, (3) a cluster-based flow representation for each subdomain, (4) an information entropy correlating the subdomains, and (5) a multi-variate probability function relating sensor input and targeted velocity estimates. This framework is demonstrated using drone flight paths through a three-building cluster as a simple example. We anticipate adaptations and applications for estimating complete cities and incorporating weather input.
我们提出了一种基于物理信息的机器学习框架,用于复杂城市地形中无人机轨迹的传感器流量估计。输入是多种风况下的丰富流体模拟数据集。输出则是目标区域的速度和不确定性估计以及随后为了达到最小不确定性的传感器优化方案。 与传统的流动估算器相比,该框架有三大创新点: 1. 算法的计算复杂度与领域复杂性成正比,使其适用于任何单一简化表示都无法处理的复杂流体情况。 2. 框架能够推断超出训练数据范围的信息,例如风速较小或较大的场景。 3. 传感器位置作为自由输入参数,显著扩展了现有文献的研究范围。 该框架的关键实现因素包括: 1. 基于雷诺数的流动变量缩放。 2. 物理基础的领域分解。 3. 每个子领域的基于聚类的流体表示。 4. 子域间的信息熵相关性。 5. 一个与传感器输入和目标速度估计相关的多变量概率函数。 我们通过无人机穿越由三栋建筑构成集群的飞行路径作为简单示例,演示了此框架的应用。预计该方法将被改编并应用于整个城市的流量估算,并结合天气数据进行应用。
https://arxiv.org/abs/2509.25222
The widespread deployment of autonomous systems in safety-critical environments such as urban air mobility hinges on ensuring reliable, performant, and safe operation under varying environmental conditions. One such approach, value function-based safety filters, minimally modifies a nominal controller to ensure safety. Recent advances leverage offline learned value functions to scale these safety filters to high-dimensional systems. However, these methods assume detailed priors on all possible sources of model mismatch, in the form of disturbances in the environment -- information that is rarely available in real world settings. Even in well-mapped environments like urban canyons or industrial sites, drones encounter complex, spatially-varying disturbances arising from payload-drone interaction, turbulent airflow, and other environmental factors. We introduce SPACE2TIME, which enables safe and adaptive deployment of offline-learned safety filters under unknown, spatially-varying disturbances. The key idea is to reparameterize spatial variations in disturbance as temporal variations, enabling the use of precomputed value functions during online operation. We validate SPACE2TIME on a quadcopter through extensive simulations and hardware experiments, demonstrating significant improvement over baselines.
在安全至关重要的环境中(如城市空中交通)广泛部署自主系统,需要确保这些系统能在各种环境条件下可靠、高效且安全地运行。一种基于价值函数的安全过滤方法,在不影响基本控制器性能的前提下最小化对系统的修改以保证安全性。近期进展利用离线学习的价值函数扩展了这种安全过滤器的应用范围,使其适用于高维系统。然而,这类方法假设所有可能的模型偏差来源都有详细先验知识——即环境中的干扰信息,而在现实环境中这些信息往往难以获得。 即使是在已经详尽绘制的地图环境中(如城市峡谷或工业场所),无人机也会遇到复杂的、空间变化的扰动因素,例如负载与无人机之间的相互作用、湍流气流以及其他环境因素。为了解决这个问题,我们提出了SPACE2TIME方法,它可以在未知且空间变化的干扰条件下安全地部署离线学习的安全过滤器,并实现自适应调整。该方法的核心思想是重新定义空间上变化的扰动为时间上的变化,从而在在线操作过程中能够使用预先计算的价值函数。 通过广泛的仿真和硬件实验,在四旋翼飞行器上验证了SPACE2TIME的有效性,结果显示其显著优于基准方法。
https://arxiv.org/abs/2509.19597
Object shape and pose estimation is a foundational robotics problem, supporting tasks from manipulation to scene understanding and navigation. We present a fast local solver for shape and pose estimation which requires only category-level object priors and admits an efficient certificate of global optimality. Given an RGB-D image of an object, we use a learned front-end to detect sparse, category-level semantic keypoints on the target object. We represent the target object's unknown shape using a linear active shape model and pose a maximum a posteriori optimization problem to solve for position, orientation, and shape simultaneously. Expressed in unit quaternions, this problem admits first-order optimality conditions in the form of an eigenvalue problem with eigenvector nonlinearities. Our primary contribution is to solve this problem efficiently with self-consistent field iteration, which only requires computing a 4-by-4 matrix and finding its minimum eigenvalue-vector pair at each iterate. Solving a linear system for the corresponding Lagrange multipliers gives a simple global optimality certificate. One iteration of our solver runs in about 100 microseconds, enabling fast outlier rejection. We test our method on synthetic data and a variety of real-world settings, including two public datasets and a drone tracking scenario. Code is released at this https URL.
物体形状和姿态估计是机器人技术中的一个基础问题,支持从操作到场景理解和导航等各种任务。我们提出了一种快速局部求解器用于形状和姿态的估计,该求解器只需要类别级别的对象先验,并允许有效的全局最优性证明。给定一个物体的RGB-D图像,我们使用学习得到的前端来检测目标物体质心上的稀疏、类别级别的语义关键点。我们将目标物体未知的形状表示为线性活动形状模型,并提出最大后验概率优化问题以同时求解位置、方向和形状。 用单位四元数表示该问题时,它允许采用形式化为特征值问题(具有非线性特征向量)的一阶最优性条件。我们的主要贡献是利用自洽场迭代高效地解决了这个问题,在每次迭代中只需要计算一个4×4矩阵并找到其最小的特征值-特征向量对。求解相应的拉格朗日乘数对应的线性系统可以给出简单的全局最优证明。 我们求解器的一个迭代大约需要100微秒,这使得快速检测和排除离群点成为可能。我们在合成数据集以及包括两个公共数据集在内的各种现实世界场景下测试了我们的方法,并在无人机追踪场景中进行了验证。代码已开源(链接见原文)。 这段描述介绍了一种新颖的方法来解决机器人视觉中的形状和姿态估计问题,它不仅速度快,而且能够提供全局最优性的证明。
https://arxiv.org/abs/2509.18979