Drone
-
Large Language Models to Enhance Multi-task Drone Operations in Simulated Environments
2026-01-13 10:21:17Yizhan Feng, Hichem Snoussi, Jing Teng, Abel Cherouat, Tian WangAbstract
Benefiting from the rapid advancements in large language models (LLMs), human-drone interaction has reached unprecedented opportunities. In this paper, we propose a method that integrates a fine-tuned CodeT5 model with the Unreal Engine-based AirSim drone simulator to efficiently execute multi-task operations using natural language commands. This approach enables users to interact with simulated drones through prompts or command descriptions, allowing them to easily access and control the drone's status, significantly lowering the operational threshold. In the AirSim simulator, we can flexibly construct visually realistic dynamic environments to simulate drone applications in complex scenarios. By combining a large dataset of (natural language, program code) command-execution pairs generated by ChatGPT with developer-written drone code as training data, we fine-tune the CodeT5 to achieve automated translation from natural language to executable code for drone tasks. Experimental results demonstrate that the proposed method exhibits superior task execution efficiency and command understanding capabilities in simulated environments. In the future, we plan to extend the model functionality in a modular manner, enhancing its adaptability to complex scenarios and driving the application of drone technologies in real-world environments.
Abstract (translated)
得益于大型语言模型（LLM）的快速进步，人类与无人机之间的互动达到了前所未有的机遇。在本文中，我们提出了一种方法，该方法整合了经过微调的CodeT5模型和基于Unreal Engine的AirSim无人机模拟器，以高效地通过自然语言命令执行多任务操作。这种方法使用户能够通过提示或命令描述来与仿真中的无人机交互，从而轻松访问和控制无人机的状态，显著降低了操作门槛。在AirSim模拟器中，我们可以灵活构建视觉上逼真的动态环境，用于模拟复杂场景下的无人机应用。 我们结合由ChatGPT生成的（自然语言、程序代码）命令执行对大数据集以及开发人员编写的无人机代码作为训练数据，微调CodeT5模型，以实现从自然语言到可执行代码的自动化翻译，从而完成无人机任务。实验结果表明，所提出的方法在模拟环境中表现出卓越的任务执行效率和命令理解能力。 未来，我们计划通过模块化方式扩展该模型的功能，增强其对复杂场景的适应性，并推动无人机技术在现实环境中的应用。
URL
https://arxiv.org/abs/2601.08405
PDF
-
YOLOBirDrone: Dataset for Bird vs Drone Detection and Classification and a YOLO based enhanced learning architecture
2026-01-13 08:17:28Dapinder Kaur, Neeraj Battish, Arnav Bhavsar, Shashi PoddarAbstract
The use of aerial drones for commercial and defense applications has benefited in many ways and is therefore utilized in several different application domains. However, they are also increasingly used for targeted attacks, posing a significant safety challenge and necessitating the development of drone detection systems. Vision-based drone detection systems currently have an accuracy limitation and struggle to distinguish between drones and birds, particularly when the birds are small in size. This research work proposes a novel YOLOBirDrone architecture that improves the detection and classification accuracy of birds and drones. YOLOBirDrone has different components, including an adaptive and extended layer aggregation (AELAN), a multi-scale progressive dual attention module (MPDA), and a reverse MPDA (RMPDA) to preserve shape information and enrich features with local and global spatial and channel information. A large-scale dataset, BirDrone, is also introduced in this article, which includes small and challenging objects for robust aerial object identification. Experimental results demonstrate an improvement in performance metrics through the proposed YOLOBirDrone architecture compared to other state-of-the-art algorithms, with detection accuracy reaching approximately 85% across various scenarios.
Abstract (translated)
用于商业和国防应用的无人机在许多方面带来了益处，因此被应用于多个不同的领域。然而，它们也越来越多地被用作有针对性的攻击手段，这提出了重大的安全挑战，并且促使开发了无人机检测系统。基于视觉的无人机检测系统目前存在准确性限制，并且难以区分鸟类与无人机，尤其是在鸟类体型较小的情况下。 这项研究工作提出了一种新的YOLOBirDrone架构，旨在提高对鸟类和无人机的检测及分类准确度。该架构包含多个组件，包括自适应扩展层聚合（AELAN）、多尺度渐进双注意力模块（MPDA）以及逆向MPDA（RMPDA），后者用于保留形状信息并利用局部和全局空间及通道信息来丰富特征。 文章还引入了一个大规模的数据集BirDrone，其中包含小型且具挑战性的物体，有助于实现稳健的空中目标识别。实验结果显示，与现有的先进算法相比，通过提出的YOLOBirDrone架构，在性能指标上有所提升，检测准确率在各种场景下达到了约85%。
URL
https://arxiv.org/abs/2601.08319
PDF
-
How vehicles change lanes after encountering crashes: Empirical analysis and modeling
2026-01-13 01:37:54Kequan Chen, Yuxuan Wang, Pan Liu, Victor L. Knoop, David Z. W. Wang, Yu HanAbstract
When a traffic crash occurs, following vehicles need to change lanes to bypass the obstruction. We define these maneuvers as post crash lane changes. In such scenarios, vehicles in the target lane may refuse to yield even after the lane change has already begun, increasing the complexity and crash risk of post crash LCs. However, the behavioral characteristics and motion patterns of post crash LCs remain unknown. To address this gap, we construct a post crash LC dataset by extracting vehicle trajectories from drone videos captured after crashes. Our empirical analysis reveals that, compared to mandatory LCs (MLCs) and discretionary LCs (DLCs), post crash LCs exhibit longer durations, lower insertion speeds, and higher crash risks. Notably, 79.4% of post crash LCs involve at least one instance of non yielding behavior from the new follower, compared to 21.7% for DLCs and 28.6% for MLCs. Building on these findings, we develop a novel trajectory prediction framework for post crash LCs. At its core is a graph based attention module that explicitly models yielding behavior as an auxiliary interaction aware task. This module is designed to guide both a conditional variational autoencoder and a Transformer based decoder to predict the lane changer's trajectory. By incorporating the interaction aware module, our model outperforms existing baselines in trajectory prediction performance by more than 10% in both average displacement error and final displacement error across different prediction horizons. Moreover, our model provides more reliable crash risk analysis by reducing false crash rates and improving conflict prediction accuracy. Finally, we validate the model's transferability using additional post crash LC datasets collected from different sites.
Abstract (translated)
当交通碰撞发生时，后续车辆需要改变车道以避开障碍物。我们将这些操作定义为事故后的换道行为（post-crash lane changes）。在这种情况下，目标车道中的车辆可能即使在换道已经开始之后仍然拒绝让行，增加了事故后换道的复杂性和风险性。然而，关于这种换道的行为特征和运动模式尚不为人知。 为了填补这一空白，我们构建了一个事故后的换道数据集，通过从碰撞事件发生后捕获的无人机视频中提取车辆轨迹来创建该数据集。我们的实证分析表明，与强制换道（MLCs）和随意换道（DLCs）相比，事故后的换道行为持续时间更长、插入速度较低且风险更高。值得注意的是，79.4%的事故后换道涉及至少一次来自新跟随者的不让行行为，相比之下，DLCS为21.7%，MLCs为28.6%。 基于这些发现，我们开发了一个针对事故后的换道行为的新轨迹预测框架。该框架的核心是一个以图为基础的关注模块（graph-based attention module），它明确地将让行行为建模为一个具有互动感知能力的辅助任务。这个模块旨在指导条件变分自编码器和基于Transformer的解码器来预测车道变更者的轨迹。 通过引入具有互动感知能力的模块，我们的模型在不同预测时间范围内的平均位移误差（ADE）和最终位移误差（FDE）方面均超过了现有基线模型10%以上。此外，我们的模型还提供了更可靠的风险分析功能，减少了误报事故率并提高了冲突预测准确性。 最后，我们通过使用来自不同地点收集的其他事故后换道数据集来验证该模型的可转移性。
URL
https://arxiv.org/abs/2601.08125
PDF
-
{mu}DopplerTag: CNN-Based Drone Recognition via Cooperative Micro-Doppler Tagging
2026-01-12 22:23:57O. Yerushalimov, D. Vovchuk, A. Glam, P. GinzburgAbstract
The rapid deployment of drones poses significant challenges for airspace management, security, and surveillance. Current detection and classification technologies, including cameras, LiDAR, and conventional radar systems, often struggle to reliably identify and differentiate drones, especially those of similar models, under diverse environmental conditions and at extended ranges. Moreover, low radar cross sections and clutter further complicate accurate drone identification. To address these limitations, we propose a novel drone classification method based on artificial micro-Doppler signatures encoded by resonant electromagnetic stickers attached to drone blades. These tags generate distinctive, configuration-specific radar returns, enabling robust identification. We develop a tailored convolutional neural network (CNN) capable of processing raw radar signals, achieving high classification accuracy. Extensive experiments were conducted both in anechoic chambers with 43 tag configurations and outdoors under realistic flight trajectories and noise conditions. Dimensionality reduction techniques, including Principal Component Analysis (PCA) and Uniform Manifold Approximation and Projection (UMAP), provided insight into code separability and robustness. Our results demonstrate reliable drone classification performance at signal-to-noise ratios as low as 7 dB, indicating the feasibility of long-range detection with advanced surveillance radar systems. Preliminary range estimations indicate potential operational distances of several kilometers, suitable for critical applications such as airport airspace monitoring. The integration of electromagnetic tagging with machine learning enables scalable and efficient drone identification, paving the way for enhanced aerial traffic management and security in increasingly congested airspaces.
Abstract (translated)
无人机的快速部署给空中交通管理和安全监控带来了重大挑战。当前的检测和分类技术，包括摄像头、LiDAR 和传统雷达系统，在各种环境条件下尤其是在远距离上可靠地识别和区分相似型号的无人机方面往往遇到困难。此外，低雷达截面以及杂波进一步复杂化了准确的无人机识别问题。为了解决这些限制，我们提出了一种基于人工微多普勒特征的新颖无人机分类方法，这种特征是由贴附在无人机旋翼上的共振电磁标签编码产生的。这些标签能够产生具有独特配置特性的雷达回波信号，从而实现稳健的识别能力。 为此，我们开发了一个专门用于处理原始雷达信号的卷积神经网络（CNN），该网络实现了高准确度分类效果。我们在消声室中进行了广泛的实验，使用了43种不同的标签配置，并在室外环境中通过实际飞行轨迹和噪声条件下进一步测试了系统的性能。降维技术如主成分分析（PCA）和均匀流形逼近及投影（UMAP）为我们提供了代码分离性和鲁棒性方面的见解。 我们的研究结果表明，在信噪比低至7dB的情况下，该系统仍能可靠地进行无人机分类，这预示着使用先进的监控雷达系统在长距离范围内检测无人机是可行的。初步的距离估计显示可能的操作范围为数公里，这对于机场空域监测等关键应用而言非常合适。 通过将电磁标记技术与机器学习相结合，这种方案能够实现大规模且高效的无人机识别，从而为进一步优化空中交通管理和增强安全措施铺平道路，在日益拥挤的领空环境下尤为重要。
URL
https://arxiv.org/abs/2601.08042
PDF
-
LWMSCNN-SE: A Lightweight Multi-Scale Network for Efficient Maize Disease Classification on Edge Devices
2026-01-12 19:45:10Fikadu Weloday, Jianmei SuAbstract
Maize disease classification plays a vital role in mitigating yield losses and ensuring food security. However, the deployment of traditional disease detection models in resource-constrained environments, such as those using smartphones and drones, faces challenges due to high computational costs. To address these challenges, we propose LWMSCNN-SE, a lightweight convolutional neural network (CNN) that integrates multi-scale feature extraction, depthwise separable convolutions, and squeeze-and-Excitation (SE) attention mechanisms. This novel combination enables the model to achieve 96.63% classification accuracy with only 241,348 parameters and 0.666 GFLOPs, making it suitable for real-time deployment in field applications. Our approach addresses the accuracy--efficiency trade-off by delivering high accuracy while maintaining low computational costs, demonstrating its potential for efficient maize disease diagnosis on edge devices in precision farming systems.
Abstract (translated)
玉米病害分类在减少产量损失和确保粮食安全方面发挥着重要作用。然而，在资源受限的环境中部署传统的疾病检测模型（如使用智能手机和无人机）面临着高昂计算成本的挑战。为了解决这些难题，我们提出了一种轻量级卷积神经网络(LWMSCNN-SE)，该模型结合了多尺度特征提取、深度可分离卷积以及挤压与激励(SE)注意力机制。这种新型组合使模型能够在仅使用241,348个参数和0.666 GFLOPS的情况下达到96.63%的分类准确率，使其适合在实际田间应用中实时部署。我们的方法通过提供高精度的同时保持低计算成本的方式解决了精度与效率之间的权衡问题，展示了其在精准农业系统中的边缘设备上进行有效玉米病害诊断的应用潜力。
URL
https://arxiv.org/abs/2601.07957
PDF
-
FlyCo: Foundation Model-Empowered Drones for Autonomous 3D Structure Scanning in Open-World Environments
2026-01-12 14:14:39Chen Feng, Guiyong Zheng, Tengkai Zhuang, Yongqian Wu, Fangzhan He, Haojia Li, Juepeng Zheng, Shaojie Shen, Boyu ZhouAbstract
Autonomous 3D scanning of open-world target structures via drones remains challenging despite broad applications. Existing paradigms rely on restrictive assumptions or effortful human priors, limiting practicality, efficiency, and adaptability. Recent foundation models (FMs) offer great potential to bridge this gap. This paper investigates a critical research problem: What system architecture can effectively integrate FM knowledge for this task? We answer it with FlyCo, a principled FM-empowered perception-prediction-planning loop enabling fully autonomous, prompt-driven 3D target scanning in diverse unknown open-world environments. FlyCo directly translates low-effort human prompts (text, visual annotations) into precise adaptive scanning flights via three coordinated stages: (1) perception fuses streaming sensor data with vision-language FMs for robust target grounding and tracking; (2) prediction distills FM knowledge and combines multi-modal cues to infer the partially observed target's complete geometry; (3) planning leverages predictive foresight to generate efficient and safe paths with comprehensive target coverage. Building on this, we further design key components to boost open-world target grounding efficiency and robustness, enhance prediction quality in terms of shape accuracy, zero-shot generalization, and temporal stability, and balance long-horizon flight efficiency with real-time computability and online collision avoidance. Extensive challenging real-world and simulation experiments show FlyCo delivers precise scene understanding, high efficiency, and real-time safety, outperforming existing paradigms with lower human effort and verifying the proposed architecture's practicality. Comprehensive ablations validate each component's contribution. FlyCo also serves as a flexible, extensible blueprint, readily leveraging future FM and robotics advances. Code will be released.
Abstract (translated)
自主无人机三维扫描开放环境中的目标结构仍然面临诸多挑战，尽管其应用范围广泛。现有的方法依赖于严格的假设或需要大量的先验人工干预，这限制了其实用性、效率和适应能力。最近的基础模型（Foundation Models, FMs）为解决这些问题提供了巨大的潜力。本文探讨了一个关键的研究问题：何种系统架构能够有效地将基础模型知识整合到该任务中？我们通过提出FlyCo来回答这个问题——这是一个基于原则的自主感知-预测-规划循环框架，它利用了强大的基础模型能力，能够在多样且未知的开放环境中进行完全自主、指令驱动的目标三维扫描。 FlyCo直接将低投入的人类指令（如文本和视觉注释）转化为精确而适应性强的无人机飞行任务，通过三个协调阶段实现这一目标： 1. **感知**：该阶段融合了实时传感器数据与视觉-语言基础模型的知识，以确保对目标对象的稳定识别和跟踪。 2. **预测**：此步骤提取并利用基础模型知识，并结合多模态线索来推断部分可见目标的整体几何结构。 3. **规划**：这一环节利用预测性的洞察力生成高效且安全的飞行路径，确保全面覆盖目标物体。 基于上述架构，我们进一步设计了关键组件以提升开放环境下的目标定位效率和鲁棒性，并增强预测质量，特别是在形状准确性、零样本泛化能力和时间稳定性方面。此外，还优化了长时段任务的执行效率与实时计算能力以及在线避障之间的平衡。一系列广泛的现实世界和仿真实验显示，FlyCo在场景理解精度、运行效率及即时安全性等方面超越现有方法，并且需要较低的人工干预量，验证了所提出架构的实际可行性。 详细消融研究进一步证实了每个组件的贡献。此外，FlyCo还作为一个灵活、可扩展的设计蓝图存在，能够轻松利用未来的基础模型和机器人技术进步。代码将在适当时候发布。
URL
https://arxiv.org/abs/2601.07558
PDF
-
NanoCockpit: Performance-optimized Application Framework for AI-based Autonomous Nanorobotics
2026-01-12 12:29:38Elia Cereda, Alessandro Giusti, Daniele PalossiAbstract
Autonomous nano-drones, powered by vision-based tiny machine learning (TinyML) models, are a novel technology gaining momentum thanks to their broad applicability and pushing scientific advancement on resource-limited embedded systems. Their small form factor, i.e., a few 10s grams, severely limits their onboard computational resources to sub-\SI{100}{\milli\watt} microcontroller units (MCUs). The Bitcraze Crazyflie nano-drone is the \textit{de facto} standard, offering a rich set of programmable MCUs for low-level control, multi-core processing, and radio transmission. However, roboticists very often underutilize these onboard precious resources due to the absence of a simple yet efficient software layer capable of time-optimal pipelining of multi-buffer image acquisition, multi-core computation, intra-MCUs data exchange, and Wi-Fi streaming, leading to sub-optimal control performances. Our \textit{NanoCockpit} framework aims to fill this gap, increasing the throughput and minimizing the system's latency, while simplifying the developer experience through coroutine-based multi-tasking. In-field experiments on three real-world TinyML nanorobotics applications show our framework achieves ideal end-to-end latency, i.e. zero overhead due to serialized tasks, delivering quantifiable improvements in closed-loop control performance ($-$30\% mean position error, mission success rate increased from 40\% to 100\%).
Abstract (translated)
自主纳米无人机，搭载基于视觉的微型机器学习（TinyML）模型，因其广泛的应用性和在资源受限嵌入式系统中推动科学进步的能力而迅速发展。这些无人机小巧的体积（几十分之一克）严重限制了其机载计算资源，仅能使用低于100毫瓦微控制器单元（MCUs）。Bitcraze Crazyflie纳米无人机成为事实上的行业标准，提供了丰富的低级控制、多核处理和无线电传输编程MCU。然而，由于缺乏一个简单而高效的软件层来实现时间最优的多缓冲图像采集、多核计算、MCU间数据交换以及Wi-Fi流媒体的时间优化流水线，机器人工程师往往未能充分利用这些宝贵的机载资源，从而导致了次优的控制性能。 我们的NanoCockpit框架旨在填补这一空白，通过协程式的多任务处理简化开发人员体验，同时提高系统吞吐量并最小化延迟。实地实验表明，在三个真实的TinyML纳米机器人应用中，我们的框架能够实现理想的端到端时延（即由于串行任务导致的零额外开销），并在闭环控制性能方面带来了可量化改进：平均位置误差减少了30%，使命成功率从40%提高到了100%。
URL
https://arxiv.org/abs/2601.07476
PDF
-
RSLCPP - Deterministic Simulations Using ROS 2
2026-01-11 20:10:54Simon Sagmeister, Marcel Weinmann, Phillip Pitschi, Markus LienkampAbstract
Simulation is crucial in real-world robotics, offering safe, scalable, and efficient environments for developing applications, ranging from humanoid robots to autonomous vehicles and drones. While the Robot Operating System (ROS) has been widely adopted as the backbone of these robotic applications in both academia and industry, its asynchronous, multiprocess design complicates reproducibility, especially across varying hardware platforms. Deterministic callback execution cannot be guaranteed when computation times and communication delays vary. This lack of reproducibility complicates scientific benchmarking and continuous integration, where consistent results are essential. To address this, we present a methodology to create deterministic simulations using ROS 2 nodes. Our ROS Simulation Library for C++ (RSLCPP) implements this approach, enabling existing nodes to be combined into a simulation routine that yields reproducible results without requiring any code changes. We demonstrate that our approach yields identical results across various CPUs and architectures when testing both a synthetic benchmark and a real-world robotics system. RSLCPP is open-sourced at this https URL.
Abstract (translated)
仿真在现实世界的机器人技术中至关重要，为从人形机器人到自主车辆和无人机的各种应用提供了安全、可扩展且高效的开发环境。虽然机器人操作系统（ROS）已被广泛采用作为学术界和工业界这些机器人应用程序的支柱，但其异步多进程设计使得跨不同硬件平台时难以保证实验结果的一致性。由于计算时间和通信延迟的变化会导致回调执行不可预测，这尤其增加了复杂性。这种一致性问题也给科学基准测试和持续集成带来了挑战，在这些场景中，一致的结果至关重要。 为了解决这一问题，我们提出了一种利用ROS 2节点创建确定性仿真的方法学。我们的C++ ROS仿真库（RSLCPP）实现了这种方法，并使现有的节点可以被组合成一个生成可重复结果的仿真流程，而无需进行任何代码更改。我们展示了当测试合成基准和实际机器人系统时，该方法能在各种CPU和架构上产生一致的结果。 RSLCPP在以下链接开源：[此URL]（请将"[此URL]"替换为实际网址）。
URL
https://arxiv.org/abs/2601.07052
PDF
-
Robust Evacuation for Multi-Drone Failure in Drone Light Shows
2026-01-11 00:36:47Minhyuk Park, Aloysius K. Mok, Tsz-Chiu AuAbstract
Drone light shows have emerged as a popular form of entertainment in recent years. However, several high-profile incidents involving large-scale drone failures -- where multiple drones simultaneously fall from the sky -- have raised safety and reliability concerns. To ensure robustness, we propose a drone parking algorithm designed specifically for multiple drone failures in drone light shows, aimed at mitigating the risk of cascading collisions by drone evacuation and enabling rapid recovery from failures by leveraging strategically placed hidden drones. Our algorithm integrates a Social LSTM model with attention mechanisms to predict the trajectories of failing drones and compute near-optimal evacuation paths that minimize the likelihood of surviving drones being hit by fallen drones. In the recovery node, our system deploys hidden drones (operating with their LED lights turned off) to replace failed drones so that the drone light show can continue. Our experiments showed that our approach can greatly increase the robustness of a multi-drone system by leveraging deep learning to predict the trajectories of fallen drones.
Abstract (translated)
近年来，无人机灯光秀作为一种流行的娱乐形式越来越受欢迎。然而，一些大型的无人机故障事件——即多架无人机同时从空中坠落——引发了安全性和可靠性的担忧。为了确保系统的稳健性，我们提出了一种专为多机故障设计的无人机停放算法，在大规模无人机表演中通过疏散策略来降低连环碰撞的风险，并利用隐藏在特定位置的备用无人机实现快速恢复。我们的算法结合了具有注意力机制的社会LSTM模型来预测坠落无人机的轨迹，并计算出最小化幸存无人机被坠落无人机撞击概率的近似最优疏散路径。在恢复阶段，系统部署备用无人机（关闭LED灯）以替换故障无人机，从而使表演得以继续进行。 实验表明，通过深度学习技术预测坠落无人机的轨迹，我们的方法可以大大提高多机系统的稳健性。
URL
https://arxiv.org/abs/2601.06728
PDF
-
Precision Meets Art: Autonomous Multi-UAV System for Large Scale Mural Drawing
2026-01-10 10:00:23Andrei A. Korigodskii, Artem E. Vasiunik, Georgii A. Varin, Adilia M. Zukhurova, Matvei V. Urvantsev, Semen A. Osipenkov, Igor S. Efremov, Georgii E. BondarAbstract
The integration of autonomous unmanned aerial vehicles (UAVs) into large-scale artistic projects has emerged as a new application in robotics. This paper presents the design, deployment, and testing of a novel multi-drone system for automated mural painting in outdoor settings. This technology makes use of new software that coordinates multiple drones simultaneously, utilizing state-machine algorithms for task execution. Key advancements are the complex positioning system that combines 2D localization using a single motion tracking camera with onboard LiDAR for precise positioning, and a novel flight control algorithm, which works differently along the trajectory and normally to it, ensuring smoothness and high precision of the drawings at the same time. A 100 square meters mural was created using the developed multi-drone system, validating the system's efficacy. Compared to single-drone approaches, our multi-UAV solution significantly improves scalability and operational speed while maintaining high stability even in harsh weather conditions. The findings highlight the potential of autonomous robotic swarms in creative applications, paving the way for further advancements in large-scale robotic art.
Abstract (translated)
将自主无人飞行器（UAV）集成到大型艺术项目中，已成为机器人技术的一个新应用领域。本文介绍了设计、部署和测试一种新型多无人机系统以在户外环境下进行自动化壁画绘制的过程。该技术利用新的软件来同时协调多个无人机的操作，并采用状态机算法执行任务。 关键技术进步包括一个复杂的定位系统，结合使用单个运动跟踪相机进行二维定位与机载LiDAR相结合的精确位置确定方法，以及一种全新的飞行控制算法，这种算法在轨迹上和正常操作中工作方式不同，确保绘制图形的同时具备流畅性和高精度。利用开发的多无人机系统成功创作了一幅100平方米大小的壁画，验证了该系统的有效性。 与单个无人机的方法相比，我们的多UAV解决方案显著提高了可扩展性以及运行速度，并且即使在恶劣天气条件下也能保持高度稳定性。研究结果突显了自主机器人集群在创意应用中的潜力，为大规模机器人艺术的发展铺平道路。
URL
https://arxiv.org/abs/2601.06508
PDF
-
Pyramidal Adaptive Cross-Gating for Multimodal Detection
2026-01-09 14:50:08Zidong Gu, Shoufu TianAbstract
Object detection in aerial imagery is a critical task in applications such as UAV reconnaissance. Although existing methods have extensively explored feature interaction between different modalities, they commonly rely on simple fusion strategies for feature aggregation. This introduces two critical flaws: it is prone to cross-modal noise and disrupts the hierarchical structure of the feature pyramid, thereby impairing the fine-grained detection of small objects. To address this challenge, we propose the Pyramidal Adaptive Cross-Gating Network (PACGNet), an architecture designed to perform deep fusion within the backbone. To this end, we design two core components: the Symmetrical Cross-Gating (SCG) module and the Pyramidal Feature-aware Multimodal Gating (PFMG) module. The SCG module employs a bidirectional, symmetrical "horizontal" gating mechanism to selectively absorb complementary information, suppress noise, and preserve the semantic integrity of each modality. The PFMG module reconstructs the feature hierarchy via a progressive hierarchical gating mechanism. This leverages the detailed features from a preceding, higher-resolution level to guide the fusion at the current, lower-resolution level, effectively preserving fine-grained details as features propagate. Through evaluations conducted on the DroneVehicle and VEDAI datasets, our PACGNet sets a new state-of-the-art benchmark, with mAP50 scores reaching 82.2% and 82.1% respectively.
Abstract (translated)
在航空影像中的目标检测是无人机侦察等应用的关键任务。尽管现有方法已经在不同模态之间的特征交互上进行了深入探索，但它们通常依赖于简单的融合策略来进行特征聚合。这引入了两个关键缺陷：容易受到跨模态噪声的影响，并且破坏了特征金字塔的层次结构，从而影响小对象的精细检测能力。 为了解决这一挑战，我们提出了一个名为“金字塔自适应交叉门控网络”（PACGNet）的新架构，该架构旨在骨干网内执行深度融合。为此，我们设计了两个核心组件：对称交叉门控（SCG）模块和金字塔特征感知多模态门控（PFMG）模块。 SCG模块采用双向、对称的“水平”门控机制，以有选择地吸收互补信息，抑制噪声，并保持每种模式的语义完整性。而PFMG模块通过逐步分层的门控机制重构了特征层次结构。它利用前一级高分辨率级别的详细特征来指导当前低分辨率级别的融合过程，从而有效地保留了在特征传递过程中精细级别的细节。 在DroneVehicle和VEDAI数据集上的评估表明，我们的PACGNet设立了一个新的行业标准，在mAP50得分方面分别达到了82.2%和82.1%，显著提升了目标检测的精度。
URL
https://arxiv.org/abs/2512.18291
PDF
-
FlyPose: Towards Robust Human Pose Estimation From Aerial Views
2026-01-09 12:01:36Hassaan Farooq, Marvin Brenner, Peter St\\"utzAbstract
Unmanned Aerial Vehicles (UAVs) are increasingly deployed in close proximity to humans for applications such as parcel delivery, traffic monitoring, disaster response and infrastructure inspections. Ensuring safe and reliable operation in these human-populated environments demands accurate perception of human poses and actions from an aerial viewpoint. This perspective challenges existing methods with low resolution, steep viewing angles and (self-)occlusion, especially if the application demands realtime feasibile models. We train and deploy FlyPose, a lightweight top-down human pose estimation pipeline for aerial imagery. Through multi-dataset training, we achieve an average improvement of 6.8 mAP in person detection across the test-sets of Manipal-UAV, VisDrone, HIT-UAV as well as our custom dataset. For 2D human pose estimation we report an improvement of 16.3 mAP on the challenging UAV-Human dataset. FlyPose runs with an inference latency of ~20 milliseconds including preprocessing on a Jetson Orin AGX Developer Kit and is deployed onboard a quadrotor UAV during flight experiments. We also publish FlyPose-104, a small but challenging aerial human pose estimation dataset, that includes manual annotations from difficult aerial perspectives: this https URL.
Abstract (translated)
无人驾驶飞行器（UAVs）在诸如包裹递送、交通监控、灾害应对和基础设施检查等人周围环境中越来越常见。为了确保这些有人类居住环境中的安全可靠运行，需要从空中视角准确感知人体姿势和动作。这一角度对现有方法提出了挑战，包括低分辨率、陡峭的观察角度以及（自我）遮挡问题，尤其是在实时模型的需求下。我们训练并部署了FlyPose，这是一种针对航拍图像的人体姿态估计轻量级顶视图管道。通过多数据集训练，我们在Manipal-UAV、VisDrone、HIT-UAV和我们自定义的数据集中实现了平均6.8 mAP（平均精度）的人物检测改进。对于2D人体姿态估计，在具有挑战性的UAV-Human数据集上，FlyPose显示出了16.3 mAP的提升。 FlyPose在Jetson Orin AGX开发套件上的推理延迟约为20毫秒，包括预处理步骤，并且已部署到四旋翼无人机进行飞行实验。此外，我们发布了FlyPose-104，这是一个小但具有挑战性的航拍人体姿态估计数据集，其中包含了从困难视角的手动注释：[链接](this https URL)。 （注意：原文中的"this https URL"似乎是一个错误的格式，应该替换为实际的数据集发布链接。）
URL
https://arxiv.org/abs/2601.05747
PDF
-
Detector-Augmented SAMURAI for Long-Duration Drone Tracking
2026-01-08 10:27:05Tamara R. Lenhard, Andreas Weinmann, Hichem Snoussi, Tobias KochAbstract
Robust long-term tracking of drone is a critical requirement for modern surveillance systems, given their increasing threat potential. While detector-based approaches typically achieve strong frame-level accuracy, they often suffer from temporal inconsistencies caused by frequent detection dropouts. Despite its practical relevance, research on RGB-based drone tracking is still limited and largely reliant on conventional motion models. Meanwhile, foundation models like SAMURAI have established their effectiveness across other domains, exhibiting strong category-agnostic tracking performance. However, their applicability in drone-specific scenarios has not been investigated yet. Motivated by this gap, we present the first systematic evaluation of SAMURAI's potential for robust drone tracking in urban surveillance settings. Furthermore, we introduce a detector-augmented extension of SAMURAI to mitigate sensitivity to bounding-box initialization and sequence length. Our findings demonstrate that the proposed extension significantly improves robustness in complex urban environments, with pronounced benefits in long-duration sequences - especially under drone exit-re-entry events. The incorporation of detector cues yields consistent gains over SAMURAI's zero-shot performance across datasets and metrics, with success rate improvements of up to +0.393 and FNR reductions of up to -0.475.
Abstract (translated)
在现代监控系统中，无人机的长期稳健追踪是一项关键要求，鉴于其潜在威胁日益增加。虽然基于检测器的方法通常能够在帧级精度上表现出色，但它们往往因频繁出现的目标丢失而产生时间不一致性的问题。尽管这一研究具有实际意义，但在基于RGB图像的无人机跟踪领域，相关研究仍然有限，并且主要依赖传统的运动模型。与此同时，诸如SAMURAI这样的基础模型在其他领域的应用中已经证明了其有效性，表现出强大的类别无关追踪性能。然而，这些模型在特定于无人机场景的应用上尚未进行过探索。 鉴于这一空白，我们首次系统性地评估了SAMURAI在城市监控环境中实现稳健无人机跟踪的潜力。此外，为了缓解对边界框初始化和序列长度敏感的问题，我们还引入了SAMURAI的一个增强版本，并结合检测器的功能。我们的研究发现表明，所提出的扩展显著提高了复杂城市环境下的鲁棒性，尤其是在长时序追踪中表现尤为突出——特别是在无人机离开再返回的事件中。通过融合检测器提供的线索，在所有数据集和评估指标上，与SAMURAI的零样本性能相比，成功率提升了高达+0.393，假阴率减少了最多-0.475。
URL
https://arxiv.org/abs/2601.04798
PDF
-
Physics-Constrained Cross-Resolution Enhancement Network for Optics-Guided Thermal UAV Image Super-Resolution
2026-01-07 02:30:27Zhicheng Zhao, Fengjiao Peng, Jinquan Yan, Wei Lu, Chenglong Li, Jin TangAbstract
Optics-guided thermal UAV image super-resolution has attracted significant research interest due to its potential in all-weather monitoring applications. However, existing methods typically compress optical features to match thermal feature dimensions for cross-modal alignment and fusion, which not only causes the loss of high-frequency information that is beneficial for thermal super-resolution, but also introduces physically inconsistent artifacts such as texture distortions and edge blurring by overlooking differences in the imaging physics between modalities. To address these challenges, we propose PCNet to achieve cross-resolution mutual enhancement between optical and thermal modalities, while physically constraining the optical guidance process via thermal conduction to enable robust thermal UAV image super-resolution. In particular, we design a Cross-Resolution Mutual Enhancement Module (CRME) to jointly optimize thermal image super-resolution and optical-to-thermal modality conversion, facilitating effective bidirectional feature interaction across resolutions while preserving high-frequency optical priors. Moreover, we propose a Physics-Driven Thermal Conduction Module (PDTM) that incorporates two-dimensional heat conduction into optical guidance, modeling spatially-varying heat conduction properties to prevent inconsistent artifacts. In addition, we introduce a temperature consistency loss that enforces regional distribution consistency and boundary gradient smoothness to ensure generated thermal images align with real-world thermal radiation principles. Extensive experiments on VGTSR2.0 and DroneVehicle datasets demonstrate that PCNet significantly outperforms state-of-the-art methods on both reconstruction quality and downstream tasks including semantic segmentation and object detection.
Abstract (translated)
基于光学引导的热气球无人机图像超分辨率技术因其在全天候监测应用中的潜力而引起了广泛的研究兴趣。然而，现有的方法通常通过压缩光学特征以匹配热特征维度来进行跨模态对齐和融合，这不仅导致了有益于热超分辨率的高频信息丢失，而且还因忽视了不同成像物理之间的差异而导致诸如纹理扭曲和边缘模糊等物理不一致的现象。 为了解决这些挑战，我们提出了PCNet（Physics-Driven Cross-modal Network），它在光学与热模态之间实现了跨分辨率相互增强，并通过基于热传导的物理约束来强化光学引导过程，从而能够实现鲁棒性的热气球无人机图像超分辨率。具体来说，我们设计了跨分辨率互增强模块（CRME），以共同优化热图的超分辨率和从光学到热的模态转换，促进不同分辨率间的有效双向特征交互，并保持高频光学先验信息。 此外，我们还提出了物理驱动的热传导模块（PDTM），该模块将二维热传导融入光学引导过程中，通过建模空间变化的热传导特性来防止不一致的现象。同时，我们引入了温度一致性损失，确保生成的热图在区域分布和边界梯度平滑性方面与现实世界的热辐射原理保持一致。 我们在VGTSR2.0和DroneVehicle数据集上的广泛实验表明，PCNet在重建质量和包括语义分割和目标检测在内的下游任务上都显著优于现有最先进的方法。
URL
https://arxiv.org/abs/2601.03526
PDF
-
Cost-Effective Radar Sensors for Field-Based Water Level Monitoring with Sub-Centimeter Accuracy
2026-01-06 22:25:19Anna Zavei-Boroda (Department of Computer Science, University of Colorado Boulder, USA), J. Toby Minear (Cooperative Institute for Research in Environmental Science, University of Colorado Boulder, USA), Kyle Harlow (Department of Computer Science, University of Colorado Boulder, USA), Dusty Woods (Department of Computer Science, University of Colorado Boulder, USA), Christoffer Heckman (Department of Computer Science, University of Colorado Boulder, USA)Abstract
Water level monitoring is critical for flood management, water resource allocation, and ecological assessment, yet traditional methods remain costly and limited in coverage. This work explores radar-based sensing as a low-cost alternative for water level estimation, leveraging its non-contact nature and robustness to environmental conditions. Commercial radar sensors are evaluated in real-world field tests, applying statistical filtering techniques to improve accuracy. Results show that a single radar sensor can achieve centimeter-scale precision with minimal calibration, making it a practical solution for autonomous water monitoring using drones and robotic platforms.
Abstract (translated)
水位监测对于洪水管理、水资源分配和生态评估至关重要，然而传统的监测方法仍然成本高昂且覆盖范围有限。本研究探索了基于雷达的传感技术作为低成本替代方案来估算水位，利用其非接触特性和对环境条件的强大适应能力。通过在实际现场测试中应用统计滤波技术来提高精度，对商用雷达传感器进行了评估。结果表明，单个雷达传感器可以实现厘米级精度，并且只需少量校准即可使用，使其成为无人机和机器人平台自主水位监测的实用解决方案。
URL
https://arxiv.org/abs/2601.03447
PDF
-
CageDroneRF: A Large-Scale RF Benchmark and Toolkit for Drone Perception
2026-01-06 03:39:59Mohammad Rostami, Atik Faysal, Hongtao Xia, Hadi Kasasbeh, Ziang Gao, Huaxia WangAbstract
We present CageDroneRF (CDRF), a large-scale benchmark for Radio-Frequency (RF) drone detection and identification built from real-world captures and systematically generated synthetic variants. CDRF addresses the scarcity and limited diversity of existing RF datasets by coupling extensive raw recordings with a principled augmentation pipeline that (i) precisely controls Signal-to-Noise Ratio (SNR), (ii) injects interfering emitters, and (iii) applies frequency shifts with label-consistent bounding-box transformations for detection. This dataset spans a wide range of contemporary drone models, many unavailable in current public datasets, and acquisition conditions, derived from data collected at the Rowan University campus and within a controlled RF-cage facility. CDRF is released with interoperable open-source tools for data generation, preprocessing, augmentation, and evaluation that also operate on existing public benchmarks. CDRF enables standardized benchmarking for classification, open-set recognition, and object detection, supporting rigorous comparisons and reproducible pipelines. By releasing this comprehensive benchmark and tooling, CDRF aims to accelerate progress toward robust, generalizable RF perception models.
Abstract (translated)
我们介绍了CageDroneRF (CDRF)，这是一个大规模的基准测试，用于基于真实世界捕捉和系统生成的合成变体构建的无线电频率（RF）无人机检测与识别。CDRF通过结合广泛的原始记录和一套有原则的数据增强流程来解决现有RF数据集稀缺性和多样性不足的问题，该流程： - 精确控制信噪比（SNR） - 注入干扰发射器 - 对于检测应用频移，并进行标签一致的边界框转换 这个数据集涵盖了多种当代无人机模型，其中许多在当前公开的数据集中不可用，以及各种获取条件，这些数据来源于罗文大学校园和受控RF笼设施收集的信息。CDRF与开源工具一起发布，用于数据生成、预处理、增强及评估，并且还能适用于现有的公共基准测试。 通过提供这个全面的基准测试和工具包，CDRF旨在加速向稳健且通用的射频感知模型的发展进程，使分类、开放式识别和目标检测的标准基准测试成为可能，支持严格比较与可重复性研究。
URL
https://arxiv.org/abs/2601.03302
PDF
-
SLGNet: Synergizing Structural Priors and Language-Guided Modulation for Multimodal Object Detection
2026-01-05 16:31:41Xiantai Xiang, Guangyao Zhou, Zixiao Wen, Wenshuai Li, Ben Niu, Feng Wang, Lijia Huang, Qiantong Wang, Yuhan Liu, Zongxu Pan, Yuxin HuAbstract
Multimodal object detection leveraging RGB and Infrared (IR) images is pivotal for robust perception in all-weather scenarios. While recent adapter-based approaches efficiently transfer RGB-pretrained foundation models to this task, they often prioritize model efficiency at the expense of cross-modal structural consistency. Consequently, critical structural cues are frequently lost when significant domain gaps arise, such as in high-contrast or nighttime environments. Moreover, conventional static multimodal fusion mechanisms typically lack environmental awareness, resulting in suboptimal adaptation and constrained detection performance under complex, dynamic scene variations. To address these limitations, we propose SLGNet, a parameter-efficient framework that synergizes hierarchical structural priors and language-guided modulation within a frozen Vision Transformer (ViT)-based foundation model. Specifically, we design a Structure-Aware Adapter to extract hierarchical structural representations from both modalities and dynamically inject them into the ViT to compensate for structural degradation inherent in ViT-based backbones. Furthermore, we propose a Language-Guided Modulation module that exploits VLM-driven structured captions to dynamically recalibrate visual features, thereby endowing the model with robust environmental awareness. Extensive experiments on the LLVIP, FLIR, KAIST, and DroneVehicle datasets demonstrate that SLGNet establishes new state-of-the-art performance. Notably, on the LLVIP benchmark, our method achieves an mAP of 66.1, while reducing trainable parameters by approximately 87% compared to traditional full fine-tuning. This confirms SLGNet as a robust and efficient solution for multimodal perception.
Abstract (translated)
基于RGB和红外（IR）图像的多模态物体检测对于全天气条件下的稳健感知至关重要。尽管最近采用适配器方法可以高效地将预训练在RGB图像上的基础模型应用于此类任务，但这些方法通常以牺牲跨模式结构一致性为代价来提高模型效率。因此，在高对比度或夜间等显著领域差距环境中，关键的结构线索常常会丢失。此外，传统的静态多模态融合机制往往缺乏环境感知能力，导致在复杂、动态场景变化下适应性差和检测性能受限。 为了克服这些限制，我们提出了SLGNet框架，这是一个参数高效的模型，它结合了基于冻结视觉变换器（ViT）基础模型内的层级结构先验与语言引导调制。具体而言，我们设计了一种“感知结构适配器”，以提取两种模态的分层结构表示，并将其动态注入到ViT中，以补偿ViT主干固有的结构退化问题。此外，我们还提出了一种“语言引导调节”模块，利用视觉-语言模型驱动的结构化说明来动态调整视觉特征，从而赋予模型强大的环境感知能力。 在LLVIP、FLIR、KAIST和DroneVehicle数据集上的广泛实验表明，SLGNet建立了新的性能标杆。特别是在LLVIP基准测试中，我们的方法实现了66.1的mAP值，并且相比于传统的全微调训练参数，可减少约87%的可训练参数数量。这证实了SLGNet是一个稳健而高效的多模态感知解决方案。 这种创新的方法不仅提高了模型在不同环境条件下的适应性和鲁棒性，而且通过更有效的参数利用实现了更高的检测精度和效率，为全天气条件下的物体识别提供了强有力的支持。
URL
https://arxiv.org/abs/2601.02249
PDF
-
A Survey of Medical Drones from Flight Dynamics, Guidance, Navigation, and Control Perspectives
2026-01-05 00:45:07Roshan Kumar Chhetri, Sarocha Jetawatthana, Thanakorn KhamvilaiAbstract
The integration of drones into the medical field has revolutionized healthcare delivery by enabling rapid transportation of medical supplies, organs, and even emergency assistance in remote or disaster-stricken areas. While other survey papers focus on the healthcare supply chain, operations, and medical emergency response aspects, this paper provides a comprehensive review of medical drones from the perspectives of flight dynamics and guidance, navigation, and control (GNC) systems. We first discuss the medical aerial delivery mission requirements and suitable uncrewed aerial system (UAS) configurations. We then address payload container design and optimization, and its effect on supplies and overall flight dynamics. We also explore the fundamental principles of GNC in the context of medical drone operations, highlighting key challenges arising from vibration, air temperature, pressure, and humidity, which affect the quality of medical supplies. The paper examines various GNC algorithms that can mitigate these challenges, as well as the algorithms' limitations. With these considerations, this survey aims to provide insights into optimizing GNC frameworks for medical drones, emphasizing research gaps and directions to improve real-world healthcare applications.
Abstract (translated)
将无人机集成到医疗领域已经通过实现医疗物资、器官甚至在偏远或受灾地区的紧急援助的快速运输，彻底改变了医疗服务。虽然其他调查论文主要关注医疗供应链、运营和急救响应方面，本文则从飞行动力学与导航控制（GNC）系统角度提供了一次全面审视医学无人机。我们首先讨论了空中医疗递送任务的需求及适合的无人驾驶航空器(UAS)配置。接着，探讨了有效载荷容器的设计优化及其对物资供应和整体飞行动态的影响。此外，本文还深入研究了在医疗无人机操作背景下GNC的基本原理，并突出振动、空气温度、气压和湿度等关键挑战如何影响医疗物资的质量。论文考察了可以缓解这些挑战的各种GNC算法以及其局限性。考虑到上述因素，这项调查旨在为优化医学无人机的GNC框架提供见解，并强调研究差距及改善现实世界医疗服务的方向。
URL
https://arxiv.org/abs/2602.06969
PDF
-
AirSpatialBot: A Spatially-Aware Aerial Agent for Fine-Grained Vehicle Attribute Recognization and Retrieval
2026-01-04 07:38:51Yue Zhou, Ran Ding, Xue Yang, Xue Jiang, Xingzhao LiuAbstract
Despite notable advancements in remote sensing vision-language models (VLMs), existing models often struggle with spatial understanding, limiting their effectiveness in real-world applications. To push the boundaries of VLMs in remote sensing, we specifically address vehicle imagery captured by drones and introduce a spatially-aware dataset AirSpatial, which comprises over 206K instructions and introduces two novel tasks: Spatial Grounding and Spatial Question Answering. It is also the first remote sensing grounding dataset to provide 3DBB. To effectively leverage existing image understanding of VLMs to spatial domains, we adopt a two-stage training strategy comprising Image Understanding Pre-training and Spatial Understanding Fine-tuning. Utilizing this trained spatially-aware VLM, we develop an aerial agent, AirSpatialBot, which is capable of fine-grained vehicle attribute recognition and retrieval. By dynamically integrating task planning, image understanding, spatial understanding, and task execution capabilities, AirSpatialBot adapts to diverse query requirements. Experimental results validate the effectiveness of our approach, revealing the spatial limitations of existing VLMs while providing valuable insights. The model, code, and datasets will be released at this https URL
Abstract (translated)
尽管远程传感视觉-语言模型（VLM）取得了显著进展，但现有模型在空间理解方面仍面临挑战，限制了其在实际应用中的有效性。为了推动远程传感领域中VLM的边界，我们特别关注由无人机捕捉到的车辆图像，并引入了一个具有空间意识的数据集AirSpatial，该数据集中包含超过206K条指令，并提出了两个新颖的任务：空间定位（Spatial Grounding）和空间问答（Spatial Question Answering）。这也是首个提供3DBB信息的远程传感定位数据集。为了有效利用现有VLM在图像理解方面的知识并将其应用到空间领域，我们采用了一种两阶段训练策略，包括图像理解预训练和空间理解微调。借助这一经过训练的空间感知VLM，我们开发了一个空中代理AirSpatialBot，它可以进行细粒度的车辆属性识别与检索。通过动态整合任务规划、图像理解、空间理解和任务执行能力，AirSpatialBot能够适应各种查询需求。实验结果验证了该方法的有效性，揭示了现有VLM在空间理解方面的局限，并提供了有价值的见解。模型、代码和数据集将在以下网址发布：[this URL]
URL
https://arxiv.org/abs/2601.01416
PDF
-
Crowded Video Individual Counting Informed by Social Grouping and Spatial-Temporal Displacement Priors
2026-01-03 14:24:12Hao Lu, Xuhui Zhu, Wenjing Zhang, Yanan Li, Xiang BaiAbstract
Video Individual Counting (VIC) is a recently introduced task aiming to estimate pedestrian flux from a video. It extends Video Crowd Counting (VCC) beyond the per-frame pedestrian count. In contrast to VCC that learns to count pedestrians across frames, VIC must identify co-existent pedestrians between frames, which turns out to be a correspondence problem. Existing VIC approaches, however, can underperform in congested scenes such as metro commuting. To address this, we build WuhanMetroCrowd, one of the first VIC datasets that characterize crowded, dynamic pedestrian flows. It features sparse-to-dense density levels, short-to-long video clips, slow-to-fast flow variations, front-to-back appearance changes, and light-to-heavy occlusions. To better adapt VIC approaches to crowds, we rethink the nature of VIC and recognize two informative priors: i) the social grouping prior that indicates pedestrians tend to gather in groups and ii) the spatial-temporal displacement prior that informs an individual cannot teleport physically. The former inspires us to relax the standard one-to-one (O2O) matching used by VIC to one-to-many (O2M) matching, implemented by an implicit context generator and a O2M matcher; the latter facilitates the design of a displacement prior injector, which strengthens not only O2M matching but also feature extraction and model training. These designs jointly form a novel and strong VIC baseline OMAN++. Extensive experiments show that OMAN++ not only outperforms state-of-the-art VIC baselines on the standard SenseCrowd, CroHD, and MovingDroneCrowd benchmarks, but also indicates a clear advantage in crowded scenes, with a 38.12% error reduction on our WuhanMetroCrowd dataset. Code, data, and pretrained models are available at this https URL.
Abstract (translated)
视频个体计数（VIC）是一个最近引入的任务，旨在通过分析视频来估算行人的流量。它将视频人群计数（VCC）扩展到超出每一帧的行人数量统计，并且需要在帧之间识别共存的行人，这实际上成为了一个对应问题。然而，现有的VIC方法在这种情况下可能表现不佳，尤其是在地铁通勤等拥挤场景中。 为了解决这个问题，我们构建了WuhanMetroCrowd，这是第一个能够描述拥挤、动态的人流变化情况的数据集之一。该数据集涵盖了从稀疏到密集的密度水平、短至长的视频片段、慢至快的流动变化范围以及从前视图到后视图的外观变化和不同程度的遮挡情况。 为了更好地适应VIC方法在人群中的应用，我们重新思考了VIC的本质，并识别出两个有用的先验知识：一是社会群体先验，表明行人倾向于聚集在一起；二是时空位移先验，指出一个人不能瞬间移动。前者启发我们将VIC中常用的严格一对一（O2O）匹配放宽为一对多（O2M）匹配，通过隐式上下文生成器和O2M匹配器来实现这一点；后者帮助设计了一个位移先验注入器，不仅加强了O2M匹配，还提升了特征提取和模型训练的效果。 这些设计理念共同构成了一个新的强VIC基准线OMAN++。大量的实验表明，OMAN++不仅在标准的SenseCrowd、CroHD和MovingDroneCrowd等基准测试中超越了现有的最佳VIC方法，在拥挤场景中的表现也有明显的改进，尤其是在我们构建的WuhanMetroCrowd数据集中，误差减少了38.12%。 代码、数据及预训练模型可在以下链接获取：[提供的URL]。
URL
https://arxiv.org/abs/2601.01192
PDF
