Abstract
Modern autonomous vehicle perception systems are often constrained by occlusions, blind spots, and limited sensing range. While existing cooperative perception paradigms, such as Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I), have demonstrated their effectiveness in mitigating these challenges, they remain limited to ground-level collaboration and cannot fully address large-scale occlusions or long-range perception in complex environments. To advance research in cross-view cooperative perception, we present V2U4Real, the first large-scale real-world multi-modal dataset for Vehicle-to-UAV (V2U) cooperative object perception. V2U4Real is collected by a ground vehicle and a UAV equipped with multi-view LiDARs and RGB cameras. The dataset covers urban streets, university campuses, and rural roads under diverse traffic scenarios, comprising over 56K LiDAR frames, 56K multi-view camera images, and 700K annotated 3D bounding boxes across four classes. To support a wide range of research tasks, we establish benchmarks for single-agent 3D object detection, cooperative 3D object detection, and object tracking. Comprehensive evaluations of several state-of-the-art models demonstrate the effectiveness of V2U cooperation in enhancing perception robustness and long-range awareness. The V2U4Real dataset and codebase is available at this https URL.
Abstract (translated)
现代自动驾驶感知系统常受到遮挡、盲点及感知范围有限的制约。尽管现有的协同感知范式（如车对车V2V、车对基础设施V2I）已展现缓解这些挑战的有效性，但它们仍局限于地面层面的协作，无法在复杂环境中完全解决大规模遮挡或远距离感知问题。为推进跨视角协同感知研究，我们推出了V2U4Real——首个面向车对无人机（V2U）协同目标感知的大规模真实世界多模态数据集。该数据集由配备多视角激光雷达与RGB相机的地面车辆及无人机协同采集，涵盖城市街道、大学校园及乡村道路等多种交通场景，包含超过5.6万组激光雷达帧、5.6万组多视角相机图像，以及跨四个类别的70万个标注3D边界框。为支持广泛的研究任务，我们建立了单智能体3D目标检测、协同3D目标检测及目标跟踪的基准测试。多项先进模型的综合评估表明，V2U协同能有效提升感知鲁棒性与远距离环境感知能力。V2U4Real数据集及代码库可通过此链接获取。
URL
https://arxiv.org/abs/2603.25275
