Abstract
We present HetroD, a dataset and benchmark for developing autonomous driving systems in heterogeneous environments. HetroD targets the critical challenge of navi- gating real-world heterogeneous traffic dominated by vulner- able road users (VRUs), including pedestrians, cyclists, and motorcyclists that interact with vehicles. These mixed agent types exhibit complex behaviors such as hook turns, lane splitting, and informal right-of-way negotiation. Such behaviors pose significant challenges for autonomous vehicles but remain underrepresented in existing datasets focused on structured, lane-disciplined traffic. To bridge the gap, we collect a large- scale drone-based dataset to provide a holistic observation of traffic scenes with centimeter-accurate annotations, HD maps, and traffic signal states. We further develop a modular toolkit for extracting per-agent scenarios to support downstream task development. In total, the dataset comprises over 65.4k high- fidelity agent trajectories, 70% of which are from VRUs. HetroD supports modeling of VRU behaviors in dense, het- erogeneous traffic and provides standardized benchmarks for forecasting, planning, and simulation tasks. Evaluation results reveal that state-of-the-art prediction and planning models struggle with the challenges presented by our dataset: they fail to predict lateral VRU movements, cannot handle unstructured maneuvers, and exhibit limited performance in dense and multi-agent scenarios, highlighting the need for more robust approaches to heterogeneous traffic. See our project page for more examples: this https URL
Abstract (translated)
我们介绍了HetroD，这是一个用于开发异质环境中自主驾驶系统的数据集和基准。HetroD旨在解决在由易受伤害的道路使用者（VRU）主导的真实世界异质交通环境中的导航挑战，包括行人、骑自行车者和摩托车手与车辆的交互行为。这些混合代理类型表现出复杂的动态行为，如钩形转弯、车道分隔以及非正式的优先权协商等。这类行为对自动驾驶汽车构成了重大挑战，但在现有的专注于结构化且遵守车道规则交通的数据集中却严重不足。 为了弥补这一差距，我们收集了一个大规模基于无人机的数据集，为异质交通场景提供了全面观察，并带有厘米级精度的标注、高清地图和交通信号状态信息。此外，我们还开发了一套模块化的工具包来提取单个代理的情景以支持下游任务的发展。总体而言，该数据集中包含了超过65,400条高保真度代理轨迹，其中70%来源于VRU。HetroD支持对密集和异质交通中VRU行为的建模，并为预测、规划及模拟任务提供了标准化基准。 评估结果显示，最先进的预测和规划模型在处理我们数据集提出的挑战时存在困难：它们无法准确预测侧向VRU运动轨迹，不能应对非结构化的操作动作，且在高密度与多代理场景中的性能表现有限。这突显了开发更稳健的方法来处理异质交通的必要性。 更多信息请参见我们的项目页面: [此URL](this https URL)
URL
https://arxiv.org/abs/2602.03447
