Abstract
Open-vocabulary semantic segmentation (OVSS) involves assigning labels to each pixel in an image based on textual descriptions, leveraging world models like CLIP. However, they encounter significant challenges in cross-domain generalization, hindering their practical efficacy in real-world applications. Embodied AI systems are transforming autonomous navigation for ground vehicles and drones by enhancing their perception abilities, and in this study, we present AetherVision-Bench, a benchmark for multi-angle segmentation across aerial, and ground perspectives, which facilitates an extensive evaluation of performance across different viewing angles and sensor modalities. We assess state-of-the-art OVSS models on the proposed benchmark and investigate the key factors that impact the performance of zero-shot transfer models. Our work pioneers the creation of a robustness benchmark, offering valuable insights and establishing a foundation for future research.
Abstract (translated)
开放词汇语义分割(OVSS)涉及根据文本描述将标签分配给图像中的每个像素,利用像CLIP这样的世界模型。然而,这类方法在跨域泛化方面面临重大挑战,这限制了它们在实际应用中的有效性。嵌入式AI系统通过增强自主导航的感知能力,正在改变地面车辆和无人机的应用方式。在这项研究中,我们提出了AetherVision-Bench,这是一个针对多角度分割的基准测试平台,涵盖了空中和地面视角,有助于对不同观察角度和传感器模态下的性能进行全面评估。我们在提出的基准上评估了最新的OVSS模型,并探讨了影响零样本迁移模型性能的关键因素。我们的工作开创了一个鲁棒性基准的先河,提供了有价值的见解并为未来的研究奠定了基础。
URL
https://arxiv.org/abs/2506.03709