Abstract
Cross-view spatial reasoning is essential for embodied AI, underpinning spatial understanding, mental simulation and planning in complex environments. Existing benchmarks primarily emphasize indoor or street settings, overlooking the unique challenges of open-ended urban spaces characterized by rich semantics, complex geometries, and view variations. To address this, we introduce CityCube, a systematic benchmark designed to probe cross-view reasoning capabilities of current VLMs in urban settings. CityCube integrates four viewpoint dynamics to mimic camera movements and spans a wide spectrum of perspectives from multiple platforms, e.g., vehicles, drones and satellites. For a comprehensive assessment, it features 5,022 meticulously annotated multi-view QA pairs categorized into five cognitive dimensions and three spatial relation expressions. A comprehensive evaluation of 33 VLMs reveals a significant performance disparity with humans: even large-scale models struggle to exceed 54.1% accuracy, remaining 34.2% below human performance. By contrast, small-scale fine-tuned VLMs achieve over 60.0% accuracy, highlighting the necessity of our benchmark. Further analyses indicate the task correlations and fundamental cognitive disparity between VLMs and human-like reasoning.
Abstract (translated)
跨视角空间推理对于具身人工智能（Embodied AI）至关重要，它是复杂环境中空间理解、心理模拟和规划的基础。现有基准测试主要集中在室内或街道场景上，忽略了开放型城市空间的独特挑战，这些地方以其丰富的语义信息、复杂的几何形状和视图变化为特点。为了应对这一问题，我们引入了CityCube，这是一个系统性基准测试工具，旨在评估当前视觉语言模型（VLMs）在城市环境中跨视角推理能力的表现。 CityCube整合了四种视角动态来模拟相机移动，并跨越多个平台的广泛视角范围进行操作，如车辆、无人机和卫星。为了进行全面评价，它包括5,022对细致标注的多视图问答配对，这些配对被分类为五种认知维度和三种空间关系表达方式。 33个VLM模型的综合评估显示了与人类表现之间的显著差异：即使大规模模型也难以超过54.1%的准确率，比人类的表现低34.2个百分点。相比之下，小型精细调优的VLM可以达到60.0%以上的准确率，这凸显了我们基准测试工具的重要性。 进一步分析表明，任务之间存在相关性以及视觉语言模型与人类类似推理之间的基本认知差异。
URL
https://arxiv.org/abs/2601.14339
