Abstract
Weakly-supervised Human-Object Interaction (HOI) detection is essential for scalable scene understanding, as it learns interactions from only image-level annotations. Due to the lack of localization signals, prior works typically rely on an external object detector to generate candidate pairs and then infer their interactions through pairwise reasoning. However, this framework often struggles to scale due to the substantial computational cost incurred by enumerating numerous instance pairs. In addition, it suffers from false positives arising from non-interactive combinations, which hinder accurate instance-level HOI reasoning. To address these issues, we introduce Relational Grounding Transformer (RegFormer), a versatile interaction recognition module for efficient and accurate HOI reasoning. Under image-level supervision, RegFormer leverages spatially grounded signals as guidance for the reasoning process and promotes locality-aware interaction learning. By learning localized interaction cues, our module distinguishes humans, objects, and their interactions, enabling direct transfer from image-level interaction reasoning to precise and efficient instance-level reasoning without additional training. Our extensive experiments and analyses demonstrate that RegFormer effectively learns spatial cues for instance-level interaction reasoning, operates with high efficiency, and even achieves performance comparable to fully supervised models. Our code is available at this https URL.
Abstract (translated)
弱监督人-物交互(HOI)检测对于可扩展的场景理解至关重要,因为它仅从图像级标注中学习交互信息。由于缺乏定位信号,先前的工作通常依赖外部目标检测器生成候选对,然后通过成对推理推断其交互。然而,该框架常因穷举大量实例对而产生的巨大计算成本而难以扩展。此外,它还会受到非交互组合产生的假阳性干扰,从而阻碍准确的实例级HOI推理。为解决这些问题,我们提出了关系定位Transformer(RegFormer),一种用于高效准确HOI推理的多功能交互识别模块。在图像级监督下,RegFormer利用空间定位信号指导推理过程,并促进具有局部感知的交互学习。通过学习局部化的交互线索,我们的模块能够区分人类、物体及其交互,从而无需额外训练即可直接从图像级交互推理迁移到精确高效的实例级推理。大量实验与分析表明,RegFormer能有效学习空间线索以进行实例级交互推理,运行效率高,甚至能达到与全监督模型相当的性能。我们的代码可在以下链接获取:https URL。
URL
https://arxiv.org/abs/2604.00507