Paper Reading AI Learner

HCCM: Hierarchical Cross-Granularity Contrastive and Matching Learning for Natural Language-Guided Drones

2025-08-29 11:50:24
Hao Ruan, Jinliang Lin, Yingxin Lai, Zhiming Luo, Shaozi Li

Abstract

Natural Language-Guided Drones (NLGD) provide a novel paradigm for tasks such as target matching and navigation. However, the wide field of view and complex compositional semantics in drone scenarios pose challenges for vision-language understanding. Mainstream Vision-Language Models (VLMs) emphasize global alignment while lacking fine-grained semantics, and existing hierarchical methods depend on precise entity partitioning and strict containment, limiting effectiveness in dynamic environments. To address this, we propose the Hierarchical Cross-Granularity Contrastive and Matching learning (HCCM) framework with two components: (1) Region-Global Image-Text Contrastive Learning (RG-ITC), which avoids precise scene partitioning and captures hierarchical local-to-global semantics by contrasting local visual regions with global text and vice versa; (2) Region-Global Image-Text Matching (RG-ITM), which dispenses with rigid constraints and instead evaluates local semantic consistency within global cross-modal representations, enhancing compositional reasoning. Moreover, drone text descriptions are often incomplete or ambiguous, destabilizing alignment. HCCM introduces a Momentum Contrast and Distillation (MCD) mechanism to improve robustness. Experiments on GeoText-1652 show HCCM achieves state-of-the-art Recall@1 of 28.8% (image retrieval) and 14.7% (text retrieval). On the unseen ERA dataset, HCCM demonstrates strong zero-shot generalization with 39.93% mean recall (mR), outperforming fine-tuned baselines.

Abstract (translated)

自然语言引导的无人机(NLGD)为诸如目标匹配和导航等任务提供了一种新颖的方法。然而,无人机场景中的宽广视野和复杂的组合语义对视觉-语言理解提出了挑战。主流的视觉-语言模型(VLMs)强调全局一致性,但缺乏细粒度语义的理解,并且现有的层次方法依赖于精确的对象分割和严格的包含关系,在动态环境中效果受限。为了解决这些问题,我们提出了一种分层跨粒度对比与匹配学习(HCCM)框架,该框架由两个部分组成: 1. 区域-全局图像文本对比学习(RG-ITC),它避免了精确场景分割的需求,并通过局部视觉区域和全局文本之间的对比捕获从局部到全局的层次化语义信息; 2. 区域-全局图像文本匹配(RG-ITM),该方法不再依赖于严格的约束,而是评估全局跨模态表示中的局部语义一致性,从而增强组合推理能力。 此外,在无人机场景中,由于描述常常不完整或含糊不清,使得对齐过程不稳定。为提高鲁棒性,HCCM引入了动量对比和蒸馏(MCD)机制。在GeoText-1652数据集上的实验表明,HCCM实现了最先进的Recall@1性能:图像检索中为28.8%,文本检索中为14.7%。此外,在未见过的ERA数据集中,HCCM展示了强大的零样本泛化能力,平均召回率(mR)达到39.93%,优于微调后的基线模型。

URL

https://arxiv.org/abs/2508.21539

PDF

https://arxiv.org/pdf/2508.21539.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot