Contrastive_Learning
-
Meta-CLIP: Metadata-Conditioned Contrastive Learning for Zero-Shot Disease Inference
2026-10-24 13:45:22Haozhe Luo, Nirosh Sivanesan, Shelley Zixin Shu, Francesco Nitti, Mauricio ReyesAbstract
Medical vision--language pretraining learns transferable representations from paired images and radiology reports, yet differences in scanners and acquisition protocols can impair transfer. Existing approaches primarily enrich clinical semantics, leaving structured acquisition information in DICOM headers underused. We introduce Meta-CLIP, a metadata-conditioned contrastive framework that uses acquisition similarity to reweight cross-modal negatives during pretraining. A kernel over numerical and categorical DICOM fields defines a bounded pairwise margin. This margin increases the contribution of acquisition-similar negatives, emphasizing distinctions among studies that share imaging conditions. The kernel remains fixed during image--text optimization, and inference follows the standard dual-encoder architecture without requiring metadata. We evaluate Meta-CLIP on chest radiography and volumetric chest CT through zero-shot classification, appearance perturbations, grounding, and representation analyses. On CT-RATE, Meta-CLIP improves macro-AUROC over CT-CLIP by 1.41 points, with a 95\% paired bootstrap confidence interval of 0.96--1.85 points. It also achieves the highest macro-AUROC on four of five CXR benchmarks. An exploratory BrgSA evaluation extends the analysis to another CT backbone. These findings support acquisition metadata as a practical training-time prior for medical vision--language learning. Code will be made publicly available upon publication.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33674
PDF
-
LoCA: Enabling Multi-Modality in Geospatial Location Encoders
2026-10-23 17:01:04Panagiotis Koromilas, Eleanna Vali, Diogenis Antonopoulos, Eleni Saka, Dimitris Christofis, Nikolaos Ioannis Bountos, Konstantin Klemmer, Ioannis Papoutsis, Yannis Panagakis, Mihalis A. NicolaouAbstract
Location encoders map geographic coordinates to high-dimensional descriptors through contrastive alignment with paired Earth Observation imagery. We prove that this construction is structurally limited: any Lipschitz location encoder produces an output of intrinsic dimension at most 2, regardless of architecture or bandwidth, which forces the contrastive loss to strictly exceed its asymptotic optimum. Empirically, joint training also collapses the encoder's embedding covariance to low rank. Although dimension-limited, current location encoders sidestep this spectral collapse by training against a frozen image encoder, which we reinterpret as a covariance regularizer that supplies a high-rank target for the location encoder to fold onto. However, extending this approach to multimodal inputs is challenging, as location encoders should simultaneously map a single location to multiple, potentially incompatible, descriptors. We identify two requirements a regularizer must satisfy for location encoding in multimodal regimes: co-alignment across modalities, since adapters alone cannot align independently pretrained encoders; and high effective rank, since the location encoder inherits its target's spectrum. We propose LoCA (Location Contrastive Alignment), a two-stage pipeline that satisfies both. Stage 1 jointly trains Sentinel-1 and Sentinel-2 encoders with contrastive learning, producing a co-aligned, high-rank multimodal target. Stage 2 freezes the image encoders and trains a location and a tabular metadata encoder as projection adapters with multi-view contrastive learning. Against five state-of-the-art encoders on 17 tasks ranging from land cover and wildfire forecasting to geophysical, socio-economic, biodiversity and climate applications, LoCA achieves the most task wins, is the only model to outperform every other on a majority of tasks, and wins 68.8% of all per-task comparisons.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32894
PDF
-
Internal Discrepancy for Contrastive Learning: Generalization Bounds and Downstream Behaviors
2026-10-23 14:48:35Jialiang Wang, Qing Ma, Xiong Zhou, Gangfeng Hu, Haoliang Li, Xianming LiuAbstract
Contrastive learning has emerged as a popular paradigm for training representation models. While prior studies pursue more practical and less restrictive assumptions to enhance theoretical understanding of contrastive learning, they essentially describe the interconnections among representations within the data or induced by the function class. To encapsulate these assumptions, we propose the measure of internal discrepancy that quantifies the difference between the mean representation of positive samples and latent cluster centers. We then prove an internal discrepancy-aware generalization bound for downstream supervised tasks following contrastive learning, without requiring additional assumptions. This generalization bound not only serves as a unifying framework connecting various assumptions but also motivates us to further reduce the internal discrepancy, thereby introducing the relaxed k-means loss as an approximation. Moreover, we investigate the downstream behaviors of contrastive learning from the perspective of contrastive learning-incurred supervised losses. In the presence of class imbalance, we shed light on why contrastive learning can facilitate imbalanced learning. Finally, we investigate the optimality of these incurred supervised losses, proving several intriguing properties.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32729
PDF
-
WiMoT: Replay-Free Continual Wi-Fi Sensing via Moment Transport
2026-10-23 14:05:37Chunchun Yang, Yang Cao, Zhitang ChenAbstract
Continual Wi-Fi sensing must balance plasticity, learning new rooms or users, against stability, keeping earlier ones, yet nodes often cannot retain channel state information (CSI) for privacy and storage reasons. We propose WiMoT (Wi-Fi Moment Transport), which stores the per-class feature mean and covariance of every earlier domain instead of samples. When the representation changes, stored statistics become stale. WiMoT restricts the change to an affine map on a frozen feature basis and applies the same map to the statistics, so they stay valid without old samples. Supervised contrastive learning learns the new domain, a closed-form loss on the stored statistics keeps earlier classes separable, and a Gaussian discriminant classifier, refitted from the transported statistics with equal weight per domain, predicts without domain identity. On 3DO, WiMANS, and Widar3.0, WiMoT's average final accuracies are 94.02%, 89.61%, and 95.16% with at most 0.58 percentage points (pp) of forgetting. On 3DO and Widar3.0, it also has the highest balance of new- and earlier-domain accuracy and is 3.77 and 13.11 pp ahead of the strongest of 21 baselines, which uses replay, staying ahead across domain orders and label budgets. On WiMANS, it matches the strongest replay baseline in accuracy (+0.41 pp) while reducing forgetting from 2.78 to 0.00 pp, and has the highest six-order average accuracy. Because full covariances dominate the memory, storing them as rank-32 factors shrinks the moment memory to 6.4%, losing at most 0.49 pp of accuracy, within edge-node storage budgets.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32660
PDF
-
GeoLEON-CLIP: Graph-Anchored Multimodal Contrastive Learning for Geographic Region Representation
2026-10-23 08:11:23Radoslaw Malek, Szymon Soltysiak, Aleksandra Kawala-Sterniuk, Piotr SzymanskiAbstract
Geographic regions are multimodal: satellite imagery captures visual morphology, text conveys human-interpretable semantics, and map structures describe urban context. Existing geospatial contrastive methods typically align image-text or image-location pairs, underusing structured map data. We introduce GeoLEON-CLIP, a graph-anchored contrastive framework that aligns three views of each H3-indexed region: an OpenStreetMap-derived graph embedding, a satellite image crop, and a textual description. Frozen pretrained LEON graph embeddings provide a semantic anchor between vision and language. Frozen graph, image, and text encoders are connected by trainable residual projection heads and trained with a symmetric tri-modal contrastive objective. Experiments show strong graph-image and graph-text alignment; graph-image retrieval achieves the best cross-modal performance in a mixed-resolution pool. To evaluate the anchoring hypothesis, we propose chained cross-modal recall $\mathrm{R}@A@B$, which routes retrieval through the graph modality and compares it with direct retrieval under a matched candidate budget. At $\mathrm{R}@1@1$, image$\rightarrow$graph$\rightarrow$text and text$\rightarrow$graph$\rightarrow$image outperform direct image-text retrieval by 4.8 and 4.4 points, respectively, despite using a single intermediate cell. Downstream probes show task-dependent gains, with the largest improvements on Airbnb and Beijing housing. These results indicate that OpenStreetMap graph structure can anchor multimodal representations combining maps, imagery, and language.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32311
PDF
-
Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data
2026-10-23 01:30:14Joonhyeok Jeong, Jy-yong Sohn, Kibok LeeAbstract
Supervised contrastive learning and its balanced variants have demonstrated their effectiveness for representation learning under long-tailed distributions. However, although prior analyses suggest that their optimal representations may exhibit a regular simplex geometry, whether such a geometry is indeed exhibited remains insufficiently understood. In this work, we present a theoretical framework for these objectives that characterizes whether a regular simplex is formed at the optimum. By applying this to representative objectives, we show that these objectives fail to form a regular simplex at the optimum because they remain dependent on the number of samples per class. Motivated by this analysis, we introduce structural balancing that can be applied to contrastive objectives, and propose a fully balanced contrastive objective that provably forms a regular simplex geometry at its optimum. Both theoretical and empirical results demonstrate that the proposed method yields a more balanced representation space and improves performance on long-tailed recognition and transfer learning tasks.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32039
PDF
-
Synthetic-to-Real Text-Based Person Anomaly Search via Multi-Model Retrieval Fusion and Cross-Encoder Reranking
2026-10-22 21:19:19Eugene Denteh, Blessing Agyei Kyem, Andrews Danyo, Joshua Kofi Asamoah, Richard Dzinyela, Solomon Owusu-Ansah, Dickson G. Asiedu, Armstrong AboahAbstract
Text-based person anomaly search requires a model to retrieve individuals from a large image gallery using natural-language descriptions of both appearance and behavior. This task is challenging because model training relies on synthetic image-text pairs, whereas evaluation is performed on real surveillance images containing domain differences, near-duplicate frames, and many visually similar distractors. We present a multi-model retrieval and reranking framework designed to improve synthetic-to-real transfer. The framework first adapts Qwen3-VL-Embedding-8B to synthetic person descriptions using bidirectional contrastive learning, explicit negative sampling, false-negative masking, and domain randomization. It then combines the normalized scores of six complementary Qwen3-VL checkpoints with those of a frozen SigLIP2 model using four-view gallery encoding. The fused retrieval scores identify the strongest candidates, which are jointly evaluated with the query using Qwen3-VL-Reranker-8B. The cross-encoder and retrieval scores are subsequently blended to produce the final ranking. We also construct a distractor-matched validation protocol entirely from synthetic data to select checkpoint weights, fusion coefficients, candidate depths, and reranking parameters without accessing the official test images. The final system achieves 94.7815% mAP and 90.8493% R@1 under the official AI City Challenge 2026 Track 4 test setting, with a reported eighth-place finish on the public leaderboard.
Abstract (translated)
URL
https://arxiv.org/abs/2610.31930
PDF
-
UniGuard: Multi-View Guardrail Learning for Cross-Interface Safety Generalization
2026-10-22 05:14:30Viet-Thanh Pham, Thuy-Trang Vu, Junae Kim, Rollin Omari, Dinh Phung, Gholamreza Haffari, Lizhen QuAbstract
Guardrail models are increasingly used as safety layers for LLMs and agentic systems, but most are trained on fixed prompt or response interfaces and become unreliable when the same safety case appears as a conversation, agent trajectory, multi-agent trace, or policy-conditioned input. We propose UniGuard, a training framework that learns interface-invariant safety representations from multiple semantically aligned views of the same safety case. We also construct the UniGuard-FT dataset by transforming 1,000 public safety cases into prompt, response, multi-turn conversation, agent-trace, and multi-agent-trace views, and train UniGuard-0.6B with classification, supervised contrastive learning, and asymmetric confidence calibration losses. Across unseen-interface benchmarks ATBench, PolicyGuardBench, and DynaBench, UniGuard-0.6B improves over its pretrained guardrail by 45.12 F1 points on average and reduces expected calibration error by 35.41 points, while remaining competitive with much larger guardrails such as Claude-Sonnet-4.5 and GPT-5.2 - despite being a 0.6B model and trained only with synthetic data.
Abstract (translated)
URL
https://arxiv.org/abs/2610.30742
PDF
-
UNIVERS: Learning UNIversal Voxel-wise Embedding via Randomized Synthesis for generalizable one-shot landmark localization
2026-10-21 02:42:14Derong Yu, Qingsong Zhao, Yanbing Yue, Yikai Li, Zhi Zhang, Guoyan ZhengAbstract
One-shot landmark localization in volume scans with varying fields of view (FOVs) is challenging. While existing self-supervised learning methods have shown promising performance, they rely on contrastive learning built upon augmented views of the same image, which inherently limits their generalization in cross-modality scenarios. This motivates research on cross-modality embedding learning using paired multi-modality images. However, existing methods suffer from not only limited training data, but also the requirement of multi-stage iterative training. To address these limitations, we propose UNIVERS, an end-to-end trainable framework that learns UNIversal Voxel-wise Embedding via Randomized Synthesis. Our framework leverages diverse pseudo-data synthesized from organ shape templates, with randomization applied to semantic layout, visual appearance, FOV, and spatial pose. Coupled with cross-domain discriminative voxel-wise contrastive learning, this domain randomization strategy forces the network to disregard domain-specific biases and learn features that are robust against variations in imaging contrast, resolution, and spatial context. Consequently, the trained model enables one-shot landmark localization in multi-modality volume scans with arbitrary FOVs. In this study, the efficacy of the proposed method is demonstrated by applying it to the task of localizing the adrenal glands, which are paired, small, and irregular endocrine organs. In particular, comprehensive experiments are conducted on three challenging adrenal gland datasets. The experimental results demonstrate that UNIVERS consistently outperforms the state-of-the-art (SOTA) one-shot landmark localization approaches, with particularly prominent advantages in cross-modality scenarios. The source code is publicly available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2610.29098
PDF
-
Test-time Adaptation for Point Cloud Denoising under Unseen Noise
2026-10-20 00:04:09Kun Jiang, Shida Fu, Fanhao Yi, Weiqing Li, Zhiyong SuAbstract
Although deep learning-based point cloud denoising methods have demonstrated impressive results in recent years, their effectiveness usually degrades substantially in real-world scenarios where the noise distribution of test point clouds deviates from that of the training set. To address this challenge, we propose a test-time adaptive denoising (TTAD) framework that enables existing denoising approaches to adapt to unknown noise patterns at test time without relying on additional supervision or retraining from source data. Taking a pre-trained denoising model and a test noisy point cloud as input, TTAD first employs a residual-driven noise modeling (RDNM) module to estimate the sample-specific noise distribution through analyzing the deviation between the noisy input and its pseudo-clean version denoised by the initial model. Then, a noise-aware contrastive adaptation (NACA) module is designed to build sample-specific noisy-noisier test-time training pair from the denoised point cloud, and update the model parameters over a few adaptation steps per sample at test time through contrastive learning. Furthermore, TTAD naturally extends to a continual TTAD (CTTAD) setting, facilitating efficient progressive refinement across sequential test samples with consistent noise distributions, while reducing the computational cost per sample. Extensive experiments on both synthetic and real-world datasets demonstrate that our approach significantly improves the generalization ability of existing denoising models under unseen noise, achieving superior performance across multiple evaluation metrics.
Abstract (translated)
URL
https://arxiv.org/abs/2610.27410
PDF
-
SimOPD: Relational Knowledge Transfer for On-Policy Distillation
2026-10-16 11:10:21Kai Tang, Xing Zheng, Haobo Wang, Guangcheng Zhu, Shenzhi Yang, Zhengqing Zang, Yingfan Ma, Bowen Song, Weiqiang WangAbstract
A key challenge in on-policy distillation (OPD) is extracting effective supervision from teacher knowledge. Supervision from token probabilities may underutilize internal knowledge, while direct hidden-state matching forces the student to shift to the teacher's feature space and complicates cross-architecture transfer. Inspired by contrastive learning's use of relationships between instances, we propose \textbf{SimOPD} to transfer teacher internal knowledge through response similarities. Specifically, SimOPD uses paired memory banks of student-generated responses to extract supervision from the teacher's internal representations as similarity distributions. The student then matches these distributions within its own representation space, retaining representational freedom and removing the need for cross-model projection. Extensive experiments show that SimOPD improves the overall average over the strongest baselines by up to 2.6 and 3.1 percentage points in same- and cross-architecture settings.
Abstract (translated)
URL
https://arxiv.org/abs/2610.23306
PDF
-
VTrajEmbed: Aligning Text with Video Object Trajectories via Spatiotemporal Embeddings
2026-10-16 02:34:07Ruizhi Chen, Zining Fan, Yudi Guo, Jiang Liu, Zhiyu Lin, Zhongjiang He, Shouqian Shi, Hao SunAbstract
Fine-grained alignment between text and video object trajectories is a core challenge in multimodal understanding. Existing embedding models excel at global retrieval, yet their single global representation entangles multiple objects and their spatiotemporal dynamics, limiting direct object-level grounding. To address this issue, we propose VTrajEmbed, an embedding model that decomposes a video into object trajectories and aligns each trajectory embedding with text for object-level grounding through retrieval. Our main contributions: (1)Slow-Fast Trajectory Encoding, a dual-branch architecture that establishes trajectory representations by densely sampling low-resolution frames for high-frequency motion cues and sparsely selecting high-resolution frames for scene context, aggregated by a shared LLM backbone; (2)Mask-Guided Contrastive Learning, a training objective built upon these representations that couples an intra-group loss and a batch-level bidirectional loss under a relation mask, excluding unverified relations and preventing confirmed matches from becoming false negatives; (3)Text2Traj-Bench, a trajectory-level text-video benchmark with model-assisted construction and human-verified query--trajectory relations. Experimental results show that VTrajEmbed-5B improves text-to-trajectory and trajectory-to-text R@1 over Qwen3-VL-Embedding-8B by 15.48 and 18.05 percentage points, respectively, and achieves 49.48% average R@1 across five global retrieval benchmarks, compared with 42.86% for Qwen3-VL-Embedding-8B.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22842
PDF
-
SCCL-DT: Semantic-Conflict-Aware Contrastive Learning with Dual-Rule Teacher Distillation for Zero-Shot Chest X-Ray Diagnosis
2026-10-15 15:44:40Ting Liu, Haoran Lai, HaobinChen, Ming Liu, S Kevin ZhouAbstract
Contrastive vision-language pre-training supports zero-shot chest X-ray diagnosis, but standard objectives treat every off-diagonal image-report pair in a batch as a negative. In radiology, recurring findings across patients mean that instance mismatch need not imply semantic incompatibility, creating false-negative supervision that we term a semantic conflict. A training-trace audit shows that pair-observations selected by the online mask used during training are enriched for report-derived finding agreement, although this proxy analysis does not establish clinical equivalence. Fixed thresholds can become miscalibrated as similarity distributions evolve, while soft reweighting leaves residual repulsive gradients. We therefore propose Semantic-Conflict-Aware Contrastive Learning (SCCL), which tracks modality-specific similarity distributions with exponential-moving-average (EMA) statistics, derives adaptive thresholds, and hard-masks flagged candidates before softmax normalization. Inclusive (OR) and consensus (AND) rules define two SCCL teachers, whose bidirectional cross-modal rankings are consolidated through standard distillation into a single deployable student, SCCL-DT. Across five public chest X-ray benchmarks, SCCL-DT achieves a mean zero-shot AUC of $0.850$, compared with $0.837$ for the strongest evaluated baseline in our comparison. Code will be made publicly available soon.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22094
PDF
-
CIM-Embed: Learning Instruction-Conditioned Multimodal Embeddings for Clustering
2026-10-15 12:54:17Yinghua Xu, Pengyang Xie, Yanxiang Zhou, Chuanzhi Tong, Xiao Xiao, Chaoqun Feng, Xiaoqing WangAbstract
Multimodal objects can require different partitions under different instructions, as when products in an e-commerce catalog are grouped by color, material, or both. Retrieval-oriented embeddings may not recover these partitions, while generative clustering often relies on intermediate descriptions or iterative decisions. This paper presents CIM-Embed, a framework for learning instruction-conditioned multimodal embeddings for clustering. Supervised contrastive learning uses same-group positives and negatives from other groups, optionally including other tasks; heterogeneous structured batches accommodate tasks with different group counts. At inference, the model maps each object's image, text, and instruction directly to an embedding; HDBSCAN then forms a partition without the ground-truth cluster count. We construct ProductMIC from product metadata in Amazon Reviews 2023, comprising training data and an evaluation benchmark for instruction-conditioned clustering. The benchmark contains 5,373 products and 449 single-attribute and compositional grouping tasks spanning 50 leaf categories excluded from our training. With full-dimensional embeddings and HDBSCAN, direct within-task SupCon training achieves a mean per-task V-measure of 95.01, compared with 53.06 for WeMM 9B, the strongest general-purpose embedding baseline evaluated. Frozen product-trained encoders also improve out-of-domain conditional grouping, while controlled transfer experiments distinguish partition transfer from instruction response. We will release the code and dataset upon acceptance.
Abstract (translated)
URL
https://arxiv.org/abs/2610.21805
PDF
-
Multimodal Contrastive Representation Learning Based Voice Conversion for Electrolaryngeal Speech Enhancement
2026-10-13 04:47:07Ding Ma, Jinyi Mi, Jiajun He, Kazuhiro Kobayashi, Naoki Nishio, Tomoki TodaAbstract
Electrolaryngeal (EL) speech suffers from severe acoustic mismatches with normal speech, resulting in degraded naturalness and intelligibility. These mismatches also make sequence-to-sequence voice conversion struggle to learn robust intermediate representations, hindering EL-speech-to-normal-speech conversion (EL2SP) performance. To address this, we propose a novel multimodal contrastive representation learning framework to 1) achieve richer multimodal-representation learning, and 2) facilitate the transferability of multimodal representations to speech-only EL2SP. First, EL speech and text representations are fused to learn multimodal representations for target speech generation, during which cross-modal contrastive learning is employed to encourage a semantically consistent speech--text representation space. Next, the fused representations are treated as a fixed target to guide speech-only representation learning through cross-stage contrastive learning, enabling multimodal knowledge to be transferred to the EL2SP task while preserving architectural and inference simplicity. Experimental results demonstrate that the proposed method consistently outperforms both the speech-only and multimodal-based baselines.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17716
PDF
-
Prior-Fitted and Frozen Scorers for Multimodal Contrastive Learning
2026-10-10 09:49:02Tillmann Rheude, Roland Eils, Benjamin WildAbstract
Multimodal contrastive learning typically relies on manually specified scoring functions, such as cosine similarity, to quantify the similarity between representations from different modalities. Several such functions have been recently proposed to extend contrastive learning beyond two modalities. In principle, however, any function that assigns high compatibility to matched samples and low compatibility to mismatched ones could serve as a scorer. Motivated by this broader view, we instantiate the scorer with a Transformer and find that even a randomly initialized model, when kept frozen, can outperform strong handcrafted multimodal contrastive baselines, whereas optimizing the scorer jointly with the encoders is less effective. Building on this observation, we introduce the Prior-Fitted Scorer (PFS): a Transformer scorer pre-trained on synthetic multimodal data generated from structural causal model (SCM) priors before transferring it to real-world data. The SCM exposes the scorer to diverse dependency structures, including shared, pairwise, and private information, providing a structured prior over higher-order cross-modal similarity. Across four real-world datasets, frozen Transformer scorers are competitive to contrastive baselines, and prior-fitting further improves the random initialization. More broadly, our results suggest that multimodal contrastive learning need not rely on manual similarity functions: Transformers can provide strong inductive biases, while causal pretraining can further shape these biases.
Abstract (translated)
URL
https://arxiv.org/abs/2610.14770
PDF
-
Co-adaptive Rule Routing and Constraining for Structured Knowledge Retrieval
2026-10-10 01:45:48Zhisheng Qi, Haoyu Han, Jiaxi Li, Bing He, Huiyuan Chen, Franck Dernoncourt, Ryan A. Rossi, Mahantesh M. Halappanavar, Jin Lu, Yu Zhang, Tianming Liu, Yu WangAbstract
Structured knowledge retrieval acquires task-relevant evidence from structured knowledge sources for downstream tasks. Existing methods rely on unconstrained retrieval or predefined query-rule supervision, such as author-paper rules for scientific discovery and anatomy-drug rules for precision medicine. Such supervision is rarely available in practice. We propose Co-Adaptive Rule Routing and Constraining Retrieval (CoRE), which jointly learns which reasoning rule to assign to each query and how to retrieve evidence under that rule, without predefined query-rule pairs. A routing agent selects a rule, and its corresponding retriever traverses the knowledge base and ranks evidence that satisfies the rule's constraints. Because routing and retrieval are interdependent, CoRE alternates between reinforcement learning for the routing policy and contrastive learning for the rule-constrained retriever. The selected rules provide supervision for retriever training, while retrieval performance rewards the routing policy. This closed-loop process enables both components to co-adapt without explicit query-rule supervision. Experiments on STaRK-QA datasets across scientific discovery, biomedical reasoning, and e-commerce knowledge management demonstrate CoRE's effectiveness. Ablations show the benefit of co-adapting the router and retriever. Generation-based routing transfers across datasets and benefits from multi-domain pretraining followed by target-specific fine-tuning. Analysis of rule usage across queries also informs automatic rule refinement and discovery.
Abstract (translated)
URL
https://arxiv.org/abs/2610.14465
PDF
-
Shortcut Learning in Geospatial Foundation Model Embeddings for Cross-Disaster Damage Assessment
2026-10-09 15:50:49Ali Shibli, Andrea Nascetti, Yifang BanAbstract
Geospatial foundation models can support disaster damage assessment, but high accuracy on familiar disasters may hide shortcut learning: prediction from event or scene characteristics rather than evidence that transfers to a new disaster. We study embeddings from TerraMind, Prithvi-EO-2.0, DOFA, and DINOv3 on the multi-disaster datasets xView2 and BRIGHT. Holding out an entire disaster, instead of randomly splitting image tiles, reduces mean balanced accuracy from 0.807 to 0.638 on xView2 and from 0.818 to 0.568 on BRIGHT. Four sets of tests examine what information supports prediction: event and acquisition context, before/after image inputs, building count, and correct image pairing. Pre-event images remain predictive, and a classifier given only the annotated number of buildings reaches balanced accuracy of 0.629 and 0.653. Combining before/after features performs best on average on xView2, but does not improve over pre-only features on BRIGHT. Shuffling image pairs within the same disaster and damage class causes modest losses. Motivated by these findings, we test whether feature transformations learned only from training disasters improve generalization to unseen disasters. These transformations improve performance, with supervised contrastive learning performing best on xView2 and PCA on BRIGHT. Our results highlight the importance of evaluation on unseen domains for assessing model robustness and show that simple feature-level transformations can improve generalization. Code and experiments can be found at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2610.13766
PDF
-
QCLR: A Plug-in Robustification Module via Quantization for Contrastive Learning in Self-Supervised Skeleton-Based Human Action Recognition
2026-10-09 09:30:15Nanzhu Chen, Feixiang He, Yunfeng Diao, Duygu Sarikaya, He WangAbstract
Skeleton-based human action recognition has gained increasing attention due to the stability of skeleton data across diverse environments and advances in motion capture technologies. Following the success of supervised learning approaches, self-supervised learning approaches, particularly contrastive learning ones, have achieved impressive performance. However, they remain vulnerable to adversarial attacks. In this work, we propose a plug-in robustification module via quantization for contrastive learning in self-supervised skeleton-based human action recognition (QCLR), which enhances the robustness of representations against adversarial attacks while preserving their representational capability. Specifically, we employ a dual-level quantization scheme across the encoding pipeline, which breaks the gradient propagation and absorbs small adversarial perturbations within the quantization bins. To mitigate the impact of inevitable quantization error, we further design an attention-guided multi-faceted quantization strategy to maintain the representation quality. Experiments on several benchmark datasets demonstrate that integrating QCLR into a progressive lineage of mainstream contrastive learning models substantially improves robustness against both gradient-based white-box and gradient-free black-box adversarial attacks, while preserving highly competitive classification accuracy.
Abstract (translated)
URL
https://arxiv.org/abs/2610.13281
PDF
-
ConClick: Element-Wise Contrastive Learning Improves GUI Grounding
2026-10-09 09:20:02Ruotian Zhang, Yuhui Zhang, Junzhe Huang, Yu Xin, Hehe Fan, Xiaoxiao SunAbstract
GUI grounding, the task of locating the interface element specified by a user instruction, is a fundamental capability of GUI agents. However, existing grounding models remain error-prone, often causing downstream task failures. We systematically evaluate 10 grounding models across five datasets and identify a major error pattern: most incorrect clicks hit non-target interface elements rather than the background, with different models repeatedly selecting the same distractors. Motivated by these findings, we propose ConClick, a contrastive learning framework that teaches models to distinguish the target element from plausible distractors. We construct ConClick-Data, a dataset containing 4,292 challenging target-distractor pairs, and train the model using complementary listwise ranking and pairwise margin objectives. ConClick improves the grounding accuracy of Qwen3-VL-8B from 54.71% to 66.03% on ScreenSpot-Pro, from 83.72% to 86.28% on MMBench-GUI L2, and from 63.33% to 70.20% on OSWorld-G, outperforming prior methods built on the same or similarly sized backbones. We further introduce ReClick, an inference-time strategy that revisits promising regions and compares predictions across competing candidates, yielding an additional 2.96 percentage-point improvement in average accuracy across the three benchmarks. Together, our analysis, dataset, and methods provide both empirical insights and practical tools for building more reliable GUI agents. Data and code are available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2610.13268
PDF
