Weakly_Supervised
-
PLESS: Pseudo-Label Enhancement with Spreading Scribbles for Weakly Supervised Segmentation
2026-02-12 06:24:05Yeva Gabrielyan (Akian College of Science and Engineering, American University of Armenia, Yerevan, Armenia), Varduhi Yeghiazaryan (Akian College of Science and Engineering, American University of Armenia, Yerevan, Armenia), Irina Voiculescu (Department of Computer Science, University of Oxford, Oxford, UK)Abstract
Weakly supervised learning with scribble annotations uses sparse user-drawn strokes to indicate segmentation labels on a small subset of pixels. This annotation reduces the cost of dense pixel-wise labeling, but suffers inherently from noisy and incomplete supervision. Recent scribble-based approaches in medical image segmentation address this limitation using pseudo-label-based training; however, the quality of the pseudo-labels remains a key performance limit. We propose PLESS, a generic pseudo-label enhancement strategy which improves reliability and spatial consistency. It builds on a hierarchical partitioning of the image into a hierarchy of spatially coherent regions. PLESS propagates scribble information to refine pseudo-labels within semantically coherent regions. The framework is model-agnostic and easily integrates into existing pseudo-label methods. Experiments on two public cardiac MRI datasets (ACDC and MSCMRseg) across four scribble-supervised algorithms show consistent improvements in segmentation accuracy. Code will be made available on GitHub upon acceptance.
Abstract (translated)
带有潦草标注的弱监督学习使用稀疏的手绘笔画来指示少量像素的分割标签，从而减少了密集像素级标记的成本。然而，这种注释本身存在噪声和不完整指导的问题。最近基于潦草标注的方法在医学图像分割中通过伪标签训练解决了这一限制；但是，伪标签的质量仍然是性能的关键瓶颈。 我们提出了PLESS（Pseudo-Label Enhancement Strategy for Scribble Annotations），这是一种通用的伪标签增强策略，旨在提高可靠性和空间一致性。该方法建立在一个将图像划分为一系列空间连贯区域的层次结构上，并利用潦草信息在语义一致区域内细化伪标签。PLESS框架与模型无关且易于整合到现有的伪标签方法中。 实验表明，在两个公开的心脏MRI数据集（ACDC和MSCMRseg）以及四种基于潦草标注的方法上，该策略均显著提高了分割精度。论文被接受后，代码将在GitHub上发布。
URL
https://arxiv.org/abs/2602.11628
PDF
-
ArtContext: Contextualizing Artworks with Open-Access Art History Articles and Wikidata Knowledge through a LoRA-Tuned CLIP Model
Abstract
Many Art History articles discuss artworks in general as well as specific parts of works, such as layout, iconography, or material culture. However, when viewing an artwork, it is not trivial to identify what different articles have said about the piece. Therefore, we propose ArtContext, a pipeline for taking a corpus of Open-Access Art History articles and Wikidata Knowledge and annotating Artworks with this information. We do this using a novel corpus collection pipeline, then learn a bespoke CLIP model adapted using Low-Rank Adaptation (LoRA) to make it domain-specific. We show that the new model, PaintingCLIP, which is weakly supervised by the collected corpus, outperforms CLIP and provides context for a given artwork. The proposed pipeline is generalisable and can be readily applied to numerous humanities areas.
Abstract (translated)
许多艺术史文章会讨论艺术品的总体情况以及作品的具体部分，如布局、图象学或物质文化。然而，在观看一件艺术品时，很难确定不同文章对该件作品的说法。因此，我们提出了ArtContext这一流程，它能够从一个开放获取的艺术史文献语料库和维基数据知识中提取信息，并将这些信息标注到艺术作品上。为此，我们使用了一种新型的语料收集流程，然后通过低秩适应（LoRA）技术对CLIP模型进行调整，使其具有特定领域的特性。我们展示了一个新模型PaintingCLIP，在该模型被收集的语料库弱监督下，其表现优于原始的CLIP模型，并能为给定的艺术作品提供上下文信息。所提出的流程是通用化的，可以轻松应用于多个人文领域。
URL
https://arxiv.org/abs/2602.11349
PDF
-
Enhancing Weakly Supervised Multimodal Video Anomaly Detection through Text Guidance
2026-02-11 05:44:30Shengyang Sun, Jiashen Hua, Junyi Feng, Xiaojin GongAbstract
Weakly supervised multimodal video anomaly detection has gained significant attention, yet the potential of the text modality remains under-explored. Text provides explicit semantic information that can enhance anomaly characterization and reduce false alarms. However, extracting effective text features is challenging due to the inability of general-purpose language models to capture anomaly-specific nuances and the scarcity of relevant descriptions. Furthermore, multimodal fusion often suffers from redundancy and imbalance. To address these issues, we propose a novel text-guided framework. First, we introduce an in-context learning-based multi-stage text augmentation mechanism to generate high-quality anomaly text samples for fine-tuning the text feature extractor. Second, we design a multi-scale bottleneck Transformer fusion module that uses compressed bottleneck tokens to progressively integrate information across modalities, mitigating redundancy and imbalance. Experiments on UCF-Crime and XD-Violence demonstrate state-of-the-art performance.
Abstract (translated)
弱监督多模态视频异常检测已经引起了广泛关注，然而文本模态的潜力仍然未被充分探索。文本提供了明确的语义信息，可以增强对异常行为的描述和减少误报率。但是，由于通用语言模型难以捕捉特定于异常的信息以及相关描述稀少的问题，提取有效文本特征颇具挑战性。此外，在多模态融合过程中经常会出现冗余和不平衡的问题。为了解决这些问题，我们提出了一种新颖的基于文本引导框架的方法。 首先，我们引入了一个基于上下文学习的多阶段文本增强机制，用于生成高质量的异常文本样本以对文本特征提取器进行微调。其次，我们设计了一种多尺度瓶颈Transformer融合模块，该模块使用压缩后的瓶颈令牌逐步整合不同模态的信息，从而缓解冗余和不平衡的问题。 在UCF-Crime和XD-Violence数据集上的实验结果表明了我们的方法达到了最先进的性能水平。
URL
https://arxiv.org/abs/2602.10549
PDF
-
Learning to Detect Baked Goods with Limited Supervision
2026-02-10 17:06:36Thomas H. Schmitt, Maximilian Bundscherer, Tobias BockletAbstract
Monitoring leftover products provides valuable insights that can be used to optimize future production. This is especially important for German bakeries because freshly baked goods have a very short shelf life. Automating this process can reduce labor costs, improve accuracy, and streamline operations. We propose automating this process using an object detection model to identify baked goods from images. However, the large diversity of German baked goods makes fully supervised training prohibitively expensive and limits scalability. Although open-vocabulary detectors (e.g., OWLv2, Grounding DINO) offer lexibility, we demonstrate that they are insufficient for our task. While motivated by bakeries, our work addresses the broader challenges of deploying computer vision in industries, where tasks are specialized and annotated datasets are scarce. We compile dataset splits with varying supervision levels, covering 19 classes of baked goods. We propose two training workflows to train an object detection model with limited supervision. First, we combine OWLv2 and Grounding DINO localization with image-level supervision to train the model in a weakly supervised manner. Second, we improve viewpoint robustness by fine-tuning on video frames annotated using Segment Anything 2 as a pseudo-label propagation model. Using these workflows, we train YOLOv11 for our detection task due to its favorable speed accuracy tradeoff. Relying solely on image-level supervision, the model achieves a mean Average Precision (mAP) of 0.91. Finetuning with pseudo-labels raises model performance by 19.3% under non-ideal deployment conditions. Combining these workflows trains a model that surpasses our fully-supervised baseline model under non-ideal deployment conditions, despite relying only on image-level supervision.
Abstract (translated)
监控剩余产品的状态可以提供有价值的见解，这些见解可用于优化未来的生产流程。对于德国的面包店来说，这一点尤为重要，因为新鲜烘焙的产品保质期非常短。自动化这一过程能够减少劳动力成本、提高准确性和简化操作流程。我们建议使用物体检测模型从图像中识别烘焙产品来实现这一目标。 然而，由于德国面包种类繁多，完全监督式的训练方法成本过高且不具备可扩展性。尽管开放词汇的检测器（如OWLv2和Grounding DINO）提供了灵活性，但我们证明它们对于我们的任务来说是不够的。虽然这项工作最初受到烘焙行业的启发，但它更广泛地解决了在行业中部署计算机视觉技术时面临的挑战：这些行业中的任务非常专业化且注释数据集稀缺。 为了应对这一问题，我们编制了不同监督水平的数据集拆分，涵盖了19类烘焙产品。为此，我们提出了两种有限监督训练物体检测模型的训练工作流程： 首先，我们将OWLv2和Grounding DINO定位与图像级监督相结合，以弱监督的方式训练模型。 其次，为了提高视角鲁棒性，我们在使用Segment Anything 2作为伪标签传播模型进行注释的视频帧上进行了微调。 利用这些工作流程，并考虑到YOLOv11在速度精度权衡方面的优势，我们针对检测任务对YOLOv11进行了训练。仅依赖图像级监督的情况下，该模型实现了0.91的平均精度（mAP）。通过使用伪标签进行微调，在非理想部署条件下使模型性能提高了19.3%。 结合这些工作流程，即使在只依靠图像级监督的情况下，也能训练出一个超过我们完全监督基准模型性能的模型。这种方法特别适用于解决那些标注数据稀缺、任务具有特殊性的行业的挑战。
URL
https://arxiv.org/abs/2602.09979
PDF
-
Weakly Supervised Contrastive Learning for Histopathology Patch Embeddings
2026-02-10 07:17:11Bodong Zhang, Xiwen Li, Hamid Manoochehri, Xiaoya Tang, Deepika Sirohi, Beatrice S. Knudsen, Tolga TasdizenAbstract
Digital histopathology whole slide images (WSIs) provide gigapixel-scale high-resolution images that are highly useful for disease diagnosis. However, digital histopathology image analysis faces significant challenges due to the limited training labels, since manually annotating specific regions or small patches cropped from large WSIs requires substantial time and effort. Weakly supervised multiple instance learning (MIL) offers a practical and efficient solution by requiring only bag-level (slide-level) labels, while each bag typically contains multiple instances (patches). Most MIL methods directly use frozen image patch features generated by various image encoders as inputs and primarily focus on feature aggregation. However, feature representation learning for encoder pretraining in MIL settings has largely been neglected. In our work, we propose a novel feature representation learning framework called weakly supervised contrastive learning (WeakSupCon) that incorporates bag-level label information during training. Our method does not rely on instance-level pseudo-labeling, yet it effectively separates patches with different labels in the feature space. Experimental results demonstrate that the image features generated by our WeakSupCon method lead to improved downstream MIL performance compared to self-supervised contrastive learning approaches in three datasets. Our related code is available at this http URL
Abstract (translated)
数字全滑病理图像（WSIs）提供了像素级的高分辨率图像，这对于疾病诊断非常有用。然而，由于标注训练数据量有限，数字病理图像分析面临着重大挑战，因为手动注释特定区域或从大型WSI中裁剪的小块需要大量的时间和精力。弱监督下的多实例学习（MIL）通过只需要袋子级别的标签提供了一种实用且高效的解决方案，而每个袋子通常包含多个实例（斑块）。大多数MIL方法直接使用由各种图像编码器生成的冻结图像斑块特征作为输入，并主要集中在特征聚合上。然而，在MIL环境中用于编码器预训练的特征表示学习被很大程度忽视了。 在我们的工作中，我们提出了一种名为弱监督对比学习（WeakSupCon）的新颖特征表示学习框架，该框架在训练过程中结合使用袋子级别的标签信息。我们的方法不依赖于实例级别伪标记，但在特征空间中有效地将具有不同标签的斑块分开。实验结果表明，在三个数据集中，我们提出的WeakSupCon方法生成的图像特征相比于自监督对比学习方法带来了更好的下游MIL性能改进。 相关代码可在此链接访问：[此URL]（请根据实际发布的地址替换）。
URL
https://arxiv.org/abs/2602.09477
PDF
-
GAFR-Net: A Graph Attention and Fuzzy-Rule Network for Interpretable Breast Cancer Image Classification
2026-02-10 01:25:57Lin-Guo Gao, Suxing LiuAbstract
Accurate classification of breast cancer histopathology images is pivotal for early oncological diagnosis and therapeutic this http URL, conventional deep learning architectures often encounter performance degradation under limited annotations and suffer from a "blackbox" nature, hindering their clinical integration. To mitigate these limitations, we propose GAFRNet, a robust and interpretable Graph Attention and FuzzyRule Network specifically engineered for histopathology image classification with scarce supervision. GAFRNet constructs a similarity-driven graph representation to model intersample relationships and employs a multihead graph attention mechanism to capture complex relational features across heterogeneous tissue this http URL, a differentiable fuzzy-rule module encodes intrinsic topological descriptorsincluding node degree, clustering coefficient, and label consistencyinto explicit, human-understandable diagnostic logic. This design establishes transparent "IF-THEN" mappings that mimic the heuristic deduction process of medical experts, providing clear reasoning behind each prediction without relying on post-hoc attribution methods. Extensive evaluations on three benchmark datasets (BreakHis, Mini-DDSM, and ICIAR2018) demonstrate that GAFR-Net consistently outperforms various state-of-the-art methods across multiple magnifications and classification tasks. These results validate the superior generalization and practical utility of GAFR-Net as a reliable decision-support tool for weakly supervised medical image analysis.
Abstract (translated)
乳腺癌组织病理图像的准确分类对于早期肿瘤诊断和治疗至关重要。然而，传统的深度学习架构在标注数据有限的情况下性能会下降，并且由于其“黑箱”特性而难以被临床应用。为了解决这些问题，我们提出了一种名为GAFRNet（Graph Attention and Fuzzy Rule Network）的强大且可解释的网络，专门用于稀缺监督条件下的组织病理图像分类。GAFRNet通过构建以相似性驱动的图表示来建模样本间的相互关系，并采用多头图注意力机制来捕捉异质组织中的复杂关联特征。此外，一个可微分模糊规则模块将内在拓扑描述符（包括节点度、聚类系数和标签一致性）编码为明确的人类易于理解的诊断逻辑。这种设计建立了透明的“如果-那么”映射关系，模拟了医疗专家基于直觉进行推断的过程，并且无需依赖事后归因方法就能提供清晰的推理依据。 在三个基准数据集（BreakHis、Mini-DDSM和ICIA2018）上的广泛评估表明，GAFRNet在多种放大倍率和分类任务中均超越了各种最新的方法。这些结果验证了GAFR-Net作为弱监督医学影像分析可靠决策支持工具的优越泛化能力和实用价值。 通过这种方式，GAFRNet不仅能够提高诊断准确性，还能增强医生对模型输出的理解与信任度，从而促进其在临床环境中的广泛采用。
URL
https://arxiv.org/abs/2602.09318
PDF
-
Tighnari v2: Mitigating Label Noise and Distribution Shift in Multimodal Plant Distribution Prediction via Mixture of Experts and Weakly Supervised Learning
2026-02-09 05:23:22Haixu Liu, Yufei Wang, Tianxiang Xu, Chuancheng Shi, Hongsheng XingAbstract
Large-scale, cross-species plant distribution prediction plays a crucial role in biodiversity conservation, yet modeling efforts in this area still face significant challenges due to the sparsity and bias of observational data. Presence-Absence (PA) data provide accurate and noise-free labels, but are costly to obtain and limited in quantity; Presence-Only (PO) data, by contrast, offer broad spatial coverage and rich spatiotemporal distribution, but suffer from severe label noise in negative samples. To address these real-world constraints, this paper proposes a multimodal fusion framework that fully leverages the strengths of both PA and PO data. We introduce an innovative pseudo-label aggregation strategy for PO data based on the geographic coverage of satellite imagery, enabling geographic alignment between the label space and remote sensing feature space. In terms of model architecture, we adopt Swin Transformer Base as the backbone for satellite imagery, utilize the TabM network for tabular feature extraction, retain the Temporal Swin Transformer for time-series modeling, and employ a stackable serial tri-modal cross-attention mechanism to optimize the fusion of heterogeneous modalities. Furthermore, empirical analysis reveals significant geographic distribution shifts between PA training and test samples, and models trained by directly mixing PO and PA data tend to experience performance degradation due to label noise in PO data. To address this, we draw on the mixture-of-experts paradigm: test samples are partitioned according to their spatial proximity to PA samples, and different models trained on distinct datasets are used for inference and post-processing within each partition. Experiments on the GeoLifeCLEF 2025 dataset demonstrate that our approach achieves superior predictive performance in scenarios with limited PA coverage and pronounced distribution shifts.
Abstract (translated)
大规模、跨物种的植物分布预测在生物多样性保护中起着关键作用，但在这一领域建模工作仍面临观测数据稀疏和偏差等重大挑战。存在-不存在（PA）数据提供准确且无噪声的标签，但获取成本高昂且数量有限；仅存在（PO）数据则提供了广泛的空间覆盖范围以及丰富的时间空间分布信息，但却因负样本中严重的标签噪声而受到影响。 为了解决这些实际限制，本文提出了一种多模态融合框架，该框架充分利用了PA和PO数据各自的优点。我们引入了一种基于卫星图像地理覆盖的创新伪标签聚合策略，从而实现标签空间与遥感特征空间之间的地理位置对齐。在模型架构方面，我们采用Swin Transformer Base作为卫星图像的基础网络，并使用TabM网络进行表格特征提取，保留时间序列建模中的Temporal Swin Transformer，同时利用可堆叠串行三模式交叉注意机制来优化异构模式的融合。 此外，实证分析显示PA训练和测试样本之间存在显著的地理分布偏移，直接混合PO和PA数据训练模型往往会因为PO数据中标签噪声而导致性能下降。为了应对这一问题，我们借鉴了专家混合范式：根据其与PA样本的空间接近程度对测试样本进行划分，并在每个分区使用不同数据集上训练的不同模型来进行推理和后处理。 通过GeoLifeCLEF 2025数据集的实验表明，我们的方法在PA覆盖范围有限且分布偏移显著的情况下达到了优越的预测性能。
URL
https://arxiv.org/abs/2602.08282
PDF
-
Weak to Strong: VLM-Based Pseudo-Labeling as a Weakly Supervised Training Strategy in Multimodal Video-based Hidden Emotion Understanding Tasks
2026-02-08 17:02:55Yufei Wang, Haixu Liu, Tianxiang Xu, Chuancheng Shi, Hongsheng XingAbstract
To tackle the automatic recognition of "concealed emotions" in videos, this paper proposes a multimodal weak-supervision framework and achieves state-of-the-art results on the iMiGUE tennis-interview dataset. First, YOLO 11x detects and crops human portraits frame-by-frame, and DINOv2-Base extracts visual features from the cropped regions. Next, by integrating Chain-of-Thought and Reflection prompting (CoT + Reflection), Gemini 2.5 Pro automatically generates pseudo-labels and reasoning texts that serve as weak supervision for downstream models. Subsequently, OpenPose produces 137-dimensional key-point sequences, augmented with inter-frame offset features; the usual graph neural network backbone is simplified to an MLP to efficiently model the spatiotemporal relationships of the three key-point streams. An ultra-long-sequence Transformer independently encodes both the image and key-point sequences, and their representations are concatenated with BERT-encoded interview transcripts. Each modality is first pre-trained in isolation, then fine-tuned jointly, with pseudo-labeled samples merged into the training set for further gains. Experiments demonstrate that, despite severe class imbalance, the proposed approach lifts accuracy from under 0.6 in prior work to over 0.69, establishing a new public benchmark. The study also validates that an "MLP-ified" key-point backbone can match - or even surpass - GCN-based counterparts in this task.
Abstract (translated)
为了应对视频中“隐蔽情绪”的自动识别问题，本文提出了一种多模态弱监督框架，并在iMiGUE网球访谈数据集上取得了最先进的成果。首先，YOLO 11x每帧检测并裁剪人物肖像，DINOv2-Base从裁剪区域提取视觉特征。接着，通过整合Chain-of-Thought和反思提示（CoT + Reflection），Gemini 2.5 Pro自动生成伪标签和推理文本，这些作为下游模型的弱监督信号。随后，OpenPose生成137维关键点序列，并加入帧间偏移特征；通常用于建模时空关系的关键点流图神经网络骨干被简化为MLP（多层感知机），以更高效地处理这一任务。一个超长序列Transformer独立编码图像和关键点序列，然后将它们的表示与BERT编码后的访谈记录拼接起来。每种模式首先单独预训练，之后共同微调，并将伪标签样本合并到训练集中以进一步提升性能。实验表明，在严重类不平衡的情况下，所提出的方法使准确性从先前研究中的不足0.6提高到了超过0.69的新水平，确立了一个新的公共基准标准。此外，该研究表明，“MLP化”的关键点骨干网络可以与基于GCN（图卷积网络）的同类方法相匹敌甚至超越。
URL
https://arxiv.org/abs/2602.08057
PDF
-
PANC: Prior-Aware Normalized Cut for Object Segmentation
2026-02-06 18:07:20Juan Guti\'errez, Victor Guti\'errez-Garcia, Jos\'e Luis Blanco-MurilloAbstract
Fully unsupervised segmentation pipelines naively seek the most salient object, should this be present. As a result, most of the methods reported in the literature deliver non-deterministic partitions that are sensitive to initialization, seed order, and threshold heuristics. We propose PANC, a weakly supervised spectral segmentation framework that uses a minimal set of annotated visual tokens to produce stable, controllable, and reproducible object masks. From the TokenCut approach, we augment the token-token affinity graph with a handful of priors coupled to anchor nodes. By manipulating the graph topology, we bias the spectral eigenspace toward partitions that are consistent with the annotations. Our approach preserves the global grouping enforced by dense self-supervised visual features, trading annotated tokens for significant gains in reproducibility, user control, and segmentation quality. Using 5 to 30 annotations per dataset, our training-free method achieves state-of-the-art performance among weakly and unsupervised approaches on standard benchmarks (e.g., DUTS-TE, ECSSD, MS COCO). Contrarily, it excels in domains where dense labels are costly or intra-class differences are subtle. We report strong and reliable results on homogeneous, fine-grained, and texture-limited domains, achieving 96.8% (+14.43% over SotA), 78.0% (+0.2%), and 78.8% (+0.37%) average mean intersection-over-union (mIoU) on CrackForest (CFD), CUB-200-2011, and HAM10000 datasets, respectively. For multi-object benchmarks, the framework showcases explicit, user-controllable semantic segmentation.
Abstract (translated)
完全无监督的分割流程通常会寻找最显著的对象，前提是该对象存在。因此，文献中报告的大多数方法提供了非确定性的分区结果，这些结果对初始化、种子顺序和阈值启发式非常敏感。我们提出了一种弱监督光谱分割框架PANC，它使用一组最小化的注释视觉标记来生成稳定、可控且可重复的对象掩码。从TokenCut方法出发，我们在令牌-令牌亲和图中添加了一些与锚节点耦合的先验知识。通过调整图的拓扑结构，我们偏向于让光谱特征空间倾向于那些与标注一致的分区结果。我们的方法保持了由密集自监督视觉特征强制执行的整体分组效果，并以注释标记为代价换取可重复性、用户控制和分割质量方面的显著提升。 在标准基准测试（如DUTS-TE、ECSSD、MS COCO）上，使用每个数据集5到30个标注的情况下，我们的无训练方法在弱监督与完全无监督的方法中达到了最佳性能。相反，在密集标签成本高或类内差异微妙的领域，该方法表现出色。我们在同质性、细粒度和纹理有限的领域报告了强大且可靠的结果，分别实现了CrackForest (CFD)上96.8%（比最先进方法提高14.43%），CUB-200-2011上78.0%（提高0.2%）以及HAM10000数据集上的78.8%（提高0.37%）的平均交并比（mIoU）。对于多对象基准测试，该框架展示了用户可控的语义分割。
URL
https://arxiv.org/abs/2602.06912
PDF
-
Towards Segmenting the Invisible: An End-to-End Registration and Segmentation Framework for Weakly Supervised Tumour Analysis
2026-02-05 08:55:26Budhaditya Mukhopadhyay, Chirag Mandal, Pavan Tummala, Naghmeh Mahmoodian, Andreas N\"urnberger, Soumick ChatterjeeAbstract
Liver tumour ablation presents a significant clinical challenge: whilst tumours are clearly visible on pre-operative MRI, they are often effectively invisible on intra-operative CT due to minimal contrast between pathological and healthy tissue. This work investigates the feasibility of cross-modality weak supervision for scenarios where pathology is visible in one modality (MRI) but absent in another (CT). We present a hybrid registration-segmentation framework that combines MSCGUNet for inter-modal image registration with a UNet-based segmentation module, enabling registration-assisted pseudo-label generation for CT images. Our evaluation on the CHAOS dataset demonstrates that the pipeline can successfully register and segment healthy liver anatomy, achieving a Dice score of 0.72. However, when applied to clinical data containing tumours, performance degrades substantially (Dice score of 0.16), revealing the fundamental limitations of current registration methods when the target pathology lacks corresponding visual features in the target modality. We analyse the "domain gap" and "feature absence" problems, demonstrating that whilst spatial propagation of labels via registration is feasible for visible structures, segmenting truly invisible pathology remains an open challenge. Our findings highlight that registration-based label transfer cannot compensate for the absence of discriminative features in the target modality, providing important insights for future research in cross-modality medical image analysis. Code an weights are available at: this https URL
Abstract (translated)
肝脏肿瘤消融在临床上是一个重大的挑战：尽管术前MRI图像上可以清晰地看到肿瘤，但在术中CT图像上由于病理组织和健康组织之间的对比度极低，这些肿瘤往往变得几乎不可见。这项研究探讨了跨模式弱监督方法的可行性，在这种方法中，一种成像方式（如MRI）能显示病理性结构，而另一种成像方式（如CT）则无法显示。我们提出了一种结合MSCGUNet进行跨模态图像配准与基于UNet的分割模块的混合框架，该框架能够生成注册辅助伪标签以供CT图像使用。 我们在CHAOS数据集上的评估表明，该流程可以成功地对健康肝脏解剖结构进行配准和分割，达到Dice分数为0.72的成绩。然而，在包含肿瘤的实际临床数据中应用时，性能显著下降（Dice分数为0.16），这揭示了现有注册方法在目标病理性特征缺乏相应视觉特性时的固有限制。 我们分析了“领域差距”和“特征缺失”的问题，表明尽管通过配准可以在可视结构间传播标签，但分割真正不可见的病理仍然是一个开放性的挑战。我们的研究结果强调，基于注册的标签转移不能弥补目标模态中缺乏区分性特征的问题，并为未来跨模式医学图像分析的研究提供了重要的见解。 代码和权重可在以下网址获得：[链接]（此处应插入具体URL）
URL
https://arxiv.org/abs/2602.05453
PDF
-
A Domain-Specific Curated Benchmark for Entity and Document-Level Relation Extraction
2026-02-04 08:38:55Marco Martinelli, Stefano Marchesin, Vanessa Bonato, Giorgio Maria Di Nunzio, Nicola Ferro, Ornella Irrera, Laura Menotti, Federica Vezzani, Gianmaria SilvelloAbstract
Information Extraction (IE), encompassing Named Entity Recognition (NER), Named Entity Linking (NEL), and Relation Extraction (RE), is critical for transforming the rapidly growing volume of scientific publications into structured, actionable knowledge. This need is especially evident in fast-evolving biomedical fields such as the gut-brain axis, where research investigates complex interactions between the gut microbiota and brain-related disorders. Existing biomedical IE benchmarks, however, are often narrow in scope and rely heavily on distantly supervised or automatically generated annotations, limiting their utility for advancing robust IE methods. We introduce GutBrainIE, a benchmark based on more than 1,600 PubMed abstracts, manually annotated by biomedical and terminological experts with fine-grained entities, concept-level links, and relations. While grounded in the gut-brain axis, the benchmark's rich schema, multiple tasks, and combination of highly curated and weakly supervised data make it broadly applicable to the development and evaluation of biomedical IE systems across domains.
Abstract (translated)
信息抽取（IE）包括命名实体识别（NER）、命名实体链接（NEL）和关系提取（RE），对于将快速增长的科学出版物转化为结构化、可操作的知识至关重要。特别是在快速发展的生物医学领域，如肠-脑轴，研究复杂地探讨了肠道微生物群与大脑相关疾病之间的相互作用，这种需求尤为明显。然而，现有的生物医学IE基准测试通常范围狭窄，并且严重依赖于远程监督或自动产生的注释，这限制了它们在推进稳健IE方法方面的实用性。 我们引入了GutBrainIE，这是一个基于超过1,600篇PubMed摘要的基准测试，这些摘要由生物医学和术语学专家手动标注，包含了细粒度实体、概念级链接以及关系。虽然该基准测试建立在肠-脑轴的基础上，但其丰富的架构、多项任务设置及高度策划与弱监督数据的结合，使其能够广泛应用于跨领域生物医学IE系统的开发与评估。
URL
https://arxiv.org/abs/2602.04320
PDF
-
SPWOOD: Sparse Partial Weakly-Supervised Oriented Object Detection
2026-02-03 15:21:01Wei Zhang, Xiang Liu, Ningjing Liu, Mingxin Liu, Wei Liao, Chunyan Xu, Xue YangAbstract
A consistent trend throughout the research of oriented object detection has been the pursuit of maintaining comparable performance with fewer and weaker annotations. This is particularly crucial in the remote sensing domain, where the dense object distribution and a wide variety of categories contribute to prohibitively high costs. Based on the supervision level, existing oriented object detection algorithms can be broadly grouped into fully supervised, semi-supervised, and weakly supervised methods. Within the scope of this work, we further categorize them to include sparsely supervised and partially weakly-supervised methods. To address the challenges of large-scale labeling, we introduce the first Sparse Partial Weakly-Supervised Oriented Object Detection framework, designed to efficiently leverage only a few sparse weakly-labeled data and plenty of unlabeled data. Our framework incorporates three key innovations: (1) We design a Sparse-annotation-Orientation-and-Scale-aware Student (SOS-Student) model to separate unlabeled objects from the background in a sparsely-labeled setting, and learn orientation and scale information from orientation-agnostic or scale-agnostic weak annotations. (2) We construct a novel Multi-level Pseudo-label Filtering strategy that leverages the distribution of model predictions, which is informed by the model's multi-layer predictions. (3) We propose a unique sparse partitioning approach, ensuring equal treatment for each category. Extensive experiments on the DOTA and DIOR datasets show that our framework achieves a significant performance gain over traditional oriented object detection methods mentioned above, offering a highly cost-effective solution. Our code is publicly available at this https URL.
Abstract (translated)
在整个面向对象检测的研究中，一个持续的趋势是追求在使用较少和较弱标注的情况下保持相当的性能。这在遥感领域尤为重要，因为密集的对象分布和众多类别导致了高昂的成本。根据监督水平，现有的面向对象检测算法可以大致分为全监督、半监督和弱监督方法。在此工作中，我们进一步将它们分类为稀疏监督和部分弱监督方法。为了应对大规模标注的挑战，我们引入了第一个稀疏部分弱监督面向对象检测框架，旨在仅利用少量稀疏弱标记数据和大量未标记数据来高效运行。 我们的框架包含三个关键创新： 1. 我们设计了一种稀疏注释方向尺度感知的学生模型（SOS-学生模型），在稀疏标注的情况下将未标注的对象从背景中分离出来，并且能够从无方向性或无尺度性的弱标签中学习到方向和尺度信息。 2. 我们构建了一个新颖的多级伪标签过滤策略，该策略利用了模型预测分布的信息，这些信息来源于模型多层次的预测结果。 3. 我们提出了一种独特的稀疏分区方法，确保每个类别都能得到平等对待。 在DOTA和DIOR数据集上的大量实验表明，我们的框架比上述传统的面向对象检测方法获得了显著的性能提升，并提供了高度成本效益的解决方案。我们公开提供代码：[此链接]（请在此处插入实际的URL）。
URL
https://arxiv.org/abs/2602.03634
PDF
-
Finding Optimal Video Moment without Training: Gaussian Boundary Optimization for Weakly Supervised Video Grounding
2026-02-03 04:01:12Sunoh Kim, Kimin Yun, Daeho UmAbstract
Weakly supervised temporal video grounding aims to localize query-relevant segments in untrimmed videos using only video-sentence pairs, without requiring ground-truth segment annotations that specify exact temporal boundaries. Recent approaches tackle this task by utilizing Gaussian-based temporal proposals to represent query-relevant segments. However, their inference strategies rely on heuristic mappings from Gaussian parameters to segment boundaries, resulting in suboptimal localization performance. To address this issue, we propose Gaussian Boundary Optimization (GBO), a novel inference framework that predicts segment boundaries by solving a principled optimization problem that balances proposal coverage and segment compactness. We derive a closed-form solution for this problem and rigorously analyze the optimality conditions under varying penalty regimes. Beyond its theoretical foundations, GBO offers several practical advantages: it is training-free and compatible with both single-Gaussian and mixture-based proposal architectures. Our experiments show that GBO significantly improves localization, achieving state-of-the-art results across standard benchmarks. Extensive experiments demonstrate the efficiency and generalizability of GBO across various proposal schemes. The code is available at \href{this https URL}{this https URL}.
Abstract (translated)
弱监督视频时间定位的目标是在未裁剪的视频中仅使用视频-句子对来定位与查询相关的片段，而不需依赖明确指定具体时间边界的地面真实标注。最近的研究方法通过利用基于高斯分布的时间提案来表示与查询相关的片段。然而，这些方法的推理策略依赖于从高斯参数到段落边界之间的启发式映射，这导致了次优的时间定位性能。为了解决这个问题，我们提出了高斯边界优化（GBO），这是一种新的推断框架，它通过解决一个平衡提案覆盖范围和片段紧实度的原则性优化问题来预测片段边界。我们为该问题导出了闭合形式的解决方案，并严格分析了在不同惩罚制度下的最优条件。 除了其理论基础外，GBO还提供了许多实用优势：它是无须训练且与单个高斯和混合型提案架构兼容的。我们的实验表明，GBO显著提高了定位性能，在标准基准测试中取得了最先进的结果。广泛的实验证明了GBO在各种提案方案中的效率和泛化能力。代码可在[此处](this https URL)获取。 原文的意思是介绍了一种新的弱监督视频时间定位方法——高斯边界优化（Gaussian Boundary Optimization, GBO），这种方法通过解决一个平衡提案覆盖范围与片段紧实度的原则性优化问题，提高了定位的准确性，并且无须额外训练。这项技术在标准基准测试中取得了领先的成绩，并展示了其在多种提案方案中的高效性和泛化能力。
URL
https://arxiv.org/abs/2602.03071
PDF
-
CIEC: Coupling Implicit and Explicit Cues for Multimodal Weakly Supervised Manipulation Localization
2026-02-02 14:46:38Xinquan Yu, Wei Lu, Xiangyang LuoAbstract
To mitigate the threat of misinformation, multimodal manipulation localization has garnered growing attention. Consider that current methods rely on costly and time-consuming fine-grained annotations, such as patch/token-level annotations. This paper proposes a novel framework named Coupling Implicit and Explicit Cues (CIEC), which aims to achieve multimodal weakly-supervised manipulation localization for image-text pairs utilizing only coarse-grained image/sentence-level annotations. It comprises two branches, image-based and text-based weakly-supervised localization. For the former, we devise the Textual-guidance Refine Patch Selection (TRPS) module. It integrates forgery cues from both visual and textual perspectives to lock onto suspicious regions aided by spatial priors. Followed by the background silencing and spatial contrast constraints to suppress interference from irrelevant areas. For the latter, we devise the Visual-deviation Calibrated Token Grounding (VCTG) module. It focuses on meaningful content words and leverages relative visual bias to assist token localization. Followed by the asymmetric sparse and semantic consistency constraints to mitigate label noise and ensure reliability. Extensive experiments demonstrate the effectiveness of our CIEC, yielding results comparable to fully supervised methods on several evaluation metrics.
Abstract (translated)
URL
https://arxiv.org/abs/2602.02175
PDF
-
WS-IMUBench: Can Weakly Supervised Methods from Audio, Image, and Video Be Adapted for IMU-based Temporal Action Localization?
2026-02-02 09:22:35Pei Li, Jiaxi Yin, Lei Ouyang, Shihan Pan, Ge Wang, Han Ding, Fei WangAbstract
IMU-based Human Activity Recognition (HAR) has enabled a wide range of ubiquitous computing applications, yet its dominant clip classification paradigm cannot capture the rich temporal structure of real-world behaviors. This motivates a shift toward IMU Temporal Action Localization (IMU-TAL), which predicts both action categories and their start/end times in continuous streams. However, current progress is strongly bottlenecked by the need for dense, frame-level boundary annotations, which are costly and difficult to scale. To address this bottleneck, we introduce WS-IMUBench, a systematic benchmark study of weakly supervised IMU-TAL (WS-IMU-TAL) under only sequence-level labels. Rather than proposing a new localization algorithm, we evaluate how well established weakly supervised localization paradigms from audio, image, and video transfer to IMU-TAL under only sequence-level labels. We benchmark seven representative weakly supervised methods on seven public IMU datasets, resulting in over 3,540 model training runs and 7,080 inference evaluations. Guided by three research questions on transferability, effectiveness, and insights, our findings show that (i) transfer is modality-dependent, with temporal-domain methods generally more stable than image-derived proposal-based approaches; (ii) weak supervision can be competitive on favorable datasets (e.g., with longer actions and higher-dimensional sensing); and (iii) dominant failure modes arise from short actions, temporal ambiguity, and proposal quality. Finally, we outline concrete directions for advancing WS-IMU-TAL (e.g., IMU-specific proposal generation, boundary-aware objectives, and stronger temporal reasoning). Beyond individual results, WS-IMUBench establishes a reproducible benchmarking template, datasets, protocols, and analyses, to accelerate community-wide progress toward scalable WS-IMU-TAL.
Abstract (translated)
URL
https://arxiv.org/abs/2602.01850
PDF
-
Offline Discovery of Interpretable Skills from Multi-Task Trajectories
2026-02-01 05:03:58Chongyu Zhu, Mithun Vanniasinghe, Jiayu Chen, Chi-Guhn LeeAbstract
Hierarchical Imitation Learning is a powerful paradigm for acquiring complex robot behaviors from demonstrations. A central challenge, however, lies in discovering reusable skills from long-horizon, multi-task offline data, especially when the data lacks explicit rewards or subtask annotations. In this work, we introduce LOKI, a three-stage end-to-end learning framework designed for offline skill discovery and hierarchical imitation. The framework commences with a two-stage, weakly supervised skill discovery process: Stage one performs coarse, task-aware macro-segmentation by employing an alignment-enforced Vector Quantized VAE guided by weak task labels. Stage two then refines these segments at a micro-level using a self-supervised sequential model, followed by an iterative clustering process to consolidate skill boundaries. The third stage then leverages these precise boundaries to construct a hierarchical policy within an option-based framework-complete with a learned termination condition beta for explicit skill switching. LOKI achieves high success rates on the challenging D4RL Kitchen benchmark and outperforms standard HIL baselines. Furthermore, we demonstrate that the discovered skills are semantically meaningful, aligning with human intuition, and exhibit compositionality by successfully sequencing them to solve a novel, unseen task.
Abstract (translated)
URL
https://arxiv.org/abs/2602.01018
PDF
-
Fire on Motion: Optimizing Video Pass-bands for Efficient Spiking Action Recognition
2026-01-30 07:47:23Shuhan Ye, Yuanbin Qian, Yi Yu, Chong Wang, Yuqi Xie, Jiazhen Xu, Kun Wang, Xudong JiangAbstract
Spiking neural networks (SNNs) have gained traction in vision due to their energy efficiency, bio-plausibility, and inherent temporal processing. Yet, despite this temporal capacity, most progress concentrates on static image benchmarks, and SNNs still underperform on dynamic video tasks compared to artificial neural networks (ANNs). In this work, we diagnose a fundamental pass-band mismatch: Standard spiking dynamics behave as a temporal low pass that emphasizes static content while attenuating motion bearing bands, where task relevant information concentrates in dynamic tasks. This phenomenon explains why SNNs can approach ANNs on static tasks yet fall behind on tasks that demand richer temporal this http URL remedy this, we propose the Pass-Bands Optimizer (PBO), a plug-and-play module that optimizes the temporal pass-band toward task-relevant motion bands. PBO introduces only two learnable parameters, and a lightweight consistency constraint that preserves semantics and boundaries, incurring negligible computational overhead and requires no architectural changes. PBO deliberately suppresses static components that contribute little to discrimination, effectively high passing the stream so that spiking activity concentrates on motion bearing content. On UCF101, PBO yields over ten percentage points improvement. On more complex multi-modal action recognition and weakly supervised video anomaly detection, PBO delivers consistent and significant gains, offering a new perspective for SNN based video processing and understanding.
Abstract (translated)
URL
https://arxiv.org/abs/2601.22675
PDF
-
Mining Forgery Traces from Reconstruction Error: A Weakly Supervised Framework for Multimodal Deepfake Temporal Localization
2026-01-29 09:35:27Midou Guo, Qilin Yin, Wei Lu, Xiangyang Luo, Rui YangAbstract
Modern deepfakes have evolved into localized and intermittent manipulations that require fine-grained temporal localization. The prohibitive cost of frame-level annotation makes weakly supervised methods a practical necessity, which rely only on video-level labels. To this end, we propose Reconstruction-based Temporal Deepfake Localization (RT-DeepLoc), a weakly supervised temporal forgery localization framework that identifies forgeries via reconstruction errors. Our framework uses a Masked Autoencoder (MAE) trained exclusively on authentic data to learn its intrinsic spatiotemporal patterns; this allows the model to produce significant reconstruction discrepancies for forged segments, effectively providing the missing fine-grained cues for localization. To robustly leverage these indicators, we introduce a novel Asymmetric Intra-video Contrastive Loss (AICL). By focusing on the compactness of authentic features guided by these reconstruction cues, AICL establishes a stable decision boundary that enhances local discrimination while preserving generalization to unseen forgeries. Extensive experiments on large-scale datasets, including LAV-DF, demonstrate that RT-DeepLoc achieves state-of-the-art performance in weakly-supervised temporal forgery localization.
Abstract (translated)
URL
https://arxiv.org/abs/2601.21458
PDF
-
Learning From a Steady Hand: A Weakly Supervised Agent for Robot Assistance under Microscopy
2026-01-28 17:03:43Huanyu Tian, Martin Huber, Lingyun Zeng, Zhe Han, Wayne Bennett, Giuseppe Silvestri, Gerardo Mendizabal-Ruiz, Tom Vercauteren, Alejandro Chavez-Badiola, Christos BergelesAbstract
This paper rethinks steady-hand robotic manipulation by using a weakly supervised framework that fuses calibration-aware perception with admittance control. Unlike conventional automation that relies on labor-intensive 2D labeling, our framework leverages reusable warm-up trajectories to extract implicit spatial information, thereby achieving calibration-aware, depth-resolved perception without the need for external fiducials or manual depth annotation. By explicitly characterizing residuals from observation and calibration models, the system establishes a task-space error budget from recorded warm-ups. The uncertainty budget yields a lateral closed-loop accuracy of approx. 49 micrometers at 95% confidence (worst-case testing subset) and a depth accuracy of <= 291 micrometers at 95% confidence bound during large in-plane moves. In a within-subject user study (N=8), the learned agent reduces overall NASA-TLX workload by 77.1% relative to the simple steady-hand assistance baseline. These results demonstrate that the weakly supervised agent improves the reliability of microscope-guided biomedical micromanipulation without introducing complex setup requirements, offering a practical framework for microscope-guided intervention.
Abstract (translated)
URL
https://arxiv.org/abs/2601.20776
PDF
-
Weakly supervised framework for wildlife detection and counting in challenging Arctic environments: a case study on caribou
2026-01-26 19:02:18Ghazaleh Serati, Samuel Foucher, Jerome TheauAbstract
Caribou across the Arctic has declined in recent decades, motivating scalable and accurate monitoring approaches to guide evidence-based conservation actions and policy decisions. Manual interpretation from this imagery is labor-intensive and error-prone, underscoring the need for automatic and reliable detection across varying scenes. Yet, such automatic detection is challenging due to severe background heterogeneity, dominant empty terrain (class imbalance), small or occluded targets, and wide variation in density and scale. To make the detection model (HerdNet) more robust to these challenges, a weakly supervised patch-level pretraining based on a detection network's architecture is proposed. The detection dataset includes five caribou herds distributed across Alaska. By learning from empty vs. non-empty labels in this dataset, the approach produces early weakly supervised knowledge for enhanced detection compared to HerdNet, which is initialized from generic weights. Accordingly, the patch-based pretrain network attained high accuracy on multi-herd imagery (2017) and on an independent year's (2019) test sets (F1: 93.7%/92.6%, respectively), enabling reliable mapping of regions containing animals to facilitate manual counting on large aerial imagery. Transferred to detection, initialization from weakly supervised pretraining yielded consistent gains over ImageNet weights on both positive patches (F1: 92.6%/93.5% vs. 89.3%/88.6%), and full-image counting (F1: 95.5%/93.3% vs. 91.5%/90.4%). Remaining limitations are false positives from animal-like background clutter and false negatives related to low animal density occlusions. Overall, pretraining on coarse labels prior to detection makes it possible to rely on weakly-supervised pretrained weights even when labeled data are limited, achieving results comparable to generic-weight initialization.
Abstract (translated)
URL
https://arxiv.org/abs/2601.18891
PDF
