Most existing sign language translation (SLT) datasets are limited in scale, lack multilingual coverage, and are costly to curate due to their reliance on expert annotation and controlled recording setup. Recently, Vision Language Models (VLMs) have demonstrated strong capabilities as evaluators and real-time assistants. Despite these advancements, their potential remains untapped in the context of sign language dataset acquisition. To bridge this gap, we introduce the first automated annotation and filtering framework that utilizes VLMs to reduce reliance on manual effort while preserving data quality. Our method is applied to TikTok videos across eight sign languages and to the already curated YouTube-SL-25 dataset in German Sign Language for the purpose of additional evaluation. Our VLM-based pipeline includes a face visibility detection, a sign activity recognition, a text extraction from video content, and a judgment step to validate alignment between video and text, implementing generic filtering, annotation and validation steps. Using the resulting corpus, TikTok-SL-8, we assess the performance of two off-the-shelf SLT models on our filtered dataset for German and American Sign Languages, with the goal of establishing baselines and evaluating the robustness of recent models on automatically extracted, slightly noisy data. Our work enables scalable, weakly supervised pretraining for SLT and facilitates data acquisition from social media.
https://arxiv.org/abs/2510.25413
Class Activation Mapping (CAM) methods are widely applied in weakly supervised learning tasks due to their ability to highlight object regions. However, conventional CAM methods highlight only the most discriminative regions of the target. These highlighted regions often fail to cover the entire object and are frequently misaligned with object boundaries, thereby limiting the performance of downstream weakly supervised learning tasks, particularly Weakly Supervised Semantic Segmentation (WSSS), which demands pixel-wise accurate activation maps to get the best results. To alleviate the above problems, we propose a novel activation method, Region-CAM. Distinct from network feature weighting approaches, Region-CAM generates activation maps by extracting semantic information maps (SIMs) and performing semantic information propagation (SIP) by considering both gradients and features in each of the stages of the baseline classification model. Our approach highlights a greater proportion of object regions while ensuring activation maps to have precise boundaries that align closely with object edges. Region-CAM achieves 60.12% and 58.43% mean intersection over union (mIoU) using the baseline model on the PASCAL VOC training and validation datasets, respectively, which are improvements of 13.61% and 13.13% over the original CAM (46.51% and 45.30%). On the MS COCO validation set, Region-CAM achieves 36.38%, a 16.23% improvement over the original CAM (20.15%). We also demonstrate the superiority of Region-CAM in object localization tasks, using the ILSVRC2012 validation set. Region-CAM achieves 51.7% in Top-1 Localization accuracy Loc1. Compared with LayerCAM, an activation method designed for weakly supervised object localization, Region-CAM achieves 4.5% better performance in Loc1.
https://arxiv.org/abs/2510.25134
Fact-checking numerical claims is critical as the presence of numbers provide mirage of veracity despite being fake potentially causing catastrophic impacts on society. The prior works in automatic fact verification do not primarily focus on natural numerical claims. A typical human fact-checker first retrieves relevant evidence addressing the different numerical aspects of the claim and then reasons about them to predict the veracity of the claim. Hence, the search process of a human fact-checker is a crucial skill that forms the foundation of the verification process. Emulating a real-world setting is essential to aid in the development of automated methods that encompass such skills. However, existing benchmarks employ heuristic claim decomposition approaches augmented with weakly supervised web search to collect evidences for verifying claims. This sometimes results in less relevant evidences and noisy sources with temporal leakage rendering a less realistic retrieval setting for claim verification. Hence, we introduce QuanTemp++: a dataset consisting of natural numerical claims, an open domain corpus, with the corresponding relevant evidence for each claim. The evidences are collected through a claim decomposition process approximately emulating the approach of human fact-checker and veracity labels ensuring there is no temporal leakage. Given this dataset, we also characterize the retrieval performance of key claim decomposition paradigms. Finally, we observe their effect on the outcome of the verification pipeline and draw insights. The code for data pipeline along with link to data can be found at this https URL
https://arxiv.org/abs/2510.22055
Video Anomaly Detection (VAD) aims to locate unusual activities or behaviors within videos. Recently, offline VAD has garnered substantial research attention, which has been invigorated by the progress in large language models (LLMs) and vision-language models (VLMs), offering the potential for a more nuanced understanding of anomalies. However, online VAD has seldom received attention due to real-time constraints and computational intensity. In this paper, we introduce a novel Memory-based online scoring queue scheme for Training-free VAD (MoniTor), to address the inherent complexities in online VAD. Specifically, MoniTor applies a streaming input to VLMs, leveraging the capabilities of pre-trained large-scale models. To capture temporal dependencies more effectively, we incorporate a novel prediction mechanism inspired by Long Short-Term Memory (LSTM) networks. This ensures the model can effectively model past states and leverage previous predictions to identify anomalous behaviors. Thereby, it better understands the current frame. Moreover, we design a scoring queue and an anomaly prior to dynamically store recent scores and cover all anomalies in the monitoring scenario, providing guidance for LLMs to distinguish between normal and abnormal behaviors over time. We evaluate MoniTor on two large datasets (i.e., UCF-Crime and XD-Violence) containing various surveillance and real-world scenarios. The results demonstrate that MoniTor outperforms state-of-the-art methods and is competitive with weakly supervised methods without training. Code is available at this https URL.
https://arxiv.org/abs/2510.21449
Weakly supervised learning often operates with coarse aggregate signals rather than instance labels. We study a setting where each training example is an $n$-tuple containing exactly m positives, while only the count m per tuple is observed. This NTMP (N-tuple with M positives) supervision arises in, e.g., image classification with region proposals and multi-instance measurements. We show that tuple counts admit a trainable unbiased risk estimator (URE) by linking the tuple-generation process to latent instance marginals. Starting from fixed (n,m), we derive a closed-form URE and extend it to variable tuple sizes, variable counts, and their combination. Identification holds whenever the effective mixing rate is separated from the class prior. We establish generalization bounds via Rademacher complexity and prove statistical consistency with standard rates under mild regularity assumptions. To improve finite-sample stability, we introduce simple ReLU corrections to the URE that preserve asymptotic correctness. Across benchmarks converted to NTMP tasks, the approach consistently outperforms representative weak-supervision baselines and yields favorable precision-recall and F1 trade-offs. It remains robust under class-prior imbalance and across diverse tuple configurations, demonstrating that count-only supervision can be exploited effectively through a theoretically grounded and practically stable objective.
弱监督学习通常处理粗略的聚合信号,而不是实例标签。我们研究了一种设置,在这种设置中,每个训练样本是一个包含恰好m个正例的n元组,而我们只观察到每组中的计数m。这种NTMP(N-tuple with M positives)监督出现在例如带有区域提案和多实例测量的图像分类场景中。我们展示了元组计数可以提供一个可训练的无偏风险估计器(URE),通过将元组生成过程与潜在实例边缘联系起来。从固定的(n, m)开始,我们推导出一种封闭形式的URE,并将其扩展到变量大小、变量数量以及它们的组合。当有效的混合速率与类先验分离时,即可进行识别。我们通过Rademacher复杂度建立泛化界限,并在适度正则性假设下证明统计一致性及标准速度。为了提高有限样本稳定性,我们在URE中引入了简单的ReLU修正,以保持渐近正确性。在转换为NTMP任务的基准测试上,该方法始终优于代表性弱监督基线,在精度-召回率和F1指标权衡方面表现良好,并且即使在类先验不平衡的情况下以及面对各种元组配置时也能保持稳健性。这表明仅通过计数监督就可以有效地利用一个理论基础坚实、实践稳定性好的目标函数。
https://arxiv.org/abs/2510.18406
Salient object detection (SOD) aims to segment visually prominent regions in images and serves as a foundational task for various computer vision applications. We posit that SOD can now reach near-supervised accuracy without a single pixel-level label, but only when reliable pseudo-masks are available. We revisit the prototype-based line of work and make two key observations. First, boundary pixels and interior pixels obey markedly different geometry; second, the global consistency enforced by optimal transport (OT) is underutilized if prototype quality is weak. To address this, we introduce POTNet, an adaptation of Prototypical Optimal Transport that replaces POT's single k-means step with an entropy-guided dual-clustering head: high-entropy pixels are organized by spectral clustering, low-entropy pixels by k-means, and the two prototype sets are subsequently aligned by OT. This split-fuse-transport design yields sharper, part-aware pseudo-masks in a single forward pass, without handcrafted priors. Those masks supervise a standard MaskFormer-style encoder-decoder, giving rise to AutoSOD, an end-to-end unsupervised SOD pipeline that eliminates SelfMask's offline voting yet improves both accuracy and training efficiency. Extensive experiments on five benchmarks show that AutoSOD outperforms unsupervised methods by up to 26% and weakly supervised methods by up to 36% in F-measure, further narrowing the gap to fully supervised models.
显著对象检测(SOD)旨在分割图像中视觉上突出的区域,并作为各种计算机视觉应用的基础任务。我们提出,现在可以通过利用可靠的伪掩码而不需要单个像素级别的标签来实现接近监督精度的SOD。我们重新审视基于原型的工作,并提出了两个关键观察结果:第一,边界像素和内部像素遵循截然不同的几何特性;第二,在原型质量较弱的情况下,最优传输(OT)所强制执行的整体一致性被低估了。 为了应对这一挑战,我们引入了POTNet,这是对基于原型的最优传输方法的一个改进版本。在新的设计中,我们将POT中的单一k-means步骤替换为由熵引导的双聚类头:高熵像素通过谱聚类组织,低熵像素则通过k-means进行处理,然后使用OT将两个原型集对齐。这种分而治之的设计在一个前向传递过程中产生了更加清晰、具备局部感知能力的伪掩码,并且没有采用手工设计的先验知识。 这些掩码监督了一个标准的MaskFormer风格的编码器-解码器模型,从而诞生了AutoSOD——一种端到端无监督SOD流水线。与SelfMask的离线投票相比,它不仅提高了精度和训练效率,而且还消除了这种依赖性。广泛的实验证明,在五个基准测试上,AutoSOD在F-measure指标上分别比无监督方法高出26%,比弱监督方法高出36%,从而进一步缩小了与完全监督模型之间的差距。
https://arxiv.org/abs/2510.17484
Cardiovascular disease (CVD) risk stratification remains a major challenge due to its multifactorial nature and limited availability of high-quality labeled datasets. While genomic and electrophysiological data such as SNP variants and ECG phenotypes are increasingly accessible, effectively integrating these modalities in low-label settings is non-trivial. This challenge arises from the scarcity of well-annotated multimodal datasets and the high dimensionality of biological signals, which limit the effectiveness of conventional supervised models. To address this, we present a few-label multimodal framework that leverages large language models (LLMs) to combine genetic and electrophysiological information for cardiovascular risk stratification. Our approach incorporates a pseudo-label refinement strategy to adaptively distill high-confidence labels from weakly supervised predictions, enabling robust model fine-tuning with only a small set of ground-truth annotations. To enhance the interpretability, we frame the task as a Chain of Thought (CoT) reasoning problem, prompting the model to produce clinically relevant rationales alongside predictions. Experimental results demonstrate that the integration of multimodal inputs, few-label supervision, and CoT reasoning improves robustness and generalizability across diverse patient profiles. Experimental results using multimodal SNP variants and ECG-derived features demonstrated comparable performance to models trained on the full dataset, underscoring the promise of LLM-based few-label multimodal modeling for advancing personalized cardiovascular care.
心血管疾病(CVD)的风险分层仍然是一个重大挑战,因为其多因素性质以及高质量标注数据的缺乏。尽管基因组和电生理学数据如单核苷酸多态性(SNP)变异和心电图表型越来越容易获得,但在标签较少的情况下有效地整合这些模态却并不简单。这一问题源于多模式注释数据稀缺及生物信号高维度的特点,这限制了传统监督模型的有效性。 为了解决这个问题,我们提出了一种少标签的多模态框架,该框架利用大型语言模型(LLM)结合基因和电生理信息进行心血管风险分层。我们的方法包括伪标签精炼策略,可以从弱监督预测中自适应地提炼出高置信度标签,从而使仅用少量真实标注的情况下也能实现稳健的模型微调。 为了提高可解释性,我们将任务构架为链式思维(CoT)推理问题,促使模型在给出预测的同时提供临床相关理由。实验结果表明,多模态输入、少标签监督和CoT推理可以增强模型在不同患者群体中的鲁棒性和泛化能力。使用多模态SNP变异和心电图派生特征进行的实验结果显示,该方法的表现与基于完整数据集训练的模型相当,这彰显了LLM为基础的少标签多模式建模技术在推进个性化心血管护理方面所展现的巨大潜力。
https://arxiv.org/abs/2510.16536
Annotation-efficient segmentation of the numerous mitochondria instances from various electron microscopy (EM) images is highly valuable for biological and neuroscience research. Although unsupervised domain adaptation (UDA) methods can help mitigate domain shifts and reduce the high costs of annotating each domain, they typically have relatively low performance in practical applications. Thus, we investigate weakly supervised domain adaptation (WDA) that utilizes additional sparse point labels on the target domain, which require minimal annotation effort and minimal expert knowledge. To take full use of the incomplete and imprecise point annotations, we introduce a multitask learning framework that jointly conducts segmentation and center detection with a novel cross-teaching mechanism and class-focused cross-domain contrastive learning. While leveraging unlabeled image regions is essential, we introduce segmentation self-training with a novel instance-aware pseudo-label (IPL) selection strategy. Unlike existing methods that typically rely on pixel-wise pseudo-label filtering, the IPL semantically selects reliable and diverse pseudo-labels with the help of the detection task. Comprehensive validations and comparisons on challenging datasets demonstrate that our method outperforms existing UDA and WDA methods, significantly narrowing the performance gap with the supervised upper bound. Furthermore, under the UDA setting, our method also achieves substantial improvements over other UDA techniques.
从各种电子显微镜(EM)图像中高效注释众多线粒体实例对于生物和神经科学研究非常有价值。虽然无监督领域适应(UDA)方法可以帮助缓解域偏移并减少对每个域进行详细标注的高昂成本,但它们在实际应用中的表现通常相对较低。因此,我们研究了弱监督领域适应(WDA),该方法利用目标域上的稀疏点标签,这些标签只需要最小的努力和专业知识即可完成注释。为了充分利用不完整且不够精确的点注释,我们引入了一个多任务学习框架,该框架同时进行分割和中心检测,并结合了一种新颖的跨教学机制以及以类别为中心的跨领域对比学习方法。 在利用未标注图像区域方面至关重要,我们引入了基于实例感知伪标签(IPL)选择策略的分割自训练。不同于现有方法通常依赖像素级伪标签过滤,IPL通过检测任务的帮助进行语义上可靠和多样化的伪标签选取。在具有挑战性的数据集上的全面验证和比较表明,我们的方法优于现有的UDA和WDA方法,并显著缩小了与监督上限性能差距。此外,在UDA设置下,我们的方法也比其他UDA技术实现了重大改进。
https://arxiv.org/abs/2510.16450
Automatic medical image segmentation is a fundamental step in computer-aided diagnosis, yet fully supervised approaches demand extensive pixel-level annotations that are costly and time-consuming. To alleviate this burden, we propose a weakly supervised segmentation framework that leverages only four extreme points as annotation. Specifically, bounding boxes derived from the extreme points are used as prompts for the Segment Anything Model 2 (SAM2) to generate reliable initial pseudo labels. These pseudo labels are progressively refined by an enhanced Feature-Guided Extreme Point Masking (FGEPM) algorithm, which incorporates Monte Carlo dropout-based uncertainty estimation to construct a unified gradient uncertainty cost map for boundary tracing. Furthermore, a dual-branch Uncertainty-aware Scale Consistency (USC) loss and a box alignment loss are introduced to ensure spatial consistency and precise boundary alignment during training. Extensive experiments on two public ultrasound datasets, BUSI and UNS, demonstrate that our method achieves performance comparable to, and even surpassing fully supervised counterparts while significantly reducing annotation cost. These results validate the effectiveness and practicality of the proposed weakly supervised framework for ultrasound image segmentation.
自动医学图像分割是计算机辅助诊断中的一个基本步骤,但完全监督的方法需要大量的像素级注释,这些注释成本高昂且耗时。为了减轻这一负担,我们提出了一种弱监督分割框架,该框架仅使用四个极端点作为标注。具体而言,从这四个极端点衍生出的边界框被用作提示,以引导Segment Anything Model 2 (SAM2)生成可靠的初始伪标签。随后,通过增强型特征指导极点掩膜(Feature-Guided Extreme Point Masking, FGEPM)算法对这些伪标签进行逐步优化,并结合基于蒙特卡洛丢弃法的不确定性估计来构建统一的梯度不确定性成本图以追踪边界。 此外,我们还引入了一种双分支不确定性感知尺度一致性损失 (Uncertainty-aware Scale Consistency, USC) 以及一个框对齐损失,确保在训练过程中空间一致性和精确边界对齐。在两个公开的超声数据集(BUSI和UNS)上的广泛实验表明,我们的方法在性能上与完全监督的方法相当甚至超越它们的同时显著降低了注释成本。这些结果验证了我们提出的弱监督框架对于超声图像分割的有效性和实用性。
https://arxiv.org/abs/2510.15666
3D weakly supervised semantic segmentation (3D WSSS) aims to achieve semantic segmentation by leveraging sparse or low-cost annotated data, significantly reducing reliance on dense point-wise annotations. Previous works mainly employ class activation maps or pre-trained vision-language models to address this challenge. However, the low quality of pseudo-labels and the insufficient exploitation of 3D geometric priors jointly create significant technical bottlenecks in developing high-performance 3D WSSS models. In this paper, we propose a simple yet effective 3D weakly supervised semantic segmentation method that integrates 3D geometric priors into a class-aware guidance mechanism to generate high-fidelity pseudo labels. Concretely, our designed methodology first employs Class-Aware Label Refinement module to generate more balanced and accurate pseudo labels for semantic categrories. This initial refinement stage focuses on enhancing label quality through category-specific optimization. Subsequently, the Geometry-Aware Label Refinement component is developed, which strategically integrates implicit 3D geometric constraints to effectively filter out low-confidence pseudo labels that fail to comply with geometric plausibility. Moreover, to address the challenge of extensive unlabeled regions, we propose a Label Update strategy that integrates Self-Training to propagate labels into these areas. This iterative process continuously enhances pseudo-label quality while expanding label coverage, ultimately fostering the development of high-performance 3D WSSS models. Comprehensive experimental validation reveals that our proposed methodology achieves state-of-the-art performance on both ScanNet and S3DIS benchmarks while demonstrating remarkable generalization capability in unsupervised settings, maintaining competitive accuracy through its robust design.
3D弱监督语义分割(3D WSSS)的目标是通过利用稀疏或低成本的标注数据来实现语义分割,从而大大减少了对密集点注释的依赖。此前的工作主要采用类别激活图或预训练的视觉-语言模型来应对这一挑战。然而,伪标签的质量低和三维几何先验的利用不足共同构成了开发高性能3D WSSS模型的技术瓶颈。在本文中,我们提出了一种简单而有效的3D弱监督语义分割方法,该方法将3D几何先验整合到类感知引导机制中以生成高质量的伪标签。 具体而言,我们的设计方法首先采用类感知标签精炼模块来为语义类别生成更平衡且准确的伪标签。这一初步细化阶段侧重于通过特定类别的优化提升标签质量。随后开发了基于几何感知的标签精炼组件,该组件战略性地整合隐含的3D几何约束以有效过滤掉不符合几何合理性的低置信度伪标签。 此外,为了解决大量未标注区域的挑战,我们提出了一种结合自训练来将标签传播到这些区域的标签更新策略。这一迭代过程持续提升伪标签质量并扩大标签覆盖范围,最终促进高性能3D WSSS模型的发展。 全面的实验验证显示,我们的方法在ScanNet和S3DIS基准测试中均达到了最先进的性能,并展示了其在无监督环境中的卓越泛化能力,通过稳健的设计保持了竞争力的准确度。
https://arxiv.org/abs/2510.17875
Oral and maxillofacial radiology plays a vital role in dental healthcare, but radiographic image interpretation is limited by a shortage of trained professionals. While AI approaches have shown promise, existing dental AI systems are restricted by their single-modality focus, task-specific design, and reliance on costly labeled data, hindering their generalization across diverse clinical scenarios. To address these challenges, we introduce DentVFM, the first family of vision foundation models (VFMs) designed for dentistry. DentVFM generates task-agnostic visual representations for a wide range of dental applications and uses self-supervised learning on DentVista, a large curated dental imaging dataset with approximately 1.6 million multi-modal radiographic images from various medical centers. DentVFM includes 2D and 3D variants based on the Vision Transformer (ViT) architecture. To address gaps in dental intelligence assessment and benchmarks, we introduce DentBench, a comprehensive benchmark covering eight dental subspecialties, more diseases, imaging modalities, and a wide geographical distribution. DentVFM shows impressive generalist intelligence, demonstrating robust generalization to diverse dental tasks, such as disease diagnosis, treatment analysis, biomarker identification, and anatomical landmark detection and segmentation. Experimental results indicate DentVFM significantly outperforms supervised, self-supervised, and weakly supervised baselines, offering superior generalization, label efficiency, and scalability. Additionally, DentVFM enables cross-modality diagnostics, providing more reliable results than experienced dentists in situations where conventional imaging is unavailable. DentVFM sets a new paradigm for dental AI, offering a scalable, adaptable, and label-efficient model to improve intelligent dental healthcare and address critical gaps in global oral healthcare.
口腔和颌面放射学在牙科医疗保健中扮演着至关重要的角色,但放射图像的解读受限于训练有素的专业人员短缺。虽然人工智能方法显示出巨大潜力,但现有的牙科AI系统因其单一模态聚焦、任务特异性设计以及对昂贵标记数据的依赖性,在不同临床场景中的泛化能力受到限制。为解决这些挑战,我们推出了DentVFM,这是首个专为牙科设计的视觉基础模型(VFMs)家族。DentVFM生成适用于广泛牙科应用的任务无关视觉表示,并通过在DentVista上进行自我监督学习来实现这一点——这是一个包含来自各种医疗机构的大约160万张多模态放射图像的大型策划数据库。 DentVFM包括基于视觉变换器(ViT)架构的2D和3D变体。为填补牙科智能评估与基准测试中的空白,我们推出了DentBench,这是一个涵盖八个牙科亚专科、更多疾病类型、成像模式以及广泛地理分布的全面基准。实验结果表明,DentVFM在多种牙科任务中展现了强大的通才智慧能力,如疾病诊断、治疗分析、生物标志物识别和解剖地标检测与分割。 实验证明,DentVFM在泛化性能、标签效率和可扩展性方面显著优于监督式学习、自我监督学习以及弱监督基础模型。此外,DentVFM支持跨模态诊断,在传统成像不可用的情况下提供比经验丰富的牙医更为可靠的诊断结果。DentVFM为牙科AI树立了新的范例,通过一个可扩展的、适应性强且标签高效的模型来改进智能牙科医疗保健,并填补全球口腔健康护理中的关键空白。
https://arxiv.org/abs/2510.14532
Procedure step recognition (PSR) aims to identify all correctly completed steps and their sequential order in videos of procedural tasks. The existing state-of-the-art models rely solely on detecting assembly object states in individual video frames. By neglecting temporal features, model robustness and accuracy are limited, especially when objects are partially occluded. To overcome these limitations, we propose Spatio-Temporal Occlusion-Resilient Modeling for Procedure Step Recognition (STORM-PSR), a dual-stream framework for PSR that leverages both spatial and temporal features. The assembly state detection stream operates effectively with unobstructed views of the object, while the spatio-temporal stream captures both spatial and temporal features to recognize step completions even under partial occlusion. This stream includes a spatial encoder, pre-trained using a novel weakly supervised approach to capture meaningful spatial representations, and a transformer-based temporal encoder that learns how these spatial features relate over time. STORM-PSR is evaluated on the MECCANO and IndustReal datasets, reducing the average delay between actual and predicted assembly step completions by 11.2% and 26.1%, respectively, compared to prior methods. We demonstrate that this reduction in delay is driven by the spatio-temporal stream, which does not rely on unobstructed views of the object to infer completed steps. The code for STORM-PSR, along with the newly annotated MECCANO labels, is made publicly available at this https URL .
步骤识别(PSR)的目标是确定程序任务视频中所有正确完成的步骤及其顺序。现有最先进的模型仅依赖于检测单个视频帧中的装配物体状态。由于忽略了时间特征,这些模型在面对部分遮挡情况时的鲁棒性和准确性受到限制。为克服这一局限性,我们提出了时空抗遮挡模型(STORM-PSR),这是一种用于程序步骤识别的双流框架,结合了空间和时间特征的优势。 装配状态检测流能够有效地处理物体未受阻碍的情况,而时空流则捕捉了时间和空间特性,在部分遮挡的情况下也能识别出步骤完成情况。这一流程包括一个通过新型弱监督方法预训练的空间编码器,用于捕获有意义的空间表示;以及一个基于变压器的时间编码器,学习这些空间特征随时间的关系。 STORM-PSR在MECCANO和IndustReal数据集上进行了评估,在预测装配步骤完成的平均延迟方面相比先前的方法分别减少了11.2%和26.1%。我们展示了这种延迟减少是由时空流驱动的,该流程不需要物体未受阻碍的视角来推断已完成的步骤。 STORM-PSR的代码以及新标注的MECCANO标签已在以下URL公开提供:[这里应插入链接](原文中的具体网址需要根据实际情况填写)。
https://arxiv.org/abs/2510.12385
3D instance segmentation is crucial for understanding complex 3D environments, yet fully supervised methods require dense point-level annotations, resulting in substantial annotation costs and labor overhead. To mitigate this, box-level annotations have been explored as a weaker but more scalable form of supervision. However, box annotations inherently introduce ambiguity in overlapping regions, making accurate point-to-instance assignment challenging. Recent methods address this ambiguity by generating pseudo-masks through training a dedicated pseudo-labeler in an additional training stage. However, such two-stage pipelines often increase overall training time and complexity, hinder end-to-end optimization. To overcome these challenges, we propose BEEP3D-Box-supervised End-to-End Pseudo-mask generation for 3D instance segmentation. BEEP3D adopts a student-teacher framework, where the teacher model serves as a pseudo-labeler and is updated by the student model via an Exponential Moving Average. To better guide the teacher model to generate precise pseudo-masks, we introduce an instance center-based query refinement that enhances position query localization and leverages features near instance centers. Additionally, we design two novel losses-query consistency loss and masked feature consistency loss-to align semantic and geometric signals between predictions and pseudo-masks. Extensive experiments on ScanNetV2 and S3DIS datasets demonstrate that BEEP3D achieves competitive or superior performance compared to state-of-the-art weakly supervised methods while remaining computationally efficient.
三维实例分割对于理解复杂的三维环境至关重要,然而完全监督的方法需要密集的点级别标注,这导致了大量的注释成本和劳动力投入。为了缓解这一问题,人们探索了使用箱级标注作为更弱但更具可扩展性的监督形式。然而,边界框标注在重叠区域内在固有地引入了模糊性,使得准确的点到实例分配变得具有挑战性。近期的方法通过训练一个专门的伪标签生成器来解决这种模糊性,但这通常需要额外的一个训练阶段,并且会产生伪掩码。这样的两阶段管道往往会增加整体训练时间和复杂度,阻碍端到端优化。 为了解决这些挑战,我们提出了BEEP3D——一种基于边界框监督的三维实例分割中的端到端伪掩码生成方法。BEEP3D采用了一种师生框架,在该框架中,教师模型充当伪标签生成器,并通过指数移动平均法由学生模型进行更新。为了更好地指导教师模型生成精确的伪掩码,我们引入了一个基于实例中心的查询细化机制,增强了位置查询的定位能力,并利用了靠近实例中心的特征信息。此外,我们设计了两种新颖的损失函数——查询一致性损失和掩码特征一致性损失,以便在预测与伪掩码之间对齐语义和几何信号。 在ScanNetV2和S3DIS数据集上的广泛实验表明,BEEP3D相比现有的弱监督方法取得了具有竞争力或更优的性能,并且保持了计算效率。
https://arxiv.org/abs/2510.12182
Patient face images provide a convenient mean for evaluating eye diseases, while also raising privacy concerns. Here, we introduce ROFI, a deep learning-based privacy protection framework for ophthalmology. Using weakly supervised learning and neural identity translation, ROFI anonymizes facial features while retaining disease features (over 98\% accuracy, $\kappa > 0.90$). It achieves 100\% diagnostic sensitivity and high agreement ($\kappa > 0.90$) across eleven eye diseases in three cohorts, anonymizing over 95\% of images. ROFI works with AI systems, maintaining original diagnoses ($\kappa > 0.80$), and supports secure image reversal (over 98\% similarity), enabling audits and long-term care. These results show ROFI's effectiveness of protecting patient privacy in the digital medicine era.
患者面部图像为评估眼科疾病提供了一种便捷的方法,同时也引发了隐私方面的担忧。在这里,我们介绍了ROFI,这是一个基于深度学习的眼科隐私保护框架。通过弱监督学习和神经身份翻译技术,ROFI能够在去除非必要面部特征的同时保留疾病的特征(超过98%的准确性,$\kappa > 0.90$)。在三个不同队列中的十一种眼科疾病上,ROFI实现了100%的诊断敏感度,并且具有高度的一致性($\kappa > 0.90$),同时去除了超过95%图像中的非必要面部信息。此外,该框架能够与AI系统兼容,保持原始诊断结果($\kappa > 0.80$),并支持安全的图像恢复功能(相似度超过98%),从而允许进行审核和长期跟踪护理。这些结果显示了ROFI在数字医疗时代有效保护患者隐私方面的效果。
https://arxiv.org/abs/2510.11073
Laryngeal cancer imaging research lacks standardised datasets to enable reproducible deep learning (DL) model development. We present LaryngealCT, a curated benchmark of 1,029 computed tomography (CT) scans aggregated from six collections from The Cancer Imaging Archive (TCIA). Uniform 1 mm isotropic volumes of interest encompassing the larynx were extracted using a weakly supervised parameter search framework validated by clinical experts. 3D DL architectures (3D CNN, ResNet18,50,101, DenseNet121) were benchmarked on (i) early (Tis,T1,T2) vs. advanced (T3,T4) and (ii) T4 vs. non-T4 classification tasks. 3D CNN (AUC-0.881, F1-macro-0.821) and ResNet18 (AUC-0.892, F1-macro-0.646) respectively outperformed the other models in the two tasks. Model explainability assessed using 3D GradCAMs with thyroid cartilage overlays revealed greater peri-cartilage attention in non-T4 cases and focal activations in T4 predictions. Through open-source data, pretrained models, and integrated explainability tools, LaryngealCT offers a reproducible foundation for AI-driven research to support clinical decisions in laryngeal oncology.
喉癌的成像研究缺乏标准化的数据集,以支持可重复的深度学习(DL)模型开发。我们提出了LaryngealCT,这是一个由六个来自癌症影像档案馆(TCIA)的集合汇总而成的基准数据集,包含1,029份计算机断层扫描(CT)图像。使用弱监督参数搜索框架并经临床专家验证后,提取了涵盖喉部的统一1毫米等轴距感兴趣体积。 三维DL架构(包括3D CNN、ResNet18、50和101、DenseNet121)在两类任务上进行了基准测试:早期阶段(Tis, T1, T2)与晚期阶段(T3, T4),以及T4与非T4的分类任务。在两个任务中,3D CNN(AUC-0.881,F1-macro-0.821)和ResNet18(AUC-0.892,F1-macro-0.646)分别超过了其他模型的表现。 通过使用带有甲状软骨叠加的三维GradCAM进行模型解释性评估,非T4病例显示了更显著的周围软骨关注区域,而T4预测则显示出焦点激活。通过开源数据、预训练模型和集成可解释工具,LaryngealCT为支持喉癌临床决策的人工智能研究提供了可重复的基础。
https://arxiv.org/abs/2510.11047
Advances in image tampering pose serious security threats, underscoring the need for effective image manipulation localization (IML). While supervised IML achieves strong performance, it depends on costly pixel-level annotations. Existing weakly supervised or training-free alternatives often underperform and lack interpretability. We propose the In-Context Forensic Chain (ICFC), a training-free framework that leverages multi-modal large language models (MLLMs) for interpretable IML tasks. ICFC integrates an objectified rule construction with adaptive filtering to build a reliable knowledge base and a multi-step progressive reasoning pipeline that mirrors expert forensic workflows from coarse proposals to fine-grained forensics results. This design enables systematic exploitation of MLLM reasoning for image-level classification, pixel-level localization, and text-level interpretability. Across multiple benchmarks, ICFC not only surpasses state-of-the-art training-free methods but also achieves competitive or superior performance compared to weakly and fully supervised approaches.
图像篡改技术的进步带来了严重的安全威胁,突显了有效图像操作定位(IML)的必要性。虽然监督下的IML取得了很好的性能,但它依赖于昂贵的像素级标注。现有的弱监督或无训练替代方法往往表现不佳且缺乏解释能力。我们提出了一个无训练框架——上下文取证链(ICFC),该框架利用多模态大型语言模型(MLLM)来进行可解释的图像操作定位任务。 ICFC结合了对象化规则构建和自适应过滤,建立了一个可靠的知识库,并形成一个多步骤渐进推理管道,这一设计模仿了专家级的取证工作流程,从粗略提案到细粒度的取证结果。这种设计能够系统地利用MLLM推理进行图像级别的分类、像素级别的定位以及文本级别的解释。 在多个基准测试中,ICFC不仅超越了现有的无训练方法,而且在性能上与弱监督和全监督方法相当或更优。
https://arxiv.org/abs/2510.10111
Automatic Speech Recognition (ASR) has undergone a profound transformation over the past decade, driven by advances in deep learning. This survey provides a comprehensive overview of the modern era of ASR, charting its evolution from traditional hybrid systems, such as Gaussian Mixture Model-Hidden Markov Models (GMM-HMMs) and Deep Neural Network-HMMs (DNN-HMMs), to the now-dominant end-to-end neural architectures. We systematically review the foundational end-to-end paradigms: Connectionist Temporal Classification (CTC), attention-based encoder-decoder models, and the Recurrent Neural Network Transducer (RNN-T), which established the groundwork for fully integrated speech-to-text systems. We then detail the subsequent architectural shift towards Transformer and Conformer models, which leverage self-attention to capture long-range dependencies with high computational efficiency. A central theme of this survey is the parallel revolution in training paradigms. We examine the progression from fully supervised learning, augmented by techniques like SpecAugment, to the rise of self-supervised learning (SSL) with foundation models such as wav2vec 2.0, which drastically reduce the reliance on transcribed data. Furthermore, we analyze the impact of largescale, weakly supervised models like Whisper, which achieve unprecedented robustness through massive data diversity. The paper also covers essential ecosystem components, including key datasets and benchmarks (e.g., LibriSpeech, Switchboard, CHiME), standard evaluation metrics (e.g., Word Error Rate), and critical considerations for real-world deployment, such as streaming inference, on-device efficiency, and the ethical imperatives of fairness and robustness. We conclude by outlining open challenges and future research directions.
自动语音识别(ASR)在过去十年中经历了一场深刻的变革,这是由于深度学习的进步所推动的。本综述提供了现代ASR时代的全面概述,回顾了其从传统的混合系统(如高斯混合模型-隐马尔可夫模型(GMM-HMM)和深层神经网络-隐马尔可夫模型(DNN-HMM))到如今占主导地位的端到端神经架构的发展历程。我们系统地回顾了基础性的端到端范式:连接时间分类(CTC)、基于注意力的编码器-解码器模型以及循环神经网络转导器(RNN-T),这些范式为完全集成的语音到文本系统奠定了基础。随后,我们详细介绍了架构向变压器和卷积变压器模型转变的过程,这些模型利用自注意力机制来高效地捕捉长程依赖关系。本综述的一个中心主题是训练范式的平行革命。我们研究了从全监督学习(通过SpecAugment等技术增强)到基于wav2vec 2.0等基础模型的自我监督学习(SSL)的发展过程,这些方法极大地减少了对转录数据的依赖。此外,我们还分析了大规模弱监督模型(如Whisper)的影响,这些模型通过海量的数据多样性实现了前所未有的鲁棒性。论文还涵盖了关键生态系统组件,包括重要的数据集和基准测试(例如LibriSpeech、Switchboard、CHiME)、标准评估指标(例如单词错误率),以及在实际部署中至关重要的考虑因素,如流推理、设备效率及公平性和稳健性的伦理要求。最后,我们概述了开放性挑战与未来的研究方向。
https://arxiv.org/abs/2510.12827
Accurate segmentation of infant brain MRI is critical for studying early neurodevelopment and diagnosing neurological disorders. Yet, it remains a fundamental challenge due to continuously evolving anatomy of the subjects, motion artifacts, and the scarcity of high-quality labeled data. In this work, we present LODi, a novel framework that utilizes prior knowledge from an adult brain MRI segmentation model to enhance the segmentation performance of infant scans. Given the abundance of publicly available adult brain MRI data, we pre-train a segmentation model on a large adult dataset as a starting point. Through transfer learning and domain adaptation strategies, we progressively adapt the model to the 0-2 year-old population, enabling it to account for the anatomical and imaging variability typical of infant scans. The adaptation of the adult model is carried out using weakly supervised learning on infant brain scans, leveraging silver-standard ground truth labels obtained with FreeSurfer. By introducing a novel training strategy that integrates hierarchical feature refinement and multi-level consistency constraints, our method enables fast, accurate, age-adaptive segmentation, while mitigating scanner and site-specific biases. Extensive experiments on both internal and external datasets demonstrate the superiority of our approach over traditional supervised learning and domain-specific models. Our findings highlight the advantage of leveraging adult brain priors as a foundation for age-flexible neuroimaging analysis, paving the way for more reliable and generalizable brain MRI segmentation across the lifespan.
婴儿脑部MRI的精确分割对于研究早期神经发育和诊断神经系统疾病至关重要。然而,由于受试者解剖结构不断变化、运动伪影以及高质量标注数据稀缺等因素,这一任务仍面临重大挑战。在这项工作中,我们提出了LODi框架,利用成人脑部MRI分割模型中的先验知识来提高婴儿扫描的分割性能。鉴于公开可用的成人脑部MRI数据丰富,我们首先在一个大型成人数据集上预训练了一个分割模型作为起点。通过迁移学习和领域适应策略,逐步将该模型调整适用于0至2岁儿童群体,使其能够应对婴儿扫描中常见的解剖学和成像变异。 我们将成年人模型应用到婴儿脑部扫描的过程中采用了弱监督学习,并利用FreeSurfer获得的银标准地面真实标签进行训练。通过引入一种结合层级特征精炼和多级一致性约束的新颖训练策略,我们的方法实现了快速、精确且适应年龄变化的分割效果,同时减少了特定扫描仪和地点带来的偏差。 在内部和外部数据集上的广泛实验表明,相较于传统的监督学习和领域特化模型,我们所提出的方法表现更优。我们的发现强调了利用成人脑部先验知识作为跨年龄段灵活神经影像分析基础的重要性,并为整个生命周期内更加可靠且通用的脑部MRI分割铺平了道路。
https://arxiv.org/abs/2510.09306
Weakly Supervised Anomaly detection (WSAD) in brain MRI scans is an important challenge useful to obtain quick and accurate detection of brain anomalies when precise pixel-level anomaly annotations are unavailable and only weak labels (e.g., slice-level) are available. In this work, we propose RASALoRE: Region Aware Spatial Attention with Location-based Random Embeddings, a novel two-stage WSAD framework. In the first stage, we introduce a Discriminative Dual Prompt Tuning (DDPT) mechanism that generates high-quality pseudo weak masks based on slice-level labels, serving as coarse localization cues. In the second stage, we propose a segmentation network with a region-aware spatial attention mechanism that relies on fixed location-based random embeddings. This design enables the model to effectively focus on anomalous regions. Our approach achieves state-of-the-art anomaly detection performance, significantly outperforming existing WSAD methods while utilizing less than 8 million parameters. Extensive evaluations on the BraTS20, BraTS21, BraTS23, and MSD datasets demonstrate a substantial performance improvement coupled with a significant reduction in computational complexity. Code is available at: this https URL.
弱监督异常检测(WSAD)在脑MRI扫描中的应用是一项重要的挑战,特别是在精确的像素级异常标注不可用且仅能获取到弱标签(如切片级别)的情况下。为了快速准确地识别大脑异常情况,在这项工作中,我们提出了一种新的两阶段WSAD框架——RASALoRE(基于位置随机嵌入的位置感知空间注意力机制)。该框架分为两个阶段: 在第一阶段,我们引入了判别性双提示调优(Discriminative Dual Prompt Tuning, DDPT)机制,能够根据切片级别的标签生成高质量的伪弱掩模,作为粗略定位线索。 第二阶段中,我们提出了一种基于位置感知空间注意力机制的分割网络,并使用固定的位置随机嵌入。这种设计使得模型能够在异常区域有效聚焦。 我们的方法在多种数据集上实现了最先进的异常检测性能,在参数量小于800万的情况下显著优于现有的WSAD方法。对BraTS20、BraTS21、BraTS23和MSD等数据集的广泛评估显示,该模型不仅提高了性能还大幅减少了计算复杂度。 代码可在以下链接获取:[提供URL]
https://arxiv.org/abs/2510.08052
Universal models for medical image segmentation, such as interactive and in-context learning (ICL) models, offer strong generalization but require extensive annotations. Interactive models need repeated user prompts for each image, while ICL relies on dense, pixel-level labels. To address this, we propose Weakly Supervised In-Context Learning (WS-ICL), a new ICL paradigm that leverages weak prompts (e.g., bounding boxes or points) instead of dense labels for context. This approach significantly reduces annotation effort by eliminating the need for fine-grained masks and repeated user prompting for all images. We evaluated the proposed WS-ICL model on three held-out benchmarks. Experimental results demonstrate that WS-ICL achieves performance comparable to regular ICL models at a significantly lower annotation cost. In addition, WS-ICL is highly competitive even under the interactive paradigm. These findings establish WS-ICL as a promising step toward more efficient and unified universal models for medical image segmentation. Our code and model are publicly available at this https URL.
通用医学图像分割模型,如交互式和上下文内学习(ICL)模型,虽然具有很强的泛化能力,但需要大量的标注数据。交互式模型要求用户对每个图像进行多次提示输入,而ICL则依赖于密集、像素级别的标签。为解决这一问题,我们提出了弱监督上下文内学习(WS-ICL),这是一种新的ICL范式,它利用弱提示(例如边界框或点)而不是密集标签来提供上下文信息。这种方法通过消除对精细标注掩模的需求以及所有图像的重复用户提示需求,显著减少了注释工作量。 我们在三个独立的基准测试集上评估了所提出的WS-ICL模型。实验结果表明,在大大减少标注成本的情况下,WS-ICL与常规ICL模型相比能够达到相似的效果表现。此外,即使在交互式范式下,WS-ICL也表现出很强的竞争性。 这些发现证明了WS-ICL作为更高效和统一的医学图像分割通用模型的重要一步。我们的代码和模型可以在以下链接公开获取:[此处插入URL]。
https://arxiv.org/abs/2510.05899