Abstract
Multimodal Sentiment Analysis (MSA) aims to infer human sentiment from textual, acoustic, and visual signals. In real-world scenarios, however, multimodal inputs are often compromised by dynamic noise or modality missingness. Existing methods typically treat these imperfections as discrete cases or assume fixed corruption ratios, which limits their adaptability to continuously varying reliability conditions. To address this, we first introduce a Continuous Reliability Spectrum to unify missingness and quality degradation into a single framework. Building on this, we propose QA-MoE, a Quality-Aware Mixture-of-Experts framework that quantifies modality reliability via self-supervised aleatoric uncertainty. This mechanism explicitly guides expert routing, enabling the model to suppress error propagation from unreliable signals while preserving task-relevant information. Extensive experiments indicate that QA-MoE achieves competitive or state-of-the-art performance across diverse degradation scenarios and exhibits a promising One-Checkpoint-for-All property in practice.
Abstract (translated)
多模态情感分析(MSA)旨在通过文本、声学和视觉信号推断人类情感。然而在现实场景中,多模态输入常因动态噪声或模态缺失而受损。现有方法通常将这些缺陷视为离散情况或假设固定的损坏比例,这限制了模型对持续变化的可靠性条件的适应能力。为解决此问题,我们首先提出连续可靠性谱系,将缺失与质量退化统一到单一框架中。在此基础上,我们设计了QA-MoE(质量感知混合专家)框架,通过自监督的偶然性不确定性量化模态可靠性。该机制能明确引导专家路由,使模型在抑制不可靠信号误差传播的同时保留任务相关信息。大量实验表明,QA-MoE在各种退化场景中均达到具有竞争力或最优性能,并展现出实践中"一检通用"的优良特性。
URL
https://arxiv.org/abs/2604.05704