Paper Reading AI Learner

Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis

2026-02-23 08:19:54
Chunlei Meng, Jiabin Luo, Zhenglin Yan, Zhenyu Yu, Rong Fu, Zhongxue Gan, Chun Ouyang

Abstract

Multimodal Sentiment Analysis (MSA) integrates language, visual, and acoustic modalities to infer human sentiment. Most existing methods either focus on globally shared representations or modality-specific features, while overlooking signals that are shared only by certain modality pairs. This limits the expressiveness and discriminative power of multimodal representations. To address this limitation, we propose a Tri-Subspace Disentanglement (TSD) framework that explicitly factorizes features into three complementary subspaces: a common subspace capturing global consistency, submodally-shared subspaces modeling pairwise cross-modal synergies, and private subspaces preserving modality-specific cues. To keep these subspaces pure and independent, we introduce a decoupling supervisor together with structured regularization losses. We further design a Subspace-Aware Cross-Attention (SACA) fusion module that adaptively models and integrates information from the three subspaces to obtain richer and more robust representations. Experiments on CMU-MOSI and CMU-MOSEI demonstrate that TSD achieves state-of-the-art performance across all key metrics, reaching 0.691 MAE on CMU-MOSI and 54.9% ACC-7 on CMU-MOSEI, and also transfers well to multimodal intent recognition tasks. Ablation studies confirm that tri-subspace disentanglement and SACA jointly enhance the modeling of multi-granular cross-modal sentiment cues.

Abstract (translated)

多模态情感分析(MSA)整合语言、视觉和听觉模式来推断人类情绪。现有大多数方法要么侧重于全局共享表示,要么关注特定模态的特征,而忽略了仅由某些模态对共有的信号。这限制了多模态表示的表达能力和判别能力。为了解决这一局限性,我们提出了一种三子空间解耦(TSD)框架,该框架明确地将特征分解成三个互补的子空间:一个捕捉全局一致性的公共子空间、两个跨模式协同作用的子模态共享子空间以及保留特定模态线索的私有子空间。为了保持这些子空间的纯净和独立性,我们引入了一个解耦监督器,并结合结构化正则化损失。此外,我们还设计了一种感知子空间的交叉注意力(SACA)融合模块,该模块能够自适应地从三个子空间中建模和集成信息,以获得更丰富、更稳健的表示。 在CMU-MOSI和CMU-MOSEI数据集上的实验表明,TSD框架实现了最先进的性能,在所有关键指标上均表现出色:在CMU-MOSI上达到0.691 MAE(平均绝对误差);在CMU-MOSEI上实现54.9%的ACC-7准确率,并且该模型还能很好地转移到多模态意图识别任务中。消融研究表明,三子空间解耦和SACA模块共同提高了对跨模式情感线索的多层次建模能力。 总结来说,TSD框架通过引入更精细的特征表示和融合机制,在多模态情感分析任务上取得了显著的进步,并展示了良好的泛化能力和应用潜力。

URL

https://arxiv.org/abs/2602.19585

PDF

https://arxiv.org/pdf/2602.19585.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot