Paper Reading AI Learner

Exploring Model Editing for LLM-based Aspect-Based Sentiment Classification

2025-03-19 11:21:37
Shichen Li, Zhongqing Wang, Zheyu Zhao, Yue Zhang, Peifeng Li

Abstract

Model editing aims at selectively updating a small subset of a neural model's parameters with an interpretable strategy to achieve desired modifications. It can significantly reduce computational costs to adapt to large language models (LLMs). Given its ability to precisely target critical components within LLMs, model editing shows great potential for efficient fine-tuning applications. In this work, we investigate model editing to serve an efficient method for adapting LLMs to solve aspect-based sentiment classification. Through causal interventions, we trace and determine which neuron hidden states are essential for the prediction of the model. By performing interventions and restorations on each component of an LLM, we identify the importance of these components for aspect-based sentiment classification. Our findings reveal that a distinct set of mid-layer representations is essential for detecting the sentiment polarity of given aspect words. Leveraging these insights, we develop a model editing approach that focuses exclusively on these critical parts of the LLM, leading to a more efficient method for adapting LLMs. Our in-domain and out-of-domain experiments demonstrate that this approach achieves competitive results compared to the currently strongest methods with significantly fewer trainable parameters, highlighting a more efficient and interpretable fine-tuning strategy.

Abstract (translated)

模型编辑旨在通过一种可解释的策略,选择性地更新神经网络模型中一小部分参数,以实现所需的修改。这种方法可以显著降低针对大规模语言模型(LLM)进行适应时的计算成本。由于其能够精准定位到LLM中的关键组件,模型编辑在高效的微调应用中展现出巨大潜力。 在这项工作中,我们探讨了使用模型编辑作为一种高效的方法来调整LLM以解决基于方面的观点分类问题。通过因果干预,我们追踪并确定哪些神经隐藏状态对于模型的预测至关重要。通过对LLM每个组成部分进行干预和恢复操作,我们识别出这些组件在基于方面的观点分类中的重要性。 我们的研究发现表明,一组独特的中间层表示对检测给定方面词的情感极性至关重要。利用这一洞察,我们开发了一种专注于这些关键部分的模型编辑方法,从而实现一种更高效的LLM调整方式。我们在领域内和跨领域的实验中展示了这种方法能够与当前最强的方法相比取得相当的结果,并且使用了显著较少的可训练参数,突显出一个更为高效和可解释的微调策略。

URL

https://arxiv.org/abs/2503.15117

PDF

https://arxiv.org/pdf/2503.15117.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot