Paper Reading AI Learner

Multi-Domain ABSA Conversation Dataset Generation via LLMs for Real-World Evaluation and Model Comparison

2025-05-30 15:24:17
Tejul Pandit, Meet Raval, Dhvani Upadhyay

Abstract

Aspect-Based Sentiment Analysis (ABSA) offers granular insights into opinions but often suffers from the scarcity of diverse, labeled datasets that reflect real-world conversational nuances. This paper presents an approach for generating synthetic ABSA data using Large Language Models (LLMs) to address this gap. We detail the generation process aimed at producing data with consistent topic and sentiment distributions across multiple domains using GPT-4o. The quality and utility of the generated data were evaluated by assessing the performance of three state-of-the-art LLMs (Gemini 1.5 Pro, Claude 3.5 Sonnet, and DeepSeek-R1) on topic and sentiment classification tasks. Our results demonstrate the effectiveness of the synthetic data, revealing distinct performance trade-offs among the models: DeepSeekR1 showed higher precision, Gemini 1.5 Pro and Claude 3.5 Sonnet exhibited strong recall, and Gemini 1.5 Pro offered significantly faster inference. We conclude that LLM-based synthetic data generation is a viable and flexible method for creating valuable ABSA resources, facilitating research and model evaluation without reliance on limited or inaccessible real-world labeled data.

Abstract (translated)

基于方面的情感分析(ABSA)提供了对意见的精细洞察,但通常会因为缺乏多样且标注完善的现实对话数据而受限。本文提出了一种使用大型语言模型(LLMs)生成合成ABSA数据的方法,以弥补这一缺口。我们详细描述了利用GPT-4o来产生在多个领域内具有一致主题和情感分布的合成数据的过程。通过评估三种最先进的LLM(Gemini 1.5 Pro、Claude 3.5 Sonnet 和 DeepSeek-R1)在主题分类和情感分析任务上的性能,我们对生成的数据质量进行了评价。我们的结果显示了合成数据的有效性,并揭示了不同模型之间的性能权衡:DeepSeekR1表现出更高的精度,Gemini 1.5 Pro和Claude 3.5 Sonnet显示出强大的召回率,而Gemini 1.5 Pro则提供了显著更快的推理速度。我们得出结论,基于LLM的合成数据生成是一种可行且灵活的方法,能够创建有价值的ABSA资源,从而在没有受限或难以获取的真实世界标注数据的情况下支持研究和模型评估。

URL

https://arxiv.org/abs/2505.24701

PDF

https://arxiv.org/pdf/2505.24701.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot