Abstract
Sentiment analysis is a crucial task in natural language processing (NLP) that enables the extraction of meaningful insights from textual data, particularly from dynamic platforms like Twitter and IMDB. This study explores a hybrid framework combining transformer-based models, specifically BERT, GPT-2, RoBERTa, XLNet, and DistilBERT, to improve sentiment classification accuracy and robustness. The framework addresses challenges such as noisy data, contextual ambiguity, and generalization across diverse datasets by leveraging the unique strengths of these models. BERT captures bidirectional context, GPT-2 enhances generative capabilities, RoBERTa optimizes contextual understanding with larger corpora and dynamic masking, XLNet models dependency through permutation-based learning, and DistilBERT offers efficiency with reduced computational overhead while maintaining high accuracy. We demonstrate text cleaning, tokenization, and feature extraction using Term Frequency Inverse Document Frequency (TF-IDF) and Bag of Words (BoW), ensure high-quality input data for the models. The hybrid approach was evaluated on benchmark datasets Sentiment140 and IMDB, achieving superior accuracy rates of 94\% and 95\%, respectively, outperforming standalone models. The results validate the effectiveness of combining multiple transformer models in ensemble-like setups to address the limitations of individual architectures. This research highlights its applicability to real-world tasks such as social media monitoring, customer sentiment analysis, and public opinion tracking which offers a pathway for future advancements in hybrid NLP frameworks.
Abstract (translated)
情感分析是自然语言处理（NLP）中的关键任务，它能够从文本数据中提取有意义的见解，特别是在如Twitter和IMDb这样的动态平台上。本研究探讨了一种结合基于变压器模型（包括BERT、GPT-2、RoBERTa、XLNet以及DistilBERT）的混合框架，以提高情感分类的准确性和鲁棒性。该框架通过利用这些模型的独特优势来应对诸如噪音数据、语境模糊以及在不同数据集上泛化等方面的挑战。 具体来说，BERT能够捕捉双向上下文信息，GPT-2提升了生成能力，RoBERTa使用更大规模的数据集和动态屏蔽技术优化了对上下文的理解，XLNet通过基于排列的学习方式建模依赖关系，而DistilBERT则在减少计算开销的同时保持高准确率。我们展示了如何利用词频逆文档频率（TF-IDF）和词汇袋（BoW）进行文本清洗、分词以及特征提取，以确保输入模型的数据质量。 混合方法在Sentiment140和IMDb基准数据集上进行了评估，在这些数据集中分别达到了94%和95%的优越准确率，超过了单一模型的表现。研究结果验证了通过类似集成的方法组合多个变压器模型可以有效解决单一架构的局限性。 这项研究强调其在实际任务中的适用性，如社交媒体监控、客户情感分析以及公众舆论跟踪等领域，并为未来混合NLP框架的发展提供了方向。
URL
https://arxiv.org/abs/2504.09896
