Sentiment_Classification
-
DyFuLM: An Advanced Multimodal Framework for Sentiment Analysis
2025-12-01 08:30:10Ruohan Zhou, Jiachen Yuan, Churui Yang, Wenzheng Huang, Guoyan Zhang, Shiyao Wei, Jiazhen Hu, Ning Xin, Md Maruf HasanAbstract
Understanding sentiment in complex textual expressions remains a fundamental challenge in affective computing. To address this, we propose a Dynamic Fusion Learning Model (DyFuLM), a multimodal framework designed to capture both hierarchical semantic representations and fine-grained emotional nuances. DyFuLM introduces two key moodules: a Hierarchical Dynamic Fusion module that adaptively integrates multi-level features, and a Gated Feature Aggregation module that regulates cross-layer information ffow to achieve balanced representation learning. Comprehensive experiments on multi-task sentiment datasets demonstrate that DyFuLM achieves 82.64% coarse-grained and 68.48% fine-grained accuracy, yielding the lowest regression errors (MAE = 0.0674, MSE = 0.0082) and the highest R^2 coefficient of determination (R^2= 0.6903). Furthermore, the ablation study validates the effectiveness of each module in DyFuLM. When all modules are removed, the accuracy drops by 0.91% for coarse-grained and 0.68% for fine-grained tasks. Keeping only the gated fusion module causes decreases of 0.75% and 0.55%, while removing the dynamic loss mechanism results in drops of 0.78% and 0.26% for coarse-grained and fine-grained sentiment classification, respectively. These results demonstrate that each module contributes significantly to feature interaction and task balance. Overall, the experimental findings further validate that DyFuLM enhances sentiment representation and overall performance through effective hierarchical feature fusion.
Abstract (translated)
理解复杂文本表达中的情感仍然是情感计算中的一个基本挑战。为了解决这一问题，我们提出了一种动态融合学习模型（DyFuLM），这是一种多模态框架，旨在捕捉层次化的语义表示和细微的情感差异。DyFuLM引入了两个关键模块：一种是层级动态融合模块，它能够自适应地整合多层次特征；另一种是有门控特性聚合的模块，该模块调节跨层信息流以实现平衡的表示学习。 在多任务情感数据集上的全面实验表明，DyFuLM实现了82.64%的粗粒度准确率和68.48%的细粒度准确率，并且获得了最低的回归误差（MAE = 0.0674，MSE = 0.0082）以及最高的R^2决定系数（R^2= 0.6903）。此外，消融实验验证了DyFuLM中每个模块的有效性。当移除所有模块时，粗粒度和细粒度任务的准确率分别下降了0.91% 和0.68%；仅保留门控融合模块导致准确率分别降低了0.75% 和0.55%，而去除动态损失机制则使粗粒度和细粒度情感分类的准确率分别下降了0.78% 和0.26%。这些结果表明，每个模块在特征交互和任务平衡中都发挥了重要作用。 总体而言，实验发现进一步验证了DyFuLM通过有效的层次化特征融合增强了情感表示和整体性能。
URL
https://arxiv.org/abs/2512.01410
PDF
-
Fine-tuning of lightweight large language models for sentiment classification on heterogeneous financial textual data
2025-11-30 15:58:22Alvaro Paredes Amorin, Andre Python, Christoph WeisserAbstract
Large language models (LLMs) play an increasingly important role in finan- cial markets analysis by capturing signals from complex and heterogeneous textual data sources, such as tweets, news articles, reports, and microblogs. However, their performance is dependent on large computational resources and proprietary datasets, which are costly, restricted, and therefore inacces- sible to many researchers and practitioners. To reflect realistic situations we investigate the ability of lightweight open-source LLMs - smaller and publicly available models designed to operate with limited computational resources - to generalize sentiment understanding from financial datasets of varying sizes, sources, formats, and languages. We compare the benchmark finance natural language processing (NLP) model, FinBERT, and three open-source lightweight LLMs, DeepSeek-LLM 7B, Llama3 8B Instruct, and Qwen3 8B on five publicly available datasets: FinancialPhraseBank, Financial Question Answering, Gold News Sentiment, Twitter Sentiment and Chinese Finance Sentiment. We find that LLMs, specially Qwen3 8B and Llama3 8B, perform best in most scenarios, even from using only 5% of the available training data. These results hold in zero-shot and few-shot learning scenarios. Our findings indicate that lightweight, open-source large language models (LLMs) consti- tute a cost-effective option, as they can achieve competitive performance on heterogeneous textual data even when trained on only a limited subset of the extensive annotated corpora that are typically deemed necessary.
Abstract (translated)
大型语言模型（LLM）在金融市场分析中扮演着越来越重要的角色，通过捕捉来自复杂且异质化的文本数据源（如推特、新闻文章、报告和微博等）的信号来发挥作用。然而，其性能依赖于大规模计算资源和专有数据集，这些资源成本高昂且受到限制，因此许多研究人员和从业者无法获取。为了反映实际情况，我们研究了轻量级开源LLM的能力——即那些设计用于有限计算资源的小型公开模型，在不同大小、来源、格式和语言的金融数据集中进行情感理解泛化的能力。我们在五个公共数据集上对基准金融自然语言处理（NLP）模型FinBERT以及三个开源轻量级LLM进行了比较，这些数据集包括FinancialPhraseBank、FinancialQuestionAnswering、GoldNewsSentiment、TwitterSentiment和ChineseFinanceSentiment。我们发现，在大多数情况下，特别是使用仅5%的可用训练数据时，LLMs表现最佳，尤其是Qwen3 8B和Llama3 8B。在零样本学习（zero-shot）和少量样例学习（few-shot）场景中，这些结果仍然成立。我们的研究结果表明，轻量级开源大型语言模型构成了一种成本效益高的选择，即使在训练数据仅限于通常被认为必要的广泛注释语料库的有限子集时，它们也能实现与同类模型相竞争的表现。
URL
https://arxiv.org/abs/2512.00946
PDF
-
Developing a Comprehensive Framework for Sentiment Analysis in Turkish
2025-11-29 15:14:57Cem Rifki AydinAbstract
In this thesis, we developed a comprehensive framework for sentiment analysis that takes its many aspects into account mainly for Turkish. We have also proposed several approaches specific to sentiment analysis in English only. We have accordingly made five major and three minor contributions. We generated a novel and effective feature set by combining unsupervised, semi-supervised, and supervised metrics. We then fed them as input into classical machine learning methods, and outperformed neural network models for datasets of different genres in both Turkish and English. We created a polarity lexicon with a semi-supervised domain-specific method, which has been the first approach applied for corpora in Turkish. We performed a fine morphological analysis for the sentiment classification task in Turkish by determining the polarities of morphemes. This can be adapted to other morphologically-rich or agglutinative languages as well. We have built a novel neural network architecture, which combines recurrent and recursive neural network models for English. We built novel word embeddings that exploit sentiment, syntactic, semantic, and lexical characteristics for both Turkish and English. We also redefined context windows as subclauses in modelling word representations in English. This can also be applied to other linguistic fields and natural language processing tasks. We have achieved state-of-the-art and significant results for all these original approaches. Our minor contributions include methods related to aspect-based sentiment in Turkish, parameter redefinition in the semi-supervised approach, and aspect term extraction techniques for English. This thesis can be considered the most detailed and comprehensive study made on sentiment analysis in Turkish as of July, 2020. Our work has also contributed to the opinion classification problem in English.
Abstract (translated)
在这篇论文中，我们开发了一个全面的框架用于情感分析，该框架充分考虑了情感分析的各种方面，主要针对土耳其语。此外，我们也提出了一些仅适用于英语的情感分析特定方法。根据这些研究，我们做出了五项重大贡献和三项次要贡献。 1. 我们生成了一套新颖且有效的特征集合，通过结合无监督、半监督和有监督度量来实现。 2. 随后将这些特征作为输入传递给传统的机器学习方法，并在土耳其语和英语不同类型的语料库中超过了神经网络模型的表现。 3. 利用一种半监督领域的特定方法创建了极性词典，这是首次应用于土耳其语语料库的方法。 4. 对于土耳其语的情感分类任务进行了精细的形态学分析，确定了词缀的情感极性。这一方法也可以适应其他富形态或黏着语系的语言。 5. 我们构建了一种新型神经网络架构，将循环和递归神经网络模型相结合用于英语。 此外，我们还创建了针对土耳其语和英语的新颖词汇嵌入技术，这些技术利用情感、句法、语义及词汇特性。另外，我们在构建英语单词表示时重新定义了上下文窗口为子句结构，该方法同样适用于其他语言学领域和自然语言处理任务。 对于所有上述原创方法，我们取得了最先进的成果，并具有显著意义。次要贡献包括土耳其语基于方面的情感分析方法、半监督方法中的参数重定义以及英语的方面术语提取技术。 截至2020年7月，这篇论文可被视为对土耳其语情感分析进行最详细和全面研究的工作之一。我们的工作还为英语的意见分类问题做出了贡献。
URL
https://arxiv.org/abs/2512.00515
PDF
-
BanglaSentNet: An Explainable Hybrid Deep Learning Framework for Multi-Aspect Sentiment Analysis with Cross-Domain Transfer Learning
2025-11-28 15:17:22Ariful Islam, Md Rifat Hossen, Tanvir MahmudAbstract
Multi-aspect sentiment analysis of Bangla e-commerce reviews remains challenging due to limited annotated datasets, morphological complexity, code-mixing phenomena, and domain shift issues, affecting 300 million Bangla-speaking users. Existing approaches lack explainability and cross-domain generalization capabilities crucial for practical deployment. We present BanglaSentNet, an explainable hybrid deep learning framework integrating LSTM, BiLSTM, GRU, and BanglaBERT through dynamic weighted ensemble learning for multi-aspect sentiment classification. We introduce a dataset of 8,755 manually annotated Bangla product reviews across four aspects (Quality, Service, Price, Decoration) from major Bangladeshi e-commerce platforms. Our framework incorporates SHAP-based feature attribution and attention visualization for transparent insights. BanglaSentNet achieves 85% accuracy and 0.88 F1-score, outperforming standalone deep learning models by 3-7% and traditional approaches substantially. The explainability suite achieves 9.4/10 interpretability score with 87.6% human agreement. Cross-domain transfer learning experiments reveal robust generalization: zero-shot performance retains 67-76% effectiveness across diverse domains (BanglaBook reviews, social media, general e-commerce, news headlines); few-shot learning with 500-1000 samples achieves 90-95% of full fine-tuning performance, significantly reducing annotation costs. Real-world deployment demonstrates practical utility for Bangladeshi e-commerce platforms, enabling data-driven decision-making for pricing optimization, service improvement, and customer experience enhancement. This research establishes a new state-of-the-art benchmark for Bangla sentiment analysis, advances ensemble learning methodologies for low-resource languages, and provides actionable solutions for commercial applications.
Abstract (translated)
URL
https://arxiv.org/abs/2511.23264
PDF
-
Sentiment Analysis Of Shopee Product Reviews Using Distilbert
2025-11-27 10:43:51Zahri Aksa Dautd, Aviv Yuniar RahmanAbstract
The rapid growth of digital commerce has led to the accumulation of a massive number of consumer reviews on online platforms. Shopee, as one of the largest e-commerce platforms in Southeast Asia, receives millions of product reviews every day containing valuable information regarding customer satisfaction and preferences. Manual analysis of these reviews is inefficient, thus requiring a computational approach such as sentiment analysis. This study examines the use of DistilBERT, a lightweight transformer-based deep learning model, for sentiment classification on Shopee product reviews. The dataset used consists of approximately one million English-language reviews that have been preprocessed and trained using the distilbert-base-uncased model. Evaluation was conducted using accuracy, precision, recall, and F1-score metrics, and compared against benchmark models such as BERT and SVM. The results show that DistilBERT achieved an accuracy of 94.8%, slightly below BERT (95.3%) but significantly higher than SVM (90.2%), with computation time reduced by more than 55%. These findings demonstrate that DistilBERT provides an optimal balance between accuracy and efficiency, making it suitable for large scale sentiment analysis on e-commerce platforms. Keywords: Sentiment Analysis, DistilBERT, Shopee Reviews, Natural Language Processing, Deep Learning, Transformer Models.
Abstract (translated)
URL
https://arxiv.org/abs/2511.22313
PDF
-
SpaceX: Exploring metrics with the SPACE model for developer productivity
2025-11-26 01:21:43Sanchit Kaul, Kevin Nhu, Jason Eissayou, Ivan Eser, Victor BorupAbstract
This empirical investigation elucidates the limitations of deterministic, unidimensional productivity heuristics by operationalizing the SPACE framework through extensive repository mining. Utilizing a dataset derived from open-source repositories, the study employs rigorous statistical methodologies including Generalized Linear Mixed Models (GLMM) and RoBERTa-based sentiment classification to synthesize a holistic, multi-faceted productivity metric. Analytical results reveal a statistically significant positive correlation between negative affective states and commit frequency, implying a cycle of iterative remediation driven by frustration. Furthermore, the investigation has demonstrated that analyzing the topology of contributor interactions yields superior fidelity in mapping collaborative dynamics compared to traditional volume-based metrics. Ultimately, this research posits a Composite Productivity Score (CPS) to address the heterogeneity of developer efficacy.
Abstract (translated)
URL
https://arxiv.org/abs/2511.20955
PDF
-
Prompt Optimization as a State-Space Search Problem
2025-11-23 21:24:13Maanas TanejaAbstract
Language Models are extremely susceptible to performance collapse with even small changes to input prompt strings. Libraries such as DSpy (from Stanford NLP) avoid this problem through demonstration-based prompt optimisation. Inspired by this, I propose an alternative approach that treats prompt optimisation as a classical state-space search problem. I model the prompt space as a graph where nodes represent prompt states and edges correspond to deliberate transformations such as shortening, adding examples, or re- ordering content. Using beam search and random walk algorithms, I systematically explore this space, evaluating candidates on development sets and pruning unpromising branches. Across five NLP tasks (sentiment classification, question answering, summarisation, reason- ing, and natural language inference), I find that even shallow search configurations (beam width=2, depth=2) improve upon seed prompts on development sets. For instance, beam search achieves development accuracy gains from 0.40 to 0.80 on reasoning tasks, though test set improvements are more modest (0.20 to 0.50), indicating overfitting to the develop- ment heuristic. Analysis of successful optimisation paths reveals that transformations that make prompts concise appear most frequently, while verbosity operators are never selected. My results validate prompt optimization as a search problem and suggest that with greater computational resources and improved evaluation metrics, deeper exploration could yield more robust prompts that generalize beyond development sets. Code and implementation are available at [this https URL].
Abstract (translated)
URL
https://arxiv.org/abs/2511.18619
PDF
-
Semantics as a Shield: Label Disguise Defense against Prompt Injection in LLM Sentiment Classification
2025-11-23 20:16:51Yanxi Li, Ruocheng ShanAbstract
Large language models are increasingly used for text classification tasks such as sentiment analysis, yet their reliance on natural language prompts exposes them to prompt injection attacks. In particular, class-directive injections exploit knowledge of the model's label set (e.g., positive vs. negative) to override its intended behavior through adversarial instructions. Existing defenses, such as detection-based filters, instruction hierarchies, and signed prompts, either require model retraining or remain vulnerable to obfuscation. This paper introduces Label Disguise Defense (LDD), a lightweight and model-agnostic strategy that conceals true labels by replacing them with semantically transformed or unrelated alias labels(e.g., blue vs. yellow). The model learns these new label mappings implicitly through few-shot demonstrations, preventing direct correspondence between injected directives and decision outputs. We evaluate LDD across nine state-of-the-art models, including GPT-5, GPT-4o, LLaMA3.2, Gemma3, and Mistral variants, under varying few-shot and an adversarial setting. Our results show that the ability of LDD to recover performance lost to the adversarial attack varies across models and alias choices. For every model evaluated, LDD is able to restore a portion of the accuracy degradation caused by the attack. Moreover, for the vast majority of models, we can identify more than one alias pair that achieves higher accuracy than the under-attack baseline, in which the model relies solely on few-shot learning without any defensive mechanism. A linguistic analysis further reveals that semantically aligned alias labels(e.g., good vs. bad) yield stronger robustness than unaligned symbols(e.g., blue vs. yellow). Overall, this study demonstrates that label semantics can serve as an effective defense layer, transforming meaning itself into a shield against prompt injection.
Abstract (translated)
URL
https://arxiv.org/abs/2511.21752
PDF
-
Large Language Models for Sentiment Analysis to Detect Social Challenges: A Use Case with South African Languages
2025-11-21 15:14:32Koena Ronny Mabokela, Tim Schlippe, Matthias W\"olfelAbstract
Sentiment analysis can aid in understanding people's opinions and emotions on social issues. In multilingual communities sentiment analysis systems can be used to quickly identify social challenges in social media posts, enabling government departments to detect and address these issues more precisely and effectively. Recently, large-language models (LLMs) have become available to the wide public and initial analyses have shown that they exhibit magnificent zero-shot sentiment analysis abilities in English. However, there is no work that has investigated to leverage LLMs for sentiment analysis on social media posts in South African languages and detect social challenges. Consequently, in this work, we analyse the zero-shot performance of the state-of-the-art LLMs GPT-3.5, GPT-4, LlaMa 2, PaLM 2, and Dolly 2 to investigate the sentiment polarities of the 10 most emerging topics in English, Sepedi and Setswana social media posts that fall within the jurisdictional areas of 10 South African government departments. Our results demonstrate that there are big differences between the various LLMs, topics, and languages. In addition, we show that a fusion of the outcomes of different LLMs provides large gains in sentiment classification performance with sentiment classification errors below 1%. Consequently, it is now feasible to provide systems that generate reliable information about sentiment analysis to detect social challenges and draw conclusions about possible needs for actions on specific topics and within different language groups.
Abstract (translated)
URL
https://arxiv.org/abs/2511.17301
PDF
-
Based on Data Balancing and Model Improvement for Multi-Label Sentiment Classification Performance Enhancement
2025-11-18 03:06:27Zijin Su, Huanzhu Lv, Yuren Niu, Yiming LiuAbstract
Multi-label sentiment classification plays a vital role in natural language processing by detecting multiple emotions within a single text. However, existing datasets like GoEmotions often suffer from severe class imbalance, which hampers model performance, especially for underrepresented emotions. To address this, we constructed a balanced multi-label sentiment dataset by integrating the original GoEmotions data, emotion-labeled samples from Sentiment140 using a RoBERTa-base-GoEmotions model, and manually annotated texts generated by GPT-4 mini. Our data balancing strategy ensured an even distribution across 28 emotion categories. Based on this dataset, we developed an enhanced multi-label classification model that combines pre-trained FastText embeddings, convolutional layers for local feature extraction, bidirectional LSTM for contextual learning, and an attention mechanism to highlight sentiment-relevant words. A sigmoid-activated output layer enables multi-label prediction, and mixed precision training improves computational efficiency. Experimental results demonstrate significant improvements in accuracy, precision, recall, F1-score, and AUC compared to models trained on imbalanced data, highlighting the effectiveness of our approach.
Abstract (translated)
URL
https://arxiv.org/abs/2511.14073
PDF
-
Aspect-Level Obfuscated Sentiment in Thai Financial Disclosures and Its Impact on Abnormal Returns
2025-11-17 15:17:46Attapol T. Rutherford, Sirisak Chueykamhang, Thachaparn Bunditlurdruk, Nanthicha AngsuwichitkulAbstract
Understanding sentiment in financial documents is crucial for gaining insights into market behavior. These reports often contain obfuscated language designed to present a positive or neutral outlook, even when underlying conditions may be less favorable. This paper presents a novel approach using Aspect-Based Sentiment Analysis (ABSA) to decode obfuscated sentiment in Thai financial annual reports. We develop specific guidelines for annotating obfuscated sentiment in these texts and annotate more than one hundred financial reports. We then benchmark various text classification models on this annotated dataset, demonstrating strong performance in sentiment classification. Additionally, we conduct an event study to evaluate the real-world implications of our sentiment analysis on stock prices. Our results suggest that market reactions are selectively influenced by specific aspects within the reports. Our findings underscore the complexity of sentiment analysis in financial texts and highlight the importance of addressing obfuscated language to accurately assess market sentiment.
Abstract (translated)
URL
https://arxiv.org/abs/2511.13481
PDF
-
Opinion Mining and Analysis Using Hybrid Deep Neural Networks
2025-11-16 19:04:44Adel Hidri, Suleiman Ali Alsaif, Muteeb Alahmari, Eman AlShehri, Minyar Sassi HidriAbstract
Understanding customer attitudes has become a critical component of decision-making due to the growing influence of social media and e-commerce. Text-based opinions are the most structured, hence playing an important role in sentiment analysis. Most of the existing methods, which include lexicon-based approaches and traditional machine learning techniques, are insufficient for handling contextual nuances and scalability. While the latter has limitations in model performance and generalization, deep learning (DL) has achieved improvement, especially on semantic relationship capturing with recurrent neural networks (RNNs) and convolutional neural networks (CNNs). The aim of the study is to enhance opinion mining by introducing a hybrid deep neural network model that combines a bidirectional gated recurrent unit (BGRU) and long short-term memory (LSTM) layers to improve sentiment analysis, particularly addressing challenges such as contextual nuance, scalability, and class imbalance. To substantiate the efficacy of the proposed model, we conducted comprehensive experiments utilizing benchmark datasets, encompassing IMDB movie critiques and Amazon product evaluations. The introduced hybrid BGRULSTM (HBGRU-LSTM) architecture attained a testing accuracy of 95%, exceeding the performance of traditional DL frameworks such as LSTM (93.06%), CNN+LSTM (93.31%), and GRU+LSTM (92.20%). Moreover, our model exhibited a noteworthy enhancement in recall for negative sentiments, escalating from 86% (unbalanced dataset) to 96% (balanced dataset), thereby ensuring a more equitable and just sentiment classification. Furthermore, the model diminished misclassification loss from 20.24% for unbalanced to 13.3% for balanced dataset, signifying enhanced generalization and resilience.
Abstract (translated)
URL
https://arxiv.org/abs/2511.14796
PDF
-
State of the Art in Text Classification for South Slavic Languages: Fine-Tuning or Prompting?
2025-11-11 08:54:15Taja Kuzman Punger\v{s}ek, Peter Rupnik, Ivan Porupski, Vuk Dini\'c, Nikola Ljube\v{s}i\'cAbstract
Until recently, fine-tuned BERT-like models provided state-of-the-art performance on text classification tasks. With the rise of instruction-tuned decoder-only models, commonly known as large language models (LLMs), the field has increasingly moved toward zero-shot and few-shot prompting. However, the performance of LLMs on text classification, particularly on less-resourced languages, remains under-explored. In this paper, we evaluate the performance of current language models on text classification tasks across several South Slavic languages. We compare openly available fine-tuned BERT-like models with a selection of open-source and closed-source LLMs across three tasks in three domains: sentiment classification in parliamentary speeches, topic classification in news articles and parliamentary speeches, and genre identification in web texts. Our results show that LLMs demonstrate strong zero-shot performance, often matching or surpassing fine-tuned BERT-like models. Moreover, when used in a zero-shot setup, LLMs perform comparably in South Slavic languages and English. However, we also point out key drawbacks of LLMs, including less predictable outputs, significantly slower inference, and higher computational costs. Due to these limitations, fine-tuned BERT-like models remain a more practical choice for large-scale automatic text annotation.
Abstract (translated)
URL
https://arxiv.org/abs/2511.07989
PDF
-
CC30k: A Citation Contexts Dataset for Reproducibility-Oriented Sentiment Analysis
2025-11-11 03:13:17Rochana R. Obadage, Sarah M. Rajtmajer, Jian WuAbstract
Sentiments about the reproducibility of cited papers in downstream literature offer community perspectives and have shown as a promising signal of the actual reproducibility of published findings. To train effective models to effectively predict reproducibility-oriented sentiments and further systematically study their correlation with reproducibility, we introduce the CC30k dataset, comprising a total of 30,734 citation contexts in machine learning papers. Each citation context is labeled with one of three reproducibility-oriented sentiment labels: Positive, Negative, or Neutral, reflecting the cited paper's perceived reproducibility or replicability. Of these, 25,829 are labeled through crowdsourcing, supplemented with negatives generated through a controlled pipeline to counter the scarcity of negative labels. Unlike traditional sentiment analysis datasets, CC30k focuses on reproducibility-oriented sentiments, addressing a research gap in resources for computational reproducibility studies. The dataset was created through a pipeline that includes robust data cleansing, careful crowd selection, and thorough validation. The resulting dataset achieves a labeling accuracy of 94%. We then demonstrated that the performance of three large language models significantly improves on the reproducibility-oriented sentiment classification after fine-tuning using our dataset. The dataset lays the foundation for large-scale assessments of the reproducibility of machine learning papers. The CC30k dataset and the Jupyter notebooks used to produce and analyze the dataset are publicly available at this https URL .
Abstract (translated)
URL
https://arxiv.org/abs/2511.07790
PDF
-
Multi-refined Feature Enhanced Sentiment Analysis Using Contextual Instruction
2025-11-01 12:46:33Peter Atandoh, Jie Zou, Weikang Guo, Jiwei Wei, Zheng WangAbstract
Sentiment analysis using deep learning and pre-trained language models (PLMs) has gained significant traction due to their ability to capture rich contextual representations. However, existing approaches often underperform in scenarios involving nuanced emotional cues, domain shifts, and imbalanced sentiment distributions. We argue that these limitations stem from inadequate semantic grounding, poor generalization to diverse linguistic patterns, and biases toward dominant sentiment classes. To overcome these challenges, we propose CISEA-MRFE, a novel PLM-based framework integrating Contextual Instruction (CI), Semantic Enhancement Augmentation (SEA), and Multi-Refined Feature Extraction (MRFE). CI injects domain-aware directives to guide sentiment disambiguation; SEA improves robustness through sentiment-consistent paraphrastic augmentation; and MRFE combines a Scale-Adaptive Depthwise Encoder (SADE) for multi-scale feature specialization with an Emotion Evaluator Context Encoder (EECE) for affect-aware sequence modeling. Experimental results on four benchmark datasets demonstrate that CISEA-MRFE consistently outperforms strong baselines, achieving relative improvements in accuracy of up to 4.6% on IMDb, 6.5% on Yelp, 30.3% on Twitter, and 4.1% on Amazon. These results validate the effectiveness and generalization ability of our approach for sentiment classification across varied domains.
Abstract (translated)
URL
https://arxiv.org/abs/2511.00537
PDF
-
Effect of Domain Generalization Techniques in Low Resource Systems
2025-10-31 14:38:27Mahi Aminu, Chisom Chibuike, Fatimo Adebanjo, Omokolade Awosanya, Samuel OyeneyeAbstract
Machine learning models typically assume that training and test data follow the same distribution, an assumption that often fails in real-world scenarios due to distribution shifts. This issue is especially pronounced in low-resource settings, where data scarcity and limited domain diversity hinder robust generalization. Domain generalization (DG) approaches address this challenge by learning features that remain invariant across domains, often using causal mechanisms to improve model robustness. In this study, we examine two distinct causal DG techniques in low-resource natural language tasks. First, we investigate a causal data augmentation (CDA) approach that automatically generates counterfactual examples to improve robustness to spurious correlations. We apply this method to sentiment classification on the NaijaSenti Twitter corpus, expanding the training data with semantically equivalent paraphrases to simulate controlled distribution shifts. Second, we explore an invariant causal representation learning (ICRL) approach using the DINER framework, originally proposed for debiasing aspect-based sentiment analysis. We adapt DINER to a multilingual setting. Our findings demonstrate that both approaches enhance robustness to unseen domains: counterfactual data augmentation yields consistent cross-domain accuracy gains in sentiment classification, while causal representation learning with DINER improves out-of-distribution performance in multilingual sentiment analysis, albeit with varying gains across languages.
Abstract (translated)
URL
https://arxiv.org/abs/2510.27512
PDF
-
Bayesian Network Fusion of Large Language Models for Sentiment Analysis
2025-10-30 13:37:58Rasoul Amirzadeh, Dhananjay Thiruvady, Fatemeh ShiriAbstract
Large language models (LLMs) continue to advance, with an increasing number of domain-specific variants tailored for specialised tasks. However, these models often lack transparency and explainability, can be costly to fine-tune, require substantial prompt engineering, yield inconsistent results across domains, and impose significant adverse environmental impact due to their high computational demands. To address these challenges, we propose the Bayesian network LLM fusion (BNLF) framework, which integrates predictions from three LLMs, including FinBERT, RoBERTa, and BERTweet, through a probabilistic mechanism for sentiment analysis. BNLF performs late fusion by modelling the sentiment predictions from multiple LLMs as probabilistic nodes within a Bayesian network. Evaluated across three human-annotated financial corpora with distinct linguistic and contextual characteristics, BNLF demonstrates consistent gains of about six percent in accuracy over the baseline LLMs, underscoring its robustness to dataset variability and the effectiveness of probabilistic fusion for interpretable sentiment classification.
Abstract (translated)
URL
https://arxiv.org/abs/2510.26484
PDF
-
HalleluBERT: Let every token that has meaning bear its weight
2025-10-24 11:52:29Raphael Scheible-SchmittAbstract
Transformer-based models have advanced NLP, yet Hebrew still lacks a large-scale RoBERTa encoder which is extensively trained. Existing models such as HeBERT, AlephBERT, and HeRo are limited by corpus size, vocabulary, or training depth. We present HalleluBERT, a RoBERTa-based encoder family (base and large) trained from scratch on 49.1~GB of deduplicated Hebrew web text and Wikipedia with a Hebrew-specific byte-level BPE vocabulary. Evaluated on NER and sentiment classification benchmarks, HalleluBERT outperforms both monolingual and multilingual baselines. HalleluBERT sets a new state of the art for Hebrew and highlights the benefits of fully converged monolingual pretraining.
Abstract (translated)
URL
https://arxiv.org/abs/2510.21372
PDF
-
Style Attack Disguise: When Fonts Become a Camouflage for Adversarial Intent
2025-10-22 14:40:24Yangshijie Zhang, Xinda Wang, Jialin Liu, Wenqiang Wang, Zhicong Ma, Xingxing JiaAbstract
With social media growth, users employ stylistic fonts and font-like emoji to express individuality, creating visually appealing text that remains human-readable. However, these fonts introduce hidden vulnerabilities in NLP models: while humans easily read stylistic text, models process these characters as distinct tokens, causing interference. We identify this human-model perception gap and propose a style-based attack, Style Attack Disguise (SAD). We design two sizes: light for query efficiency and strong for superior attack performance. Experiments on sentiment classification and machine translation across traditional models, LLMs, and commercial services demonstrate SAD's strong attack performance. We also show SAD's potential threats to multimodal tasks including text-to-image and text-to-speech generation.
Abstract (translated)
随着社交媒体的发展，用户开始使用风格化的字体和类似表情符号的字符来表达个性，创造出既美观又易于人类阅读的文字。然而，这些风格化文本引入了自然语言处理（NLP）模型中的隐秘漏洞：虽然人类可以轻松读取风格化文字，但模型会将这些特殊字符视为独立的标记进行处理，从而导致干扰。我们识别出这种人机感知差距，并提出了一种基于样式的攻击方法——样式攻击伪装（SAD）。我们设计了两种版本：一种轻量级用于查询效率，另一种强力版则在攻击性能上更胜一筹。 实验表明，在传统的模型、大型语言模型和商业服务上的情感分类及机器翻译任务中，SAD展现出了强大的攻击效果。此外，我们也展示了SAD对多模态任务（如文本到图像生成和文本到语音合成）潜在威胁的证据。
URL
https://arxiv.org/abs/2510.19641
PDF
-
Topoformer: brain-like topographic organization in Transformer language models through spatial querying and reweighting
2025-10-21 15:54:57Taha Binhuraib, Greta Tuckute, Nicholas BlauchAbstract
Spatial functional organization is a hallmark of biological brains: neurons are arranged topographically according to their response properties, at multiple scales. In contrast, representations within most machine learning models lack spatial biases, instead manifesting as disorganized vector spaces that are difficult to visualize and interpret. Here, we propose a novel form of self-attention that turns Transformers into "Topoformers" with topographic organization. We introduce spatial querying - where keys and queries are arranged on 2D grids, and local pools of queries are associated with a given key - and spatial reweighting, where we convert the standard fully connected layer of self-attention into a locally connected layer. We first demonstrate the feasibility of our approach by training a 1-layer Topoformer on a sentiment classification task. Training with spatial querying encourages topographic organization in the queries and keys, and spatial reweighting separately encourages topographic organization in the values and self-attention outputs. We then apply the Topoformer motifs at scale, training a BERT architecture with a masked language modeling objective. We find that the topographic variant performs on par with a non-topographic control model on NLP benchmarks, yet produces interpretable topographic organization as evaluated via eight linguistic test suites. Finally, analyzing an fMRI dataset of human brain responses to a large set of naturalistic sentences, we demonstrate alignment between low-dimensional topographic variability in the Topoformer model and human brain language network. Scaling up Topoformers further holds promise for greater interpretability in NLP research, and for more accurate models of the organization of linguistic information in the human brain.
Abstract (translated)
空间功能组织是生物大脑的一个显著特征：根据神经元的响应特性，它们在多个尺度上按拓扑结构排列。相比之下，大多数机器学习模型中的表示缺乏空间偏差，反而表现为难以可视化和解释的无序向量空间。在这里，我们提出了一种新颖的自我注意形式，将Transformer转化为具有拓扑组织的"Topoformer"。我们引入了空间查询——其中键和查询在二维网格上排列，并且特定键与局部查询池相关联——以及空间重新加权，即将自注意力的标准全连接层转换为局部连接层。首先，我们通过在一个情感分类任务上训练一个单层的Topoformer来证明我们的方法的可行性。使用空间查询进行训练鼓励在查询和键中形成拓扑组织，并且空间重新加权单独地促进了值和自我注意输出中的拓扑组织。然后，我们在大规模上应用了Topoformer模式，在BERT架构的基础上以掩码语言建模为目标进行了训练。我们发现，拓扑变体在NLP基准测试中与非拓扑对照模型的表现相当，但通过八套语言测试评估时产生了可解释的拓扑结构。最后，通过对大量自然句子的人脑功能磁共振成像（fMRI）数据集进行分析，我们展示了Topoformer模型中的低维拓扑变化与人类大脑语言网络之间的对齐关系。扩大Topoformers的规模有望进一步提高NLP研究的可解释性，并为更好地描述人脑中语言信息组织提供更准确的模型。
URL
https://arxiv.org/abs/2510.18745
PDF
