Generative large language models (LLMs) are now used extensively for text classification in computational social science (CSS). In this work, focus on the steps before and after LLM prompting -- conceptualization of concepts to be classified and using LLM predictions in downstream statistical inference -- which we argue have been overlooked in much of LLM-era CSS. We claim LLMs can tempt analysts to skip the conceptualization step, creating conceptualization errors that bias downstream estimates. Using simulations, we show that this conceptualization-induced bias cannot be corrected for solely by increasing LLM accuracy or post-hoc bias correction methods. We conclude by reminding CSS analysts that conceptualization is still a first-order concern in the LLM-era and provide concrete advice on how to pursue low-cost, unbiased, low-variance downstream estimates.
生成式大型语言模型(LLM)现在广泛用于计算社会科学(CSS)中的文本分类。在这项工作中,我们重点关注在LLM提示之前和之后的步骤——即被分类概念的概念化过程以及将LLM预测应用于下游统计推断的过程,我们认为这些环节在大多数使用LLM的时代CSS研究中被忽略了。我们主张,LLM可能会诱使分析人员跳过概念化的步骤,从而产生概念错误,进而导致后续估计结果出现偏差。通过模拟实验,我们展示了仅靠提高LLM的准确性或采用事后偏置校正方法是不足以纠正这种由概念化引起的偏差的。最后,我们提醒CSS分析师,在LLM时代,概念化仍然是一个首要关注点,并提供了如何追求低成本、无偏且方差低的下游估计的具体建议。
https://arxiv.org/abs/2510.03541
Large Language Models (LLMs) have shown extraordinary success across various text generation tasks; however, their potential for simple yet essential text classification remains underexplored, as LLM pre-training tends to emphasize generation over classification. While LLMs with instruction tuning can transform classification into a generation task, they often struggle to categorize nuanced texts. One such example is text revision, which involves nuanced edits between pairs of texts. Although simply fine-tuning LLMs for revision classification seems plausible, it requires a large amount of revision annotations, which are exceptionally expensive and scarce in the community. To address this issue, we introduce a plug-and-play layer-wise parameter-efficient fine-tuning (PEFT) framework, i.e., IR-Tuning, which fine-tunes a subset of important LLM layers that are dynamically selected based on their gradient norm distribution, while freezing those of redundant layers. Extensive experiments suggest that IR-Tuning surpasses several layer-wise PEFT baselines over diverse text revisions, while achieving fast convergence, low GPU memory consumption, and effectiveness on small revision corpora.
大型语言模型(LLMs)在各种文本生成任务中展示了非凡的成功;然而,它们对于简单而基本的文本分类的潜力尚未得到充分探索,因为大规模预训练倾向于强调生成而非分类。虽然通过指令微调可以使LLM将分类问题转化为生成任务,但它们常常难以对细微差别较大的文本进行准确分类。其中一个例子就是文本修订,这涉及到两段文本之间的微妙编辑差异。尽管简单地为LLMs的修订分类进行微调似乎是可行的,但这需要大量的修订注释,而这些数据在社区中极其昂贵且稀缺。 为了应对这一问题,我们引入了一种即插即用层级参数高效微调(PEFT)框架,即IR-Tuning。该方法选择性地对一组重要层级进行细调,这些层级是基于它们的梯度范数分布动态选定的,并冻结那些冗余层级。通过广泛的实验表明,IR-Tuning在多种文本修订任务中超越了几种层级PEFT基线模型,同时实现了快速收敛、低GPU内存消耗以及对于小型修订语料库的有效性。
https://arxiv.org/abs/2510.00268
The combination of linear transformations and non-linear activation functions forms the foundation of most modern deep neural networks, enabling them to approximate highly complex functions. This paper explores the introduction of quadratic transformations to further increase nonlinearity in neural networks, with the aim of enhancing the performance of existing architectures. To reduce parameter complexity and computational complexity, we propose a lightweight quadratic enhancer that uses low-rankness, weight sharing, and sparsification techniques. For a fixed architecture, the proposed approach introduces quadratic interactions between features at every layer, while only adding negligible amounts of additional model parameters and forward computations. We conduct a set of proof-of-concept experiments for the proposed method across three tasks: image classification, text classification, and fine-tuning large-language models. In all tasks, the proposed approach demonstrates clear and substantial performance gains.
线性变换与非线性激活函数的结合构成了大多数现代深度神经网络的基础,使它们能够近似复杂度极高的函数。本文探讨了引入二次变换以进一步增加神经网络中的非线性程度,旨在提升现有架构的表现能力。为了降低参数复杂性和计算复杂性,我们提出了一种轻量级的二次增强器,该增强器利用低秩、权重共享和稀疏化技术。对于固定的架构,在每一层中我们的方法引入了特征之间的二次交互作用,并且仅添加了几乎可以忽略不计的额外模型参数和前向计算。 我们在三个任务上进行了概念验证实验:图像分类、文本分类以及大型语言模型的微调。在所有这些任务中,所提出的方法均展示了明显的性能提升。
https://arxiv.org/abs/2510.03276
In-context learning (ICL) enables Large Language Models (LLMs) to generate predictions based on prompts without additional fine-tuning. While prompt engineering has been widely studied, the impact of role design within prompts remains underexplored. This study examines the influence of role configurations in zero-shot and few-shot learning scenarios using GPT-3.5 and GPT-4o from OpenAI and Llama2-7b and Llama2-13b from Meta. We evaluate the models' performance across datasets, focusing on tasks like sentiment analysis, text classification, question answering, and math reasoning. Our findings suggest the potential of role-based prompt structuring to enhance LLM performance.
上下文学习(ICL)使大型语言模型(LLM)能够在没有额外微调的情况下根据提示生成预测。虽然提示工程已经得到了广泛研究,但提示中角色设计的影响仍然鲜为人知。本研究通过使用来自OpenAI的GPT-3.5和GPT-4o以及来自Meta的Llama2-7b和Llama2-13b模型,在零样本和少量样本学习场景下考察了角色配置对性能的影响。我们评估了这些模型在情感分析、文本分类、问答和数学推理等任务上的表现。我们的研究结果表明,基于角色的提示结构有可能提升LLM的表现。 注意:原文中提到了“GPT-4o”,假设这是一个笔误或者是特定版本的名称,在翻译过程中保持了一致性处理,请根据实际情况进行调整。如果"GPT-4o"是专有名词或者具体指某个版本,则不需要更改。
https://arxiv.org/abs/2509.23501
We explore how large language models (LLMs) assess offensiveness in political discourse when prompted to adopt specific political and cultural perspectives. Using a multilingual subset of the MD-Agreement dataset centered on tweets from the 2020 US elections, we evaluate several recent LLMs - including DeepSeek-R1, o4-mini, GPT-4.1-mini, Qwen3, Gemma, and Mistral - tasked with judging tweets as offensive or non-offensive from the viewpoints of varied political personas (far-right, conservative, centrist, progressive) across English, Polish, and Russian contexts. Our results show that larger models with explicit reasoning abilities (e.g., DeepSeek-R1, o4-mini) are more consistent and sensitive to ideological and cultural variation, while smaller models often fail to capture subtle distinctions. We find that reasoning capabilities significantly improve both the personalization and interpretability of offensiveness judgments, suggesting that such mechanisms are key to adapting LLMs for nuanced sociopolitical text classification across languages and ideologies.
我们探讨了大型语言模型(LLMs)在被提示采用特定政治和文化视角时,如何评估政治话语中的冒犯性。使用一个多语种的MD-Agreement数据集子集,该数据集以2020年美国大选期间的推特为主,我们评估了几种最近的大型语言模型——包括DeepSeek-R1、o4-mini、GPT-4.1-mini、Qwen3、Gemma和Mistral——这些模型被要求从英语、波兰语和俄语中的不同政治人物视角(极右翼、保守派、中间派、进步派)来判断推特内容是否冒犯。我们的结果显示,具备明确推理能力的大型模型(如DeepSeek-R1、o4-mini),在捕捉意识形态和文化差异方面更为一致且敏感,而较小的模型往往无法捕捉到细微的区别。我们发现,推理能力显著提高了冒犯性评判的个性化和可解释性,这表明此类机制对于跨语言和意识形态进行复杂社会政治文本分类来说至关重要。
https://arxiv.org/abs/2510.02351
Legal Article Prediction (LAP) is a critical task in legal text classification, leveraging natural language processing (NLP) techniques to automatically predict relevant legal articles based on the fact descriptions of cases. As a foundational step in legal decision-making, LAP plays a pivotal role in determining subsequent judgments, such as charges and penalties. Despite its importance, existing methods face significant challenges in addressing the complexities of LAP. Supervised classification models (SCMs), such as CNN and BERT, struggle to fully capture intricate fact patterns due to their inherent limitations. Conversely, large language models (LLMs), while excelling in generative tasks, perform suboptimally in predictive scenarios due to the abstract and ID-based nature of legal articles. Furthermore, the diversity of legal systems across jurisdictions exacerbates the issue, as most approaches are tailored to specific countries and lack broader applicability. To address these limitations, we propose Uni-LAP, a universal framework for legal article prediction that integrates the strengths of SCMs and LLMs through tight collaboration. Specifically, in Uni-LAP, the SCM is enhanced with a novel Top-K loss function to generate accurate candidate articles, while the LLM employs syllogism-inspired reasoning to refine the final predictions. We evaluated Uni-LAP on datasets from multiple jurisdictions, and empirical results demonstrate that our approach consistently outperforms existing baselines, showcasing its effectiveness and generalizability.
法律文章预测(LAP)是法律文本分类中的关键任务,通过运用自然语言处理(NLP)技术来自动预测与案件事实描述相关的法律条文。作为法律决策的基础步骤,LAP在确定后续判决如指控和处罚方面发挥着核心作用。尽管它的重要性不言而喻,现有的方法在解决LAP的复杂性方面面临重大挑战。 监督分类模型(SCMs),例如卷积神经网络(CNN)和BERT,在捕捉复杂的事实模式时往往力有未逮,这主要是由于它们自身的局限性所致。另一方面,大型语言模型(LLMs)虽然在生成任务中表现出色,但在预测场景中的表现却不如人意,这是由法律条文的抽象性和基于ID的特点决定的。此外,不同法域之间法律系统的多样性进一步加剧了这一问题,因为大多数方法都是针对特定国家设计的,并不具备广泛的适用性。 为了解决这些局限性,我们提出了Uni-LAP,这是一个用于法律文章预测的通用框架,通过紧密合作整合了SCMs和LLMs的优势。具体而言,在Uni-LAP中,监督分类模型利用一种新颖的Top-K损失函数来生成准确的候选条文,而大型语言模型则运用类比于三段论的推理方法来优化最终预测结果。 我们在来自多个法域的数据集上评估了Uni-LAP,并通过实验证明我们的方法在各种基准测试中均表现优异,展示了其有效性和普适性。
https://arxiv.org/abs/2509.22119
AutoIntent is an automated machine learning tool for text classification tasks. Unlike existing solutions, AutoIntent offers end-to-end automation with embedding model selection, classifier optimization, and decision threshold tuning, all within a modular, sklearn-like interface. The framework is designed to support multi-label classification and out-of-scope detection. AutoIntent demonstrates superior performance compared to existing AutoML tools on standard intent classification datasets and enables users to balance effectiveness and resource consumption.
AutoIntent 是一种用于文本分类任务的自动化机器学习工具。与现有的解决方案不同,AutoIntent 提供了端到端的自动化功能,包括嵌入模型选择、分类器优化和决策阈值调整,并且在模块化界面中操作,类似于 sklearn 接口。该框架设计支持多标签分类和未定义范围检测。在标准意图分类数据集上,AutoIntent 的表现优于现有的 AutoML 工具,并使用户能够平衡效率与资源消耗。
https://arxiv.org/abs/2509.21138
Attention-based models have become the leading approach in modeling medical language for Natural Language Processing (NLP) in clinical notes. These models outperform traditional techniques by effectively capturing contextual rep- resentations of language. In this research a comparative analysis is done amongst pre- trained attention based models namely Bert Base, BioBert, two variations of Bio+Clinical Bert, RoBerta, and Clinical Long- former on task related to Electronic Health Record (EHR) information extraction. The tasks from Track 1 of Harvard Medical School's 2022 National Clinical NLP Challenges (n2c2) are considered for this comparison, with the Contextualized Medication Event Dataset (CMED) given for these task. CMED is a dataset of unstructured EHRs and annotated notes that contain task relevant information about the EHRs. The goal of the challenge is to develop effective solutions for extracting contextual information related to patient medication events from EHRs using data driven methods. Each pre-trained model is fine-tuned and applied on CMED to perform medication extraction, medical event detection, and multi-dimensional medication event context classification. Pro- cessing methods are also detailed for breaking down EHRs for compatibility with the applied models. Performance analysis has been carried out using a script based on constructing medical terms from the evaluation portion of CMED with metrics including recall, precision, and F1-Score. The results demonstrate that models pre-trained on clinical data are more effective in detecting medication and medication events, but Bert Base, pre- trained on general domain data showed to be the most effective for classifying the context of events related to medications.
基于注意力的模型已经成为自然语言处理(NLP)在临床笔记中建模医学语言的领先方法。这些模型通过有效捕捉语言的上下文表示,超越了传统的技术。在这项研究中,对几个预训练的注意力模型进行比较分析,包括Bert Base、BioBert、两种不同的Bio+Clinical Bert变体、RoBerta和Clinical LongFormer,在与电子健康记录(EHR)信息提取相关的任务上进行了对比。这些任务来自哈佛医学院2022年全国临床NLP挑战赛(n2c2)的Track 1,使用的数据集是上下文化的药物事件数据集(CMED),该数据集包含未结构化和标注良好的EHR注释,其中包含了与EHR相关的任务相关信息。比赛的目标是开发有效的方法从EHR中提取患者用药情况的相关信息,并采用基于数据驱动的方法。 每个预训练模型都在CMED上进行了微调并应用于药物提取、医疗事件检测以及多维度的药物事件上下文分类。还详细说明了处理方法,用于分解EHR以使其与应用模型兼容。性能分析使用了一个基于从CMED评估部分构建医学术语的脚本进行,该脚本采用了包括召回率(recall)、精确度(precision)和F1分数在内的指标。 结果显示,在检测药物及药物事件方面,经过临床数据预训练的模型更有效;然而,在分类与药物相关的事件上下文时,通用领域数据上预训练的Bert Base被证明是最有效的。
https://arxiv.org/abs/2509.19224
As large language models (LLMs) are trained on increasingly diverse and extensive multilingual corpora, they demonstrate cross-lingual transfer capabilities. However, these capabilities often fail to effectively extend to low-resource languages, particularly those utilizing non-Latin scripts. While transliterating low-resource languages into Latin script presents a natural solution, there currently lacks a comprehensive framework for integrating transliteration into LLMs training and deployment. Taking a pragmatic approach, this paper innovatively combines character transliteration with Huffman coding to design a complete transliteration framework. Our proposed framework offers the following advantages: 1) Compression: Reduces storage requirements for low-resource language content, achieving up to 50% reduction in file size and 50-80% reduction in token count. 2) Accuracy: Guarantees 100% lossless conversion from transliterated text back to the source language. 3) Efficiency: Eliminates the need for vocabulary expansion for low-resource languages, improving training and inference efficiency. 4) Scalability: The framework can be extended to other low-resource languages. We validate the effectiveness of our framework across multiple downstream tasks, including text classification, machine reading comprehension, and machine translation. Experimental results demonstrate that our method significantly enhances the model's capability to process low-resource languages while maintaining performance on high-resource languages. Our data and code are publicly available at this https URL.
随着大型语言模型(LLMs)在日益多样化和广泛的多语种语料库上进行训练,它们展示出了跨语言迁移能力。然而,这些能力往往未能有效地扩展到低资源语言中,尤其是那些使用非拉丁字母的语言。虽然将低资源语言转写为拉丁字母提供了一个自然的解决方案,但目前尚缺乏一个全面的框架来将这种转写集成到LLMs的训练和部署过程中。本文采用了一种实用的方法,创新性地结合了字符转写与霍夫曼编码,设计出一套完整的转写框架。 我们提出的框架具有以下优势: 1) 压缩:减少了低资源语言内容所需的存储要求,文件大小可减少高达50%,标记数量可以减少50-80%。 2) 准确性:确保从转写字体文本转换回源语言时无损。 3) 效率:消除了对低资源语言词汇表扩展的需求,提高了训练和推理的效率。 4) 可扩展性:该框架可以应用于其他低资源语言。 我们在多个下游任务中验证了我们框架的有效性,包括文本分类、机器阅读理解以及机器翻译。实验结果表明,我们的方法显著提升了模型处理低资源语言的能力,并且同时保持在高资源语言上的性能不变。我们的数据和代码可在以下网址公开获取:[此URL](请根据实际情况替换为具体的链接)。
https://arxiv.org/abs/2509.17493
In natural language processing tasks, pure reinforcement learning (RL) fine-tuning methods often suffer from inefficient exploration and slow convergence; while supervised fine-tuning (SFT) methods, although efficient in training, have limited performance ceiling and less solid theoretical foundation compared to RL. To address efficiency-capability trade-off, we propose the Guess-Think-Answer (GTA) framework that combines the efficiency of SFT with the capability gains of RL in a unified training paradigm. GTA works by having the model first produce a provisional guess (optimized via cross-entropy loss), then reflect on this guess before generating the final answer, with RL rewards shaping both the final output and the format of the entire GTA structure. This hybrid approach achieves both faster convergence than pure RL and higher performance ceiling than pure SFT. To mitigate gradient conflicts between the two training signals, we employ loss masking and gradient constraints. Empirical results on four text classification benchmarks demonstrate that GTA substantially accelerates convergence while outperforming both standalone SFT and RL baselines.
在自然语言处理任务中,纯粹的强化学习(RL)微调方法常常因为探索效率低下和收敛速度慢而面临挑战;而监督微调(SFT)方法虽然训练效率高,但其性能上限较低,并且与RL相比理论基础较弱。为了解决效率与能力之间的权衡问题,我们提出了Guess-Think-Answer (GTA) 框架,该框架结合了SFT的高效性和RL的能力提升,在统一的训练范式中实现了两者的优点。GTA的工作机制是让模型首先生成一个初步猜测(通过交叉熵损失进行优化),然后基于这个初步猜测进行反思,最终输出答案,其中强化学习奖励不仅影响最终输出,还塑造整个GTA结构的形式。这种混合方法比纯粹的RL收敛更快,并且在性能上限方面优于纯粹的SFT。为了缓解两种训练信号之间的梯度冲突,我们采用了损失掩码和梯度约束技术。实验证明,在四个文本分类基准数据集上,GTA不仅大大加速了收敛速度,还超过了独立使用的SFT和RL基线方法的表现。
https://arxiv.org/abs/2509.12108
The most widely used large language models in the social sciences (such as BERT, and its derivatives, e.g. RoBERTa) have a limitation on the input text length that they can process to produce predictions. This is a particularly pressing issue for some classification tasks, where the aim is to handle long input texts. One such area deals with laws and draft laws (bills), which can have a length of multiple hundred pages and, therefore, are not particularly amenable for processing with models that can only handle e.g. 512 tokens. In this paper, we show results from experiments covering 5 languages with XLM-RoBERTa, Longformer, GPT-3.5, GPT-4 models for the multiclass classification task of the Comparative Agendas Project, which has a codebook of 21 policy topic labels from education to health care. Results show no particular advantage for the Longformer model, pre-trained specifically for the purposes of handling long inputs. The comparison between the GPT variants and the best-performing open model yielded an edge for the latter. An analysis of class-level factors points to the importance of support and substance overlaps between specific categories when it comes to performance on long text inputs.
在社会科学领域中广泛使用的大规模语言模型(如BERT及其衍生版本,例如RoBERTa)在处理生成预测的输入文本长度方面存在限制。对于需要处理长篇输入文本的一些分类任务而言,这个问题尤为突出。其中一个研究领域涉及法律和法案草案,这些文件可能长达数百页,因此不适合只能处理512个令牌等有限长度文本的模型进行处理。在这篇文章中,我们展示了使用XLM-RoBERTa、Longformer、GPT-3.5及GPT-4模型在比较议程项目(Comparative Agendas Project)多类分类任务中的实验结果,该任务涉及从教育到医疗保健等21个政策主题标签的代码簿。结果显示,专门针对处理长输入文本而预训练的Longformer模型并无显著优势。将GPT变体与表现最佳的开源模型进行比较后发现,后者具有明显的优势。对类别层面因素的分析表明,在处理长篇文本时,特定类别之间的支持和实质重叠对于性能至关重要。
https://arxiv.org/abs/2509.10199
The output of large language models (LLM) is unstable, due to both non-determinism of the decoding process as well as to prompt brittleness. While the intrinsic non-determinism of LLM generation may mimic existing uncertainty in human annotations through distributional shifts in outputs, it is largely assumed, yet unexplored, that the prompt brittleness effect is unique to LLMs. This raises the question: do human annotators show similar sensitivity to instruction changes? If so, should prompt brittleness in LLMs be considered problematic? One may alternatively hypothesize that prompt brittleness correctly reflects human annotation variances. To fill this research gap, we systematically compare the effects of prompt modifications on LLMs and identical instruction modifications for human annotators, focusing on the question of whether humans are similarly sensitive to prompt perturbations. To study this, we prompt both humans and LLMs for a set of text classification tasks conditioned on prompt variations. Our findings indicate that both humans and LLMs exhibit increased brittleness in response to specific types of prompt modifications, particularly those involving the substitution of alternative label sets or label formats. However, the distribution of human judgments is less affected by typographical errors and reversed label order than that of LLMs.
大型语言模型(LLM)的输出不稳定,这既由于解码过程中的非确定性,也由于提示的脆弱性。虽然LLM生成内在的非确定性可以通过输出分布的变化模拟现有标注中的不确定性,但人们普遍认为但尚未充分探讨的是,提示脆弱性的效果是特定于大型语言模型的独特现象。这就提出了一个问题:人类注释者对指令变化是否表现出类似的敏感度?如果是的话,那么大型语言模型中的提示脆性是否应被视为问题?另一种假设可能是,提示脆性准确反映了人类标注的变化。为填补这一研究空白,我们系统地比较了对LLM和人类注释者的相同指令修改的提示更改效果,并重点关注人类是否同样敏感于提示扰动的问题。为此,我们在一系列文本分类任务中分别向人类和大型语言模型提出了根据提示变化条件的任务。我们的发现表明,无论是人类还是大型语言模型,在面对特定类型的提示更改时都会表现出更大的脆弱性,尤其是那些涉及替代标签集或标签格式替换的情况。然而,与LLM相比,人类判断的分布受到拼写错误和标签顺序反转的影响较小。
https://arxiv.org/abs/2509.07869
Legal documents pose unique challenges for text classification due to their domain-specific language and often limited labeled data. This paper proposes a hybrid approach for classifying legal texts by combining unsupervised topic and graph embeddings with a supervised model. We employ Top2Vec to learn semantic document embeddings and automatically discover latent topics, and Node2Vec to capture structural relationships via a bipartite graph of legal documents. The embeddings are combined and clustered using KMeans, yielding coherent groupings of documents. Our computations on a legal document dataset demonstrate that the combined Top2Vec+Node2Vec approach improves clustering quality over text-only or graph-only embeddings. We conduct a sensitivity analysis of hyperparameters, such as the number of clusters and the dimensionality of the embeddings, and demonstrate that our method achieves competitive performance against baseline Latent Dirichlet Allocation (LDA) and Non-Negative Matrix Factorization (NMF) models. Key findings indicate that while the pipeline presents an innovative approach to unsupervised legal document analysis by combining semantic topic modeling with graph embedding techniques, its efficacy is contingent upon the quality of initial topic generation and the representational power of the chosen embedding models for specialized legal language. Strategic recommendations include the exploration of domain-specific embeddings, more comprehensive hyperparameter tuning for Node2Vec, dynamic determination of cluster numbers, and robust human-in-the-loop validation processes to enhance legal relevance and trustworthiness. The pipeline demonstrates potential for exploratory legal data analysis and as a precursor to supervised learning tasks but requires further refinement and domain-specific adaptation for practical legal applications.
法律文件的分类对于文本分类来说具有独特的挑战,因为这些文档含有特定领域的语言,并且通常标注数据有限。本文提出了一种混合方法来对法律文本进行分类,该方法结合了无监督的主题和图嵌入与有监督模型。我们利用Top2Vec学习语义文档嵌入并自动发现潜在主题,同时使用Node2Vec通过法律文件的二部图捕捉结构关系。将这些嵌入合并并通过KMeans聚类分析生成一致的文档组群。我们在一个法律文档数据集上的计算表明,结合了Top2Vec和Node2Vec的方法在文本或仅基于图的嵌入方法上提高了聚类的质量。 我们对超参数(如聚类数量和嵌入维度)进行了敏感性分析,并证明我们的方法在基线潜在狄利克雷分配(LDA)和非负矩阵分解(NMF)模型中具有竞争力。关键发现表明,虽然此流程通过结合语义主题建模与图嵌入技术提供了创新的无监督法律文档分析方法,但其有效性取决于初始主题生成的质量以及所选嵌入模型对特定法律语言的表达能力。 战略建议包括探索领域特异性嵌入、更全面地调整Node2Vec中的超参数、动态确定聚类数量以及实施稳健的人机交互验证过程来增强法律相关性和可信度。该流程展示了在法律数据探索性分析和作为监督学习任务之前的前奏方面的潜力,但需要进一步的细化及特定领域的适应以满足实际应用需求。 以下是对原文的主要翻译: 法律文件分类因其领域专有语言及标注数据有限而具有独特挑战。本文提出了一种结合无监督主题与图嵌入以及监督模型对法律文本进行分类的方法。我们使用Top2Vec学习语义文档嵌入并自动发现潜在话题,通过在法律文档二部图中运用Node2Vec捕捉结构关系。我们将这些嵌入合并并通过KMeans聚类生成一致的文档组群。 我们在一个法律文件数据集上的计算表明,结合Top2Vec与Node2Vec的方法相较于仅使用文本或仅基于图的嵌入方法能提升聚类质量。我们对超参数(如聚类数量、嵌入维度等)进行了敏感性分析,并证明该方法在基线潜在狄利克雷分配(LDA)和非负矩阵分解(NMF)模型中具有竞争力。 关键发现表明,虽然此流程通过结合语义主题建模与图嵌入技术提供了创新的无监督法律文档分析方法,但其有效性依赖于初始话题生成的质量及所选嵌入模型对特定语言表达能力。战略建议包括探索领域特异性嵌入、更细致地调整Node2Vec中的超参数、动态确定聚类数量以及实施稳健的人机交互验证过程来增强相关性和可信度。 该流程展示了法律数据的探索性分析和作为监督学习任务前导方面的潜力,但需进一步细化及特定领域的适应以满足实际应用需求。
https://arxiv.org/abs/2509.00990
The demand for text classification is growing significantly in web searching, data mining, web ranking, recommendation systems, and so many other fields of information and technology. This paper illustrates the text classification process on different datasets using some standard supervised machine learning techniques. Text documents can be classified through various kinds of classifiers. Labeled text documents are used to classify the text in supervised classifications. This paper applies these classifiers on different kinds of labeled documents and measures the accuracy of the classifiers. An Artificial Neural Network (ANN) model using Back Propagation Network (BPN) is used with several other models to create an independent platform for labeled and supervised text classification process. An existing benchmark approach is used to analyze the performance of classification using labeled documents. Experimental analysis on real data reveals which model works well in terms of classification accuracy.
文本分类的需求在网页搜索、数据挖掘、网页排名、推荐系统以及其他许多信息和技术领域中正显著增长。本文通过使用一些标准的监督机器学习技术,在不同的数据集上展示了文本分类的过程。可以通过多种类型的分类器对文本文件进行分类。在监督分类中,带有标签的文本文档被用来分类文本。本文将这些分类器应用于不同种类的标记文档,并衡量这些分类器的准确性。 使用人工神经网络(ANN)模型和反向传播网络(BPN),与其他几种模型一起构建了一个独立平台,用于对带标签和监督下的文本分类过程进行处理。利用现有的基准方法来分析使用标记文档时分类性能的表现。通过真实数据上的实验分析揭示了哪种模型在分类准确性方面表现良好。
https://arxiv.org/abs/2509.00983
Legal text classification is a fundamental NLP task in the legal domain. Benchmark datasets in this area often exhibit a long-tail label distribution, where many labels are underrepresented, leading to poor model performance on rare classes. This paper proposes Selective Retrieval-Augmentation (SRA) as a solution to this problem. SRA focuses on augmenting samples belonging to low-frequency labels in the training set, preventing the introduction of noise for well-represented classes, and requires no changes to the model architecture. Retrieval is performed only from the training data to ensure there is no potential information leakage, removing the need for external corpora simultaneously. The proposed SRA method is tested on two legal text classification benchmark datasets with long-tail distributions: LEDGAR (single-label) and UNFAIR-ToS (multi-label). The results indicate that SRA attains higher micro-F1 and macro-F1 scores compared to all current LexGLUE baselines across both datasets, illustrating consistent improvements in long-tail legal text classification. The code repository is available at: this https URL
法律文本分类是法律领域中的基本自然语言处理任务。在这个领域的基准数据集中,标签分布往往呈现出长尾特性,即许多标签的代表性不足,导致模型在罕见类别上的表现较差。本文提出了一种名为选择性检索增强(Selective Retrieval-Augmentation,简称SRA)的方法来解决这个问题。SRA专注于增强训练集中的低频标签样本,同时避免为那些已有足够代表性的类引入噪声,并且无需对模型架构进行任何更改。检索仅从训练数据中进行,以确保没有潜在的信息泄露风险,并且不需要外部语料库。 本文提出的SRA方法在两个具有长尾分布的法律文本分类基准数据集上进行了测试:LEDGAR(单标签)和UNFAIR-ToS(多标签)。实验结果显示,在这两个数据集中,与所有当前LexGLUE基线相比,SRA获得了更高的微平均F1分数和宏平均F1分数,表明其在长尾法律文本分类方面取得了持续的改进。该方法的代码库可在此处获得:[此链接](https://this https URL)。 请注意,原文中的“this https URL”应更正为实际的有效链接地址以提供访问相关代码资源的可能性。
https://arxiv.org/abs/2508.19997
Despite its significance, Arabic, a linguistically rich and morphologically complex language, faces the challenge of being under-resourced. The scarcity of large annotated datasets hampers the development of accurate tools for subjectivity analysis in Arabic. Recent advances in deep learning and Transformers have proven highly effective for text classification in English and French. This paper proposes a new approach for subjectivity assessment in Arabic textual data. To address the dearth of specialized annotated datasets, we developed a comprehensive dataset, AraDhati+, by leveraging existing Arabic datasets and collections (ASTD, LABR, HARD, and SANAD). Subsequently, we fine-tuned state-of-the-art Arabic language models (XLM-RoBERTa, AraBERT, and ArabianGPT) on AraDhati+ for effective subjectivity classification. Furthermore, we experimented with an ensemble decision approach to harness the strengths of individual models. Our approach achieves a remarkable accuracy of 97.79\,\% for Arabic subjectivity classification. Results demonstrate the effectiveness of the proposed approach in addressing the challenges posed by limited resources in Arabic language processing.
尽管阿拉伯语作为一种语言丰富且形态复杂的语言具有重要性,但它面临着资源不足的挑战。大型标注数据集的匮乏阻碍了开发准确进行主观分析工具的发展。最近在深度学习和Transformer方面的进展已经证明对英语和法语文本分类非常有效。本文提出了一种新的方法来评估阿拉伯语文本数据中的主观性。为了应对专门标注数据集缺乏的问题,我们通过利用现有的阿拉伯语数据集和集合(ASTD、LABR、HARD 和 SANAD)开发了一个全面的数据集AraDhati+。随后,我们将最先进的阿拉伯语言模型(XLM-RoBERTa、AraBERT 和 ArabianGPT)在AraDhati+上进行了微调,以实现有效的主观性分类。此外,我们还尝试了集成决策方法来利用各个模型的优势。我们的方法实现了97.79% 的阿拉伯语主观性分类准确率。结果表明,所提出的方法在解决阿拉伯语文本处理中有限资源带来的挑战方面非常有效。
https://arxiv.org/abs/2508.19966
LSTM models used in current Machine Learning literature and applications, has a promising solution for permitting long term information using gating mechanisms that forget and reduce effect of current input information. However, even with this pipeline, they do not optimally focus on specific old index or long-term information. This paper elaborates upon input reordering approaches to prioritize certain input indices. Moreover, no LSTM based approach is found in the literature that examines weight normalization while choosing the right weight and exponent of Lp norms through main supervised loss function. In this paper, we find out which norm best finds relationship between weights to either smooth or sparsify them. Lastly, gates, as weighted representations of inputs and states, which control reduction-extent of current input versus previous inputs (~ state), are not nonlinearized enough (through a small FFNN). As analogous to attention mechanisms, gates easily filter current information to bold (emphasize on) past inputs. Nonlinearized gates can more easily tune up to peculiar nonlinearities of specific input in the past. This type of nonlinearization is not proposed in the literature, to the best of author's knowledge. The proposed approaches are implemented and compared with a simple LSTM to understand their performance in text classification tasks. The results show they improve accuracy of LSTM.
当前的机器学习文献和应用中使用的LSTM模型通过使用门控机制来遗忘旧信息并减少当前输入的影响,从而允许可长期存储信息。然而,即使有了这些机制,它们也无法在处理特定旧索引或长时间信息时达到最佳聚焦效果。本文探讨了重新排序输入的方法以优先考虑某些输入索引。 此外,在现有的文献中尚未发现任何基于LSTM的方法会考察在选择合适的权重和$L_p$范数指数时进行权重归一化的做法,特别是在主要的监督损失函数范围内。本文通过研究哪种范数能最好地找到权重之间的关系(无论是平滑还是稀疏化),来寻找最优解决方案。 另外,门控机制作为输入和状态的加权表示,控制了当前输入与先前输入(~ 状态)之间的影响程度。但这些门控机制并未进行足够的非线性处理(通过小型前馈神经网络)。类似注意力机制,在过滤当前信息以突出显示过去的信息时,它们很容易做到这一点。经过非线性的门控可以更有效地调整个别过去的输入的特殊非线性。 据作者所知,这种类型的非线性化在文献中尚未被提出过。本文提出的几种方法与简单的LSTM进行对比测试,用以评估其在文本分类任务中的性能表现。实验结果表明,这些方法确实提高了LSTM模型的准确率。
https://arxiv.org/abs/2509.00087
Autoprompting is the process of automatically selecting optimized prompts for language models, which is gaining popularity due to the rapid development of prompt engineering driven by extensive research in the field of large language models (LLMs). This paper presents DistillPrompt -- a novel autoprompting method based on large language models that employs a multi-stage integration of task-specific information into prompts using training data. DistillPrompt utilizes distillation, compression, and aggregation operations to explore the prompt space more thoroughly. The method was tested on different datasets for text classification and generation tasks using the t-lite-instruct-0.1 language model. The results demonstrate a significant average improvement (e.g., 20.12% across the entire dataset compared to Grips) in key metrics over existing methods in the field, establishing DistillPrompt as one of the most effective non-gradient approaches in autoprompting.
自动生成提示词(Autoprompting)是指自动选择优化后的提示词供语言模型使用,这一方法由于在大规模语言模型(LLMs)领域中广泛研究而推动的提示工程快速发展变得越来越受欢迎。本文介绍了一种基于大型语言模型的新颖自动生成提示词方法——DistillPrompt。该方法通过多阶段整合任务特定信息到提示词中,并利用训练数据进行操作,包括蒸馏、压缩和聚合等步骤来更彻底地探索提示空间。 在文本分类和生成任务的不同数据集上使用t-lite-instruct-0.1语言模型对DistillPrompt进行了测试。实验结果显示,在关键性能指标方面,与现有方法相比(例如,相较于Grips在整个数据集上的平均改进达20.12%),DistillPrompt表现出显著的平均改善效果,并确立了其作为非梯度自动生成提示词方法中最有效的技术之一的地位。
https://arxiv.org/abs/2508.18992
Transformer-based models like BERT excel at short text classification but struggle with long document classification (LDC) due to input length limitations and computational inefficiencies. In this work, we propose an efficient, zero-shot approach to LDC that leverages sentence ranking to reduce input context without altering the model architecture. Our method enables the adaptation of models trained on short texts, such as headlines, to long-form documents by selecting the most informative sentences using a TF-IDF-based ranking strategy. Using the MahaNews dataset of long Marathi news articles, we evaluate three context reduction strategies that prioritize essential content while preserving classification accuracy. Our results show that retaining only the top 50\% ranked sentences maintains performance comparable to full-document inference while reducing inference time by up to 35\%. This demonstrates that sentence ranking is a simple yet effective technique for scalable and efficient zero-shot LDC.
基于Transformer的模型(如BERT)在处理短文本分类时表现出色,但在长文档分类(LDC)方面却面临输入长度限制和计算效率低下的挑战。为此,我们提出了一种高效且无需微调的方法来应对长文档分类问题,该方法利用句子排名减少输入上下文而不改变模型架构。我们的方法允许将训练于短文本(如新闻标题)上的模型应用于长篇文档,并通过使用基于TF-IDF的排名策略选择最具信息量的句子实现这一目标。 我们使用MahaNews数据集中的长马拉地语新闻文章,评估了三种减少上下文的方法,这些方法优先考虑关键内容的同时保持分类准确性。实验结果表明,保留前50%排名最高的句子可以将推理时间最多缩短35%,而同时维持与完整文档推理时相当的性能表现。 这表明基于句子排名的技术是一种简单且有效的方法,可用于零样本长文档分类(LDC),以实现可扩展性和效率。
https://arxiv.org/abs/2508.17490
We address the problem of data scarcity in harmful text classification for guardrailing applications and introduce GRAID (Geometric and Reflective AI-Driven Data Augmentation), a novel pipeline that leverages Large Language Models (LLMs) for dataset augmentation. GRAID consists of two stages: (i) generation of geometrically controlled examples using a constrained LLM, and (ii) augmentation through a multi-agentic reflective process that promotes stylistic diversity and uncovers edge cases. This combination enables both reliable coverage of the input space and nuanced exploration of harmful content. Using two benchmark data sets, we demonstrate that augmenting a harmful text classification dataset with GRAID leads to significant improvements in downstream guardrail model performance.
我们解决了护拦应用中危害性文本分类数据不足的问题,并引入了GRAID(几何和反思型AI驱动的数据增强),这是一种利用大型语言模型(LLMs)进行数据集扩增的新型流程。GRAID包括两个阶段:(i) 使用受约束的LLM生成几何控制示例,以及(ii) 通过一个多代理反思过程进行扩充,该过程促进了风格多样性并揭示了边缘案例。这种组合使得输入空间的可靠覆盖和有害内容的细腻探索成为可能。使用两个基准数据集,我们证明将GRAID用于扩增危害性文本分类数据集可以显著提高下游护拦模型的表现。
https://arxiv.org/abs/2508.17057