Abstract
FreeTxt-Vi is a free and open source web based toolkit for creating and analysing bilingual Vietnamese English text collections. Positioned at the intersection of corpus linguistics and natural language processing NLP it enables users to build explore and interpret free text data without requiring programming expertise. The system combines corpus analysis features such as concordancing keyword analysis word relation exploration and interactive visualisation with transformer based NLP components for sentiment analysis and summarisation. A key contribution of this work is the design of a unified bilingual NLP pipeline that integrates a hybrid VnCoreNLP and Byte Pair Encoding BPE segmentation strategy a fine tuned TabularisAI sentiment classifier and a fine tuned Qwen2.5 model for abstractive summarisation. Unlike existing text analysis platforms FreeTxt Vi is evaluated as a set of language processing components. We conduct a three part evaluation covering segmentation sentiment analysis and summarisation and show that our approach achieves competitive or superior performance compared to widely used baselines in both Vietnamese and English. By reducing technical barriers to multilingual text analysis FreeTxt Vi supports reproducible research and promotes the development of language resources for Vietnamese a widely spoken but underrepresented language in NLP. The toolkit is applicable to domains including education digital humanities cultural heritage and the social sciences where qualitative text data are common but often difficult to process at scale.
Abstract (translated)
FreeTxt-Vi 是一个免费且开源的基于网页的工具包,用于创建和分析越南语-英语双语文本集合。它位于语料库语言学与自然语言处理(NLP)的交叉点上,使用户能够在无需编程技能的情况下构建、探索并解释自由文本数据。该系统结合了如词汇共现分析、关键词分析、词关系探索以及交互式可视化等语料库分析功能,并集成了基于转换器的NLP组件以进行情感分析和摘要生成。 这项工作的关键贡献在于设计了一个统一的双语文本处理流水线,整合了一种混合策略(VnCoreNLP与字节对编码BPE分词),一个经过微调的TabularisAI情感分类器以及用于抽象式摘要生成的Qwen2.5模型。不同于现有的文本分析平台,FreeTxt-Vi作为一个语言处理组件集合进行评估。我们进行了三部分评估,涵盖分割、情感分析和总结,并展示了我们的方法在越南语与英语中均达到了相比广泛使用的基础线模型更为竞争或优越的表现。 通过降低多语言文本分析的技术门槛,FreeTxt-Vi支持可重复的研究并促进越南语资源的发展——这是一种被广泛使用的但又在NLP领域代表性不足的语言。该工具包适用于教育、数字人文、文化遗产和社科等领域,在这些领域中质性文本数据很常见但往往难以大规模处理。
URL
https://arxiv.org/abs/2603.05690