Abstract
Environmental, Social, and Governance (ESG) principles are reshaping the foundations of global financial gover- nance, transforming capital allocation architectures, regu- latory frameworks, and systemic risk coordination mecha- nisms. However, as the core medium for assessing corpo- rate ESG performance, the ESG reports present significant challenges for large-scale understanding, due to chaotic read- ing order from slide-like irregular layouts and implicit hier- archies arising from lengthy, weakly structured content. To address these challenges, we propose Pharos-ESG, a uni- fied framework that transforms ESG reports into structured representations through multimodal parsing, contextual nar- ration, and hierarchical labeling. It integrates a reading-order modeling module based on layout flow, hierarchy-aware seg- mentation guided by table-of-contents anchors, and a multi- modal aggregation pipeline that contextually transforms vi- sual elements into coherent natural language. The framework further enriches its outputs with ESG, GRI, and sentiment labels, yielding annotations aligned with the analytical de- mands of financial research. Extensive experiments on anno- tated benchmarks demonstrate that Pharos-ESG consistently outperforms both dedicated document parsing systems and general-purpose multimodal models. In addition, we release Aurora-ESG, the first large-scale public dataset of ESG re- ports, spanning Mainland China, Hong Kong, and U.S. mar- kets, featuring unified structured representations of multi- modal content, enriched with fine-grained layout and seman- tic annotations to better support ESG integration in financial governance and decision-making.
Abstract (translated)
环境、社会与治理(ESG)原则正在重塑全球金融治理的基础,改变资本配置架构、监管框架和系统性风险协调机制。然而,作为评估企业ESG表现的核心媒介,ESG报告因其类似幻灯片的非规整布局导致的阅读顺序混乱,以及内容冗长且结构松散而产生的隐性层级关系,给大规模理解带来了显著挑战。为应对这些挑战,我们提出了Pharos-ESG统一框架,该框架通过多模态解析、情境化叙述和层级标注,将ESG报告转化为结构化表示。它整合了基于布局流的阅读顺序建模模块、由目录锚点引导的层级感知分割,以及一个将视觉元素情境化转换为连贯自然语言的多模态聚合流程。该框架进一步融合ESG、GRI及情感标签,生成与金融研究分析需求对齐的标注结果。在标注基准上的大量实验表明,Pharos-ESG在表现上持续优于专用文档解析系统和通用多模态模型。此外,我们发布了首个ESG报告大规模公开数据集Aurora-ESG,覆盖中国大陆、香港及美国市场,提供多模态内容的统一结构化表示,并辅以细粒度的布局与语义标注,以更好地支持ESG在金融治理与决策中的整合应用。
URL
https://arxiv.org/abs/2511.16417