Abstract
In-context learning (ICL) is a crucial capability of current large language models (LLMs), where the selection of examples plays a key role in performance. While most existing approaches focus on selecting the most similar examples to the query, the impact of diversity in example selection remains underexplored. We systematically investigate the role of diversity in in-context example selection through experiments across a range of tasks, from sentiment classification to more challenging math and code problems. Experiments on Llama-3.1, Gemma-2, and Mistral-v0.3 families of models show that diversity-aware selection methods improve performance, particularly on complex tasks like math and code, and enhance robustness to out-of-distribution queries. To support these findings, we introduce a theoretical framework that explains the benefits of incorporating diversity in in-context example selection.
Abstract (translated)
上下文学习（ICL）是当前大型语言模型（LLM）的一项关键能力，其中选择示例在性能中起着重要作用。虽然大多数现有方法侧重于选取与查询最相似的示例，但多样性的选择对性能的影响仍鲜有探索。我们通过一系列实验系统地研究了多样性在上下文示例选择中的作用，涵盖了从情感分类到更具挑战性的数学和编程问题等各种任务。在Llama-3.1、Gemma-2和Mistral-v0.3系列模型上的实验证明，具有多样性的示例选择方法能够提高性能，特别是在复杂的数学和编程等任务上，并且还能增强对分布外查询的鲁棒性。为了支持这些发现，我们引入了一个理论框架来解释在上下文示例选择中融入多样性的好处。
URL
https://arxiv.org/abs/2505.19426
