Abstract
Language models increasingly "show their work" by writing step-by-step reasoning before answering. But are these reasoning steps genuinely used, or decorative narratives generated after the model has already decided? Consider: a medical AI writes "The patient's eosinophilia and livedo reticularis following catheterization suggest cholesterol embolization syndrome. Answer: B." If we remove the eosinophilia observation, does the diagnosis change? For most frontier models, the answer is no - the step was decorative. We introduce step-level evaluation: remove one reasoning sentence at a time and check whether the answer changes. This simple test requires only API access -- no model weights -- and costs approximately $1-2 per model per task. Testing 10 frontier models (GPT-5.4, Claude Opus, DeepSeek-V3.2, MiniMax-M2.5, Kimi-K2.5, and others) across sentiment, mathematics, topic classification, and medical QA (N=376-500 each), the majority produce decorative reasoning: removing any step changes the answer less than 17% of the time, while any single step alone recovers the answer. This holds even on math, where smaller models (0.8-8B) show genuine step dependence (55% necessity). Two models break the pattern: MiniMax-M2.5 on sentiment (37% necessity) and Kimi-K2.5 on topic classification (39%) - but both shortcut other tasks. Faithfulness is model-specific and task-specific. We also discover "output rigidity": on the same medical questions, Claude Opus writes 11 diagnostic steps while GPT-OSS-120B outputs a single token. Mechanistic analysis (attention patterns) confirms that CoT attention drops more in late layers for decorative tasks (33%) than faithful ones (20%). Implications: step-by-step explanations from frontier models are largely decorative, per-model per-domain evaluation is essential, and training objectives - not scale - determine whether reasoning is genuine.
Abstract (translated)
语言模型越来越多地通过逐步推理来“展示其思考过程”,但这些推理步骤是真正被使用,还是模型早已决定答案后生成的装饰性叙述?考虑以下案例:一个医疗AI写道“患者导管置入后出现嗜酸性粒细胞增多和网状青斑,提示胆固醇栓塞综合征。答案:B。”若移除嗜酸性粒细胞增多的观察,诊断会改变吗?对于大多数前沿模型,答案是否定的——该步骤纯属装饰。我们提出步骤级评估方法:每次移除一个推理句子,检查答案是否变化。这一简单测试仅需API访问权限(无需模型权重),每个模型每项任务成本约1-2美元。我们在情感分析、数学、主题分类和医疗问答(每类任务N=376-500)中测试了10个前沿模型(GPT-5.4、Claude Opus、DeepSeek-V3.2、MiniMax-M2.5、Kimi-K2.5等),发现多数模型产生装饰性推理:移除任意步骤后答案改变的概率低于17%,而任意单步本身即可恢复答案。即使在数学领域,小型模型(0.8-8B)仍表现出真实的步骤依赖性(55%的必要性)。两个模型例外:MiniMax-M2.5在情感分析中(37%必要性)和Kimi-K2.5在主题分类中(39%)——但两者在其他任务中均存在捷径现象。忠实度具有模型特定性和任务特定性。我们还发现“输出刚性”现象:针对相同医疗问题,Claude Opus生成11个诊断步骤,而GPT-OSS-120B仅输出一个标记。机制分析(注意力模式)证实,装饰性任务的思维链注意力在深层下降更显著(33%),而忠实任务仅下降20%。启示:前沿模型的逐步解释大多具有装饰性,必须进行模型与领域特定的评估,且训练目标——而非模型规模——决定推理是否真实。
URL
https://arxiv.org/abs/2603.22816