Paper Reading AI Learner

When AI Shows Its Work, Is It Actually Working? Step-Level Evaluation Reveals Frontier Language Models Frequently Bypass Their Own Reasoning

2026-03-24 05:38:13
Abhinaba Basu, Pavan Chakraborty

Abstract

Language models increasingly "show their work" by writing step-by-step reasoning before answering. But are these reasoning steps genuinely used, or decorative narratives generated after the model has already decided? Consider: a medical AI writes "The patient's eosinophilia and livedo reticularis following catheterization suggest cholesterol embolization syndrome. Answer: B." If we remove the eosinophilia observation, does the diagnosis change? For most frontier models, the answer is no - the step was decorative. We introduce step-level evaluation: remove one reasoning sentence at a time and check whether the answer changes. This simple test requires only API access -- no model weights -- and costs approximately $1-2 per model per task. Testing 10 frontier models (GPT-5.4, Claude Opus, DeepSeek-V3.2, MiniMax-M2.5, Kimi-K2.5, and others) across sentiment, mathematics, topic classification, and medical QA (N=376-500 each), the majority produce decorative reasoning: removing any step changes the answer less than 17% of the time, while any single step alone recovers the answer. This holds even on math, where smaller models (0.8-8B) show genuine step dependence (55% necessity). Two models break the pattern: MiniMax-M2.5 on sentiment (37% necessity) and Kimi-K2.5 on topic classification (39%) - but both shortcut other tasks. Faithfulness is model-specific and task-specific. We also discover "output rigidity": on the same medical questions, Claude Opus writes 11 diagnostic steps while GPT-OSS-120B outputs a single token. Mechanistic analysis (attention patterns) confirms that CoT attention drops more in late layers for decorative tasks (33%) than faithful ones (20%). Implications: step-by-step explanations from frontier models are largely decorative, per-model per-domain evaluation is essential, and training objectives - not scale - determine whether reasoning is genuine.

Abstract (translated)

语言模型越来越多地通过逐步推理来“展示其思考过程”,但这些推理步骤是真正被使用,还是模型早已决定答案后生成的装饰性叙述?考虑以下案例:一个医疗AI写道“患者导管置入后出现嗜酸性粒细胞增多和网状青斑,提示胆固醇栓塞综合征。答案:B。”若移除嗜酸性粒细胞增多的观察,诊断会改变吗?对于大多数前沿模型,答案是否定的——该步骤纯属装饰。我们提出步骤级评估方法:每次移除一个推理句子,检查答案是否变化。这一简单测试仅需API访问权限(无需模型权重),每个模型每项任务成本约1-2美元。我们在情感分析、数学、主题分类和医疗问答(每类任务N=376-500)中测试了10个前沿模型(GPT-5.4、Claude Opus、DeepSeek-V3.2、MiniMax-M2.5、Kimi-K2.5等),发现多数模型产生装饰性推理:移除任意步骤后答案改变的概率低于17%,而任意单步本身即可恢复答案。即使在数学领域,小型模型(0.8-8B)仍表现出真实的步骤依赖性(55%的必要性)。两个模型例外:MiniMax-M2.5在情感分析中(37%必要性)和Kimi-K2.5在主题分类中(39%)——但两者在其他任务中均存在捷径现象。忠实度具有模型特定性和任务特定性。我们还发现“输出刚性”现象:针对相同医疗问题,Claude Opus生成11个诊断步骤,而GPT-OSS-120B仅输出一个标记。机制分析(注意力模式)证实,装饰性任务的思维链注意力在深层下降更显著(33%),而忠实任务仅下降20%。启示:前沿模型的逐步解释大多具有装饰性,必须进行模型与领域特定的评估,且训练目标——而非模型规模——决定推理是否真实。

URL

https://arxiv.org/abs/2603.22816

PDF

https://arxiv.org/pdf/2603.22816.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot