Abstract
Vision transformer (ViT) models, when coupled with interpretation models, are regarded as secure and challenging to deceive, making them well-suited for security-critical domains such as medical applications, autonomous vehicles, drones, and robotics. However, successful attacks on these systems can lead to severe consequences. Recent research on threats targeting ViT models primarily focuses on generating the smallest adversarial perturbations that can deceive the models with high confidence, without considering their impact on model interpretations. Nevertheless, the use of interpretation models can effectively assist in detecting adversarial examples. This study investigates the vulnerability of transformer models to adversarial attacks, even when combined with interpretation models. We propose an attack called "AdViT" that generates adversarial examples capable of misleading both a given transformer model and its coupled interpretation model. Through extensive experiments on various transformer models and two transformer-based interpreters, we demonstrate that AdViT achieves a 100% attack success rate in both white-box and black-box scenarios. In white-box scenarios, it reaches up to 98% misclassification confidence, while in black-box scenarios, it reaches up to 76% misclassification confidence. Remarkably, AdViT consistently generates accurate interpretations in both scenarios, making the adversarial examples more difficult to detect.
Abstract (translated)
当视觉变换器(Vision Transformer,简称 ViT)模型与解释模型结合使用时,这些组合系统被认为具有很高的安全性,并且难以被欺骗。因此,它们非常适合需要高度安全性的领域,例如医疗应用、自动驾驶汽车、无人机和机器人技术。然而,成功攻击这些系统的威胁可能导致严重后果。 最近针对ViT模型的威胁研究主要集中在生成最小的对抗性扰动上,这种扰动能够以很高的置信度欺骗模型,但没有考虑对模型解释的影响。不过,使用解释模型可以有效帮助检测对抗样本。这项研究表明了即使与解释模型结合后,Transformer模型仍然容易受到对抗攻击的影响。 我们提出了一种名为“AdViT”的攻击方法,该方法生成能够同时误导给定变换器模型及其耦合的解释模型的对抗性样例。通过在多种变压器模型和两种基于变压器的解释器上进行广泛的实验,我们展示了AdViT在白盒和黑盒场景下均能实现100%的攻击成功率。在白盒场景中,它能够达到高达98%的误分类置信度,而在黑盒场景中,则能达到76%的误分类置信度。 特别值得注意的是,在两种情况下,AdViT都能持续生成准确的解释,使对抗样本更难以被检测出来。
URL
https://arxiv.org/abs/2507.14248