Abstract
Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning grounded in perception and action to generalize beyond training distributions and adapt to novel real-world situations. These capabilities, however, also create new security risks. In this paper, we introduce CHAI (Command Hijacking against embodied AI), a new class of prompt-based attacks that exploit the multimodal language interpretation abilities of Large Visual-Language Models (LVLMs). CHAI embeds deceptive natural language instructions, such as misleading signs, in visual input, systematically searches the token space, builds a dictionary of prompts, and guides an attacker model to generate Visual Attack Prompts. We evaluate CHAI on four LVLM agents; drone emergency landing, autonomous driving, and aerial object tracking, and on a real robotic vehicle. Our experiments show that CHAI consistently outperforms state-of-the-art attacks. By exploiting the semantic and multimodal reasoning strengths of next-generation embodied AI systems, CHAI underscores the urgent need for defenses that extend beyond traditional adversarial robustness.
Abstract (translated)
嵌入式人工智能(AI)承诺通过使用基于感知和行动的常识推理来处理机器人车辆系统中数据稀缺的情况,从而能够超越训练分布并在新的现实世界情境下进行适应。然而,这些能力也带来了新的安全风险。本文介绍了 CHAI (针对嵌入式 AI 的命令劫持攻击),这是一种新型的基于提示的攻击方法,利用大型视觉-语言模型(LVLM)的多模态语言解释能力。CHAI 将欺骗性的自然语言指令,如误导性标志,嵌入到视觉输入中,并系统地搜索词元空间、构建提示字典,引导攻击者模型生成视觉攻击提示。 我们在四种 LVLM 代理上评估了 CHAI:无人机紧急着陆、自动驾驶和空中物体跟踪,以及在实际的机器人车辆上。我们的实验表明,CHAI 在面对最先进的攻击时始终表现出色。通过利用下一代嵌入式 AI 系统的语义和多模态推理能力,CHAI 强调了需要超越传统对抗鲁棒性的防御措施以应对安全挑战。
URL
https://arxiv.org/abs/2510.00181