Abstract
Adapting trajectories to dynamic situations and user preferences is crucial for robot operation in unstructured environments with non-expert users. Natural language enables users to express these adjustments in an interactive manner. We introduce OVITA, an interpretable, open-vocabulary, language-driven framework designed for adapting robot trajectories in dynamic and novel situations based on human instructions. OVITA leverages multiple pre-trained Large Language Models (LLMs) to integrate user commands into trajectories generated by motion planners or those learned through demonstrations. OVITA employs code as an adaptation policy generated by an LLM, enabling users to adjust individual waypoints, thus providing flexible control. Another LLM, which acts as a code explainer, removes the need for expert users, enabling intuitive interactions. The efficacy and significance of the proposed OVITA framework is demonstrated through extensive simulations and real-world environments with diverse tasks involving spatiotemporal variations on heterogeneous robotic platforms such as a KUKA IIWA robot manipulator, Clearpath Jackal ground robot, and CrazyFlie drone.
Abstract (translated)
适应动态情况和用户偏好的轨迹对于非专业用户在无结构环境中操作机器人至关重要。自然语言使用户能够以互动方式表达这些调整。我们介绍了OVITA,这是一种可解释的、开放式词汇的语言驱动框架,旨在根据人类指令来适应机器人在动态及新颖情景中的轨迹。OVITA利用多个预训练的大规模语言模型(LLMs)将用户的命令集成到由运动规划器生成或通过演示学习得到的轨迹中。 OVITA采用代码作为适应策略,该策略是由一个LLM生成的,使用户能够调整个别航点,从而提供灵活的控制。另一个充当代码解释者的LLM则消除了对专家用户的需求,促进了直观互动。通过在多种任务中的广泛模拟和真实世界环境(包括KUKA IIWA机械臂、Clearpath Jackal地面机器人以及CrazyFlie无人机等异构机器人平台上涉及时空变化的任务)展示了OVITA框架的有效性和重要性。
URL
https://arxiv.org/abs/2508.17260