Paper Reading AI Learner

OVITA: Open-Vocabulary Interpretable Trajectory Adaptations

2025-08-24 09:01:40
Anurag Maurya, Tashmoy Ghosh, Anh Nguyen, Ravi Prakash

Abstract

Adapting trajectories to dynamic situations and user preferences is crucial for robot operation in unstructured environments with non-expert users. Natural language enables users to express these adjustments in an interactive manner. We introduce OVITA, an interpretable, open-vocabulary, language-driven framework designed for adapting robot trajectories in dynamic and novel situations based on human instructions. OVITA leverages multiple pre-trained Large Language Models (LLMs) to integrate user commands into trajectories generated by motion planners or those learned through demonstrations. OVITA employs code as an adaptation policy generated by an LLM, enabling users to adjust individual waypoints, thus providing flexible control. Another LLM, which acts as a code explainer, removes the need for expert users, enabling intuitive interactions. The efficacy and significance of the proposed OVITA framework is demonstrated through extensive simulations and real-world environments with diverse tasks involving spatiotemporal variations on heterogeneous robotic platforms such as a KUKA IIWA robot manipulator, Clearpath Jackal ground robot, and CrazyFlie drone.

Abstract (translated)

适应动态情况和用户偏好的轨迹对于非专业用户在无结构环境中操作机器人至关重要。自然语言使用户能够以互动方式表达这些调整。我们介绍了OVITA,这是一种可解释的、开放式词汇的语言驱动框架,旨在根据人类指令来适应机器人在动态及新颖情景中的轨迹。OVITA利用多个预训练的大规模语言模型(LLMs)将用户的命令集成到由运动规划器生成或通过演示学习得到的轨迹中。 OVITA采用代码作为适应策略,该策略是由一个LLM生成的,使用户能够调整个别航点,从而提供灵活的控制。另一个充当代码解释者的LLM则消除了对专家用户的需求,促进了直观互动。通过在多种任务中的广泛模拟和真实世界环境(包括KUKA IIWA机械臂、Clearpath Jackal地面机器人以及CrazyFlie无人机等异构机器人平台上涉及时空变化的任务)展示了OVITA框架的有效性和重要性。

URL

https://arxiv.org/abs/2508.17260

PDF

https://arxiv.org/pdf/2508.17260.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot