Paper Reading AI Learner

Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectories

2025-09-19 17:35:51
Yifan Lin, Sophie Ziyu Liu, Ran Qi, George Z. Xue, Xinping Song, Chao Qin, Hugh H. -T. Liu

Abstract

We present Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectories (ACDC), an autonomous drone cinematography system driven by natural language communication between human directors and drones. The main limitation of previous drone cinematography workflows is that they require manual selection of waypoints and view angles based on predefined human intent, which is labor-intensive and yields inconsistent performance. In this paper, we propose employing large language models (LLMs) and vision foundation models (VFMs) to convert free-form natural language prompts directly into executable indoor UAV video tours. Specifically, our method comprises a vision-language retrieval pipeline for initial waypoint selection, a preference-based Bayesian optimization framework that refines poses using aesthetic feedback, and a motion planner that generates safe quadrotor trajectories. We validate ACDC through both simulation and hardware-in-the-loop experiments, demonstrating that it robustly produces professional-quality footage across diverse indoor scenes without requiring expertise in robotics or cinematography. These results highlight the potential of embodied AI agents to close the loop from open-vocabulary dialogue to real-world autonomous aerial cinematography.

Abstract (translated)

我们介绍了代理式空中摄影系统“从对话提示到电影轨迹”(ACDC),这是一个由人与无人机之间自然语言交流驱动的自主无人机摄影系统。以往无人机摄影工作流程的主要限制在于,它们需要手动选择航点和视角角度,这些选择基于预先定义的人类意图进行,这既费时又会导致性能不一致。在本文中,我们提出使用大型语言模型(LLMs)和视觉基础模型(VFMs),将自由形式的自然语言提示直接转换为可执行的室内无人机视频游览。具体而言,我们的方法包括一个用于初始航点选择的视觉-语言检索管道、一个基于偏好的贝叶斯优化框架,该框架使用美学反馈来精炼姿态,并且还有一个运动规划器生成安全的四旋翼轨迹。 我们通过仿真和硬件在环实验验证了ACDC的有效性,证明它能够稳健地在各种室内场景中生成专业级别的影像素材,而无需具备机器人技术或电影摄影的专业知识。这些结果突显了具身AI代理从开放式词汇对话到现实世界自主空中摄影闭环的潜力。

URL

https://arxiv.org/abs/2509.16176

PDF

https://arxiv.org/pdf/2509.16176.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot