Paper Reading AI Learner

All Eyes, no IMU: Learning Flight Attitude from Vision Alone

2025-07-15 13:31:27
Jesse J. Hagenaars, Stein Stroobants, Sander M. Bohte, Guido C. H. E. De Croon

Abstract

Vision is an essential part of attitude control for many flying animals, some of which have no dedicated sense of gravity. Flying robots, on the other hand, typically depend heavily on accelerometers and gyroscopes for attitude stabilization. In this work, we present the first vision-only approach to flight control for use in generic environments. We show that a quadrotor drone equipped with a downward-facing event camera can estimate its attitude and rotation rate from just the event stream, enabling flight control without inertial sensors. Our approach uses a small recurrent convolutional neural network trained through supervised learning. Real-world flight tests demonstrate that our combination of event camera and low-latency neural network is capable of replacing the inertial measurement unit in a traditional flight control loop. Furthermore, we investigate the network's generalization across different environments, and the impact of memory and different fields of view. While networks with memory and access to horizon-like visual cues achieve best performance, variants with a narrower field of view achieve better relative generalization. Our work showcases vision-only flight control as a promising candidate for enabling autonomous, insect-scale flying robots.

Abstract (translated)

视觉在许多飞行动物的姿态控制中扮演着至关重要的角色,其中一些动物甚至没有专门的重力感知器官。相比之下,飞行机器人通常高度依赖于加速度计和陀螺仪来进行姿态稳定。在这项工作中,我们提出了首个仅依靠视觉信息进行飞行控制的方法,并适用于通用环境。我们展示了装备有向下朝向事件相机的四旋翼无人机能够仅通过事件流来估计其姿态和旋转速率,从而实现无需惯性传感器的飞行控制。我们的方法采用了一种小型循环卷积神经网络(RNN-CNN),并通过监督学习进行训练。 实际飞行测试表明,我们组合使用的事件相机和低延迟神经网络能够在传统飞行控制回路中替代惯性测量单元(IMU)。此外,我们还研究了该网络在不同环境中的泛化能力以及记忆和视野范围变化的影响。虽然具有记忆功能并能访问类似地平线视觉线索的网络表现最佳,但视野较窄的变体则表现出更好的相对泛化能力。 这项工作展示了仅依赖视觉信息进行飞行控制作为一种有前景的技术,能够为实现自主、昆虫规模级别的飞行机器人提供支持。

URL

https://arxiv.org/abs/2507.11302

PDF

https://arxiv.org/pdf/2507.11302.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot