Abstract
Vision is an essential part of attitude control for many flying animals, some of which have no dedicated sense of gravity. Flying robots, on the other hand, typically depend heavily on accelerometers and gyroscopes for attitude stabilization. In this work, we present the first vision-only approach to flight control for use in generic environments. We show that a quadrotor drone equipped with a downward-facing event camera can estimate its attitude and rotation rate from just the event stream, enabling flight control without inertial sensors. Our approach uses a small recurrent convolutional neural network trained through supervised learning. Real-world flight tests demonstrate that our combination of event camera and low-latency neural network is capable of replacing the inertial measurement unit in a traditional flight control loop. Furthermore, we investigate the network's generalization across different environments, and the impact of memory and different fields of view. While networks with memory and access to horizon-like visual cues achieve best performance, variants with a narrower field of view achieve better relative generalization. Our work showcases vision-only flight control as a promising candidate for enabling autonomous, insect-scale flying robots.
Abstract (translated)
视觉在许多飞行动物的姿态控制中扮演着至关重要的角色,其中一些动物甚至没有专门的重力感知器官。相比之下,飞行机器人通常高度依赖于加速度计和陀螺仪来进行姿态稳定。在这项工作中,我们提出了首个仅依靠视觉信息进行飞行控制的方法,并适用于通用环境。我们展示了装备有向下朝向事件相机的四旋翼无人机能够仅通过事件流来估计其姿态和旋转速率,从而实现无需惯性传感器的飞行控制。我们的方法采用了一种小型循环卷积神经网络(RNN-CNN),并通过监督学习进行训练。 实际飞行测试表明,我们组合使用的事件相机和低延迟神经网络能够在传统飞行控制回路中替代惯性测量单元(IMU)。此外,我们还研究了该网络在不同环境中的泛化能力以及记忆和视野范围变化的影响。虽然具有记忆功能并能访问类似地平线视觉线索的网络表现最佳,但视野较窄的变体则表现出更好的相对泛化能力。 这项工作展示了仅依赖视觉信息进行飞行控制作为一种有前景的技术,能够为实现自主、昆虫规模级别的飞行机器人提供支持。
URL
https://arxiv.org/abs/2507.11302