Abstract
Learning-based control approaches like reinforcement learning (RL) have recently produced a slew of impressive results for tasks like quadrotor trajectory tracking and drone racing. Naturally, it is common to demonstrate the advantages of these new controllers against established methods like analytical controllers. We observe, however, that reliably comparing the performance of such very different classes of controllers is more complicated than might appear at first sight. As a case study, we take up the problem of agile tracking of an end-effector for a quadrotor with a fixed arm. We develop a set of best practices for synthesizing the best-in-class RL and geometric controllers (GC) for benchmarking. In the process, we resolve widespread RL-favoring biases in prior studies that provide asymmetric access to: (1) the task definition, in the form of an objective function, (2) representative datasets, for parameter optimization, and (3) feedforward information, describing the desired future trajectory. The resulting findings are the following: our improvements to the experimental protocol for comparing learned and classical controllers are critical, and each of the above asymmetries can yield misleading conclusions. Prior works have claimed that RL outperforms GC, but we find the gaps between the two controller classes are much smaller than previously published when accounting for symmetric comparisons. Geometric control achieves lower steady-state error than RL, while RL has better transient performance, resulting in GC performing better in relatively slow or less agile tasks, but RL performing better when greater agility is required. Finally, we open-source implementations of geometric and RL controllers for these aerial vehicles, implementing best practices for future development. Website and code is available at this https URL
Abstract (translated)
基于学习的控制方法,如强化学习(RL),最近在诸如四旋翼飞行器轨迹跟踪和无人机竞速等任务上取得了令人瞩目的成果。自然地,在展示这些新控制器相对于分析型控制器的优势时,往往会进行对比演示。然而,我们观察到可靠地比较这两种非常不同的控制器类型的表现比乍一看要复杂得多。 作为案例研究,我们探讨了四旋翼飞行器固定臂末端执行器敏捷跟踪的问题,并为此开发了一套最佳实践方法,用于综合最佳的RL和几何控制(GC)控制器以进行基准测试。在此过程中,我们解决了先前研究中的广泛存在的偏向于RL的偏见问题,这些问题提供了非对称访问: 1. 任务定义的形式为目标函数; 2. 代表性数据集,用于参数优化; 3. 描述期望未来轨迹的前馈信息。 结果发现如下:我们在比较学习型控制器和经典控制器方面的实验方法改进至关重要,并且上述每种不对称情况都可能导致误导性结论。先前的研究曾声称RL优于GC,但当我们考虑对称对比时,这两种控制类别的差距远比之前发布的要小得多。几何控制在稳态误差方面低于RL,而RL在瞬态性能上表现更好,这导致GC在相对较慢或敏捷度要求较低的任务中表现出色,而在需要更高敏捷性的情况下则不如RL。 最后,我们开源了这些空中飞行器的几何和RL控制器的实现,并为未来的开发实施最佳实践。该网站和代码可在以下网址获得:[https URL](实际操作时请替换为正确的URL)。
URL
https://arxiv.org/abs/2506.17832