Abstract
Sub-30g nano-sized aerial robots can leverage their agility and form factor to autonomously explore cluttered and narrow environments, like in industrial inspection and search and rescue missions. However, the price for their tiny size is a strong limit in their resources, i.e., sub-100 mW microcontroller units (MCUs) delivering $\sim$100 GOps/s at best, and memory budgets well below 100 MB. Despite these strict constraints, we aim to enable complex vision-based tasks aboard nano-drones, such as dense 3D scene reconstruction: a key robotic task underlying fundamental capabilities like spatial awareness and motion planning. Top-performing 3D reconstruction methods leverage neural radiance fields (NeRF) models, which require GBs of memory and massive computation, usually delivered by high-end GPUs consuming 100s of Watts. Our work introduces Tiny-DroNeRF, a lightweight NeRF model, based on Instant-NGP, and optimized for running on a GAP9 ultra-low-power (ULP) MCU aboard our nano-drones. Then, we further empower our Tiny-DroNeRF by leveraging a collaborative federated learning scheme, which distributes the model training among multiple nano-drones. Our experimental results show a 96% reduction in Tiny-DroNeRF's memory footprint compared to Instant-NGP, with only a 5.7 dB drop in reconstruction accuracy. Finally, our federated learning scheme allows Tiny-DroNeRF to train with an amount of data otherwise impossible to keep in a single drone's memory, increasing the overall reconstruction accuracy. Ultimately, our work combines, for the first time, NeRF training on an ULP MCU with federated learning on nano-drones.
Abstract (translated)
重量低于30克的纳米级空中机器人可以利用其敏捷性和外形尺寸自主探索拥挤和狭窄的环境，例如在工业检查和搜索救援任务中。然而，它们的小尺寸带来了资源限制的代价，即微控制器单元（MCU）提供的计算能力不超过100毫瓦，提供大约100 GOPs/s 的处理速度，并且内存预算低于100MB。尽管存在这些严格的约束条件，我们的目标是在纳米无人机上启用复杂的基于视觉的任务，例如密集的3D场景重建：这是机器人技术中支持空间感知和运动规划等基本能力的关键任务之一。 高性能的3D重建方法通常依赖于神经辐射场（NeRF）模型，这类模型需要数GB的内存以及大量的计算量，这通常由消耗数百瓦电力的高端GPU提供。我们的研究工作介绍了一种基于Instant-NGP且轻量化优化过的Tiny-DroNeRF NeRF模型，并针对我们纳米无人机上搭载的GAP9超低功耗（ULP）MCU进行了专门优化。 接下来，通过利用一种协作联邦学习方案，我们将进一步增强我们的Tiny-DroNeRF，该方案将模型训练任务分发到多个纳米无人机中。实验结果显示，与Instant-NGP相比，Tiny-DroNeRF的内存占用减少了96%，同时仅导致重建精度下降了5.7dB。此外，通过联邦学习方案，Tiny-DroNeRF能够使用单个无人机无法存储的数据量进行训练，从而提高了整体重建精度。 最终，我们的工作首次实现了在超低功耗微控制器单元（ULP MCU）上对神经辐射场模型进行训练，并结合纳米级无人机上的联邦学习。这一组合为资源受限的微型机器人系统带来了重要的技术突破，开启了新的可能性。
URL
https://arxiv.org/abs/2603.01850
