Abstract
We introduce a U-net model for 360° acoustic source localization formulated as a spherical semantic segmentation task. Rather than regressing discrete direction-of-arrival (DoA) angles, our model segments beamformed audio maps (azimuth and elevation) into regions of active sound presence. Using delay-and-sum (DAS) beamforming on a custom 24-microphone array, we generate signals aligned with drone GPS telemetry to create binary supervision masks. A modified U-Net, trained on frequency-domain representations of these maps, learns to identify spatially distributed source regions while addressing class imbalance via the Tversky loss. Because the network operates on beamformed energy maps, the approach is inherently array-independent and can adapt to different microphone configurations without retraining from scratch. The segmentation outputs are post-processed by computing centroids over activated regions, enabling robust DoA estimates. Our dataset includes real-world open-field recordings of a DJI Air 3 drone, synchronized with 360° video and flight logs across multiple dates and locations. Experimental results show that U-net generalizes across environments, providing improved angular precision, offering a new paradigm for dense spatial audio understanding beyond traditional Sound Source Localization (SSL).
Abstract (translated)
我们介绍了一种用于360°声源定位的U-Net模型,该模型被形式化为一个球面语义分割任务。我们的方法不是回归离散的方向角(DoA),而是将波束形成的音频地图(方位和仰角)划分为有声音活动的区域。我们使用延迟与求和(DAS)波束形成技术在自定义24麦克风阵列上生成与无人机GPS数据对齐的信号,从而创建二值监督掩码。通过在这些地图的频域表示形式上训练修改后的U-Net模型,并采用Tversky损失来处理类别不平衡问题,该网络能够识别空间分布的声音源区域。由于网络操作基于波束形成的能量图,因此此方法本质上与阵列无关,可以适应不同麦克风配置而无需从头开始重新训练。 在激活区域内计算质心的后处理步骤使我们的模型能够提供稳健的方向角(DoA)估计。我们数据集包括在现实世界开放环境中录制的DJI Air 3无人机的声音样本,这些声音与多个日期和地点同步记录的360°视频和飞行日志相结合。 实验结果表明,U-Net模型能够在不同环境条件下泛化,并提供改进的角度精度,这为传统的声源定位(SSL)之外的空间音频理解提供了新的范式。
URL
https://arxiv.org/abs/2508.00307