Paper Reading AI Learner

Beamformed 360{deg} Sound Maps: U-Net-Driven Acoustic Source Segmentation and Localization

2025-08-01 04:23:18
Belman Jahir Rodriguez, Sergio F. Chevtchenko, Marcelo Herrera Martinez, Yeshwant Bethy, Saeed Afshar

Abstract

We introduce a U-net model for 360° acoustic source localization formulated as a spherical semantic segmentation task. Rather than regressing discrete direction-of-arrival (DoA) angles, our model segments beamformed audio maps (azimuth and elevation) into regions of active sound presence. Using delay-and-sum (DAS) beamforming on a custom 24-microphone array, we generate signals aligned with drone GPS telemetry to create binary supervision masks. A modified U-Net, trained on frequency-domain representations of these maps, learns to identify spatially distributed source regions while addressing class imbalance via the Tversky loss. Because the network operates on beamformed energy maps, the approach is inherently array-independent and can adapt to different microphone configurations without retraining from scratch. The segmentation outputs are post-processed by computing centroids over activated regions, enabling robust DoA estimates. Our dataset includes real-world open-field recordings of a DJI Air 3 drone, synchronized with 360° video and flight logs across multiple dates and locations. Experimental results show that U-net generalizes across environments, providing improved angular precision, offering a new paradigm for dense spatial audio understanding beyond traditional Sound Source Localization (SSL).

Abstract (translated)

我们介绍了一种用于360°声源定位的U-Net模型,该模型被形式化为一个球面语义分割任务。我们的方法不是回归离散的方向角(DoA),而是将波束形成的音频地图(方位和仰角)划分为有声音活动的区域。我们使用延迟与求和(DAS)波束形成技术在自定义24麦克风阵列上生成与无人机GPS数据对齐的信号,从而创建二值监督掩码。通过在这些地图的频域表示形式上训练修改后的U-Net模型,并采用Tversky损失来处理类别不平衡问题,该网络能够识别空间分布的声音源区域。由于网络操作基于波束形成的能量图,因此此方法本质上与阵列无关,可以适应不同麦克风配置而无需从头开始重新训练。 在激活区域内计算质心的后处理步骤使我们的模型能够提供稳健的方向角(DoA)估计。我们数据集包括在现实世界开放环境中录制的DJI Air 3无人机的声音样本,这些声音与多个日期和地点同步记录的360°视频和飞行日志相结合。 实验结果表明,U-Net模型能够在不同环境条件下泛化,并提供改进的角度精度,这为传统的声源定位(SSL)之外的空间音频理解提供了新的范式。

URL

https://arxiv.org/abs/2508.00307

PDF

https://arxiv.org/pdf/2508.00307.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot