Abstract
Real-time image segmentation is a key enabler for AR/VR, robotics, drones, and autonomous systems, where tight accuracy, latency, and energy budgets must be met on resource-constrained edge devices. While U-Net offers a favorable balance of accuracy and efficiency compared to large transformer-based models, achieving real-time performance on high-resolution input remains challenging due to compute, memory, and power limits. Extreme quantization, particularly binary networks, is appealing for its hardware-friendly operations. However, two obstacles limit practicality: (1) severe accuracy degradation, and (2) a lack of end-to-end implementations that deliver efficiency on general-purpose GPUs. We make two empirical observations that guide our design. (1) An explicit zero state is essential: training with zero masking to binary U-Net weights yields noticeable sparsity. (2) Quantization sensitivity is uniform across layers. Motivated by these findings, we introduce Masked Binary U-Net (MBU-Net), obtained through a cost-aware masking strategy that prioritizes masking where it yields the highest accuracy-per-cost, reconciling accuracy with near-binary efficiency. To realize these gains in practice, we develop a GPU execution framework that maps MBU-Net to Tensor Cores via a subtractive bit-encoding scheme, efficiently implementing masked binary weights with binary activations. This design leverages native binary Tensor Core BMMA instructions, enabling high throughput and energy savings on widely available GPUs. Across 3 segmentation benchmarks, MBU-Net attains near full-precision accuracy (3% average drop) while delivering 2.04x speedup and 3.54x energy reductions over a 16-bit floating point U-Net.
Abstract (translated)
实时图像分割是增强现实（AR）/虚拟现实（VR）、机器人技术、无人机和自主系统的关键推动力，这些应用需要在计算资源受限的边缘设备上实现高精度、低延迟和节能。虽然U-Net架构相比基于大型变换器模型而言，在准确性与效率之间提供了一个有利的平衡，但在处理高分辨率输入时要实现实时性能仍然面临挑战，因为这会受到计算能力、内存以及能耗限制的影响。极端量化（特别是二值网络）因其硬件友好的操作而具有吸引力。然而，有两个障碍阻碍了其实用性：一是准确性的严重下降；二是缺乏在通用GPU上实现高效能的端到端实施方案。 基于两个经验观察结果，我们设计出Masked Binary U-Net (MBU-Net)架构以克服上述挑战： 1. 显式的零态是必不可少的：使用零掩码训练二值U-Net权重能够显著提高稀疏性。 2. 量化敏感度在各层间均匀分布。 受到这些发现的启发，我们通过一种成本感知掩蔽策略引入了MBU-Net架构，这种策略优先选择那些能提供最高准确性和效率比的位置进行掩码处理，从而实现了准确性与近似二值化效率之间的协调。为了实现这一设计的实际效益，我们开发了一个GPU执行框架，该框架通过减法位编码方案将MBU-Net映射到Tensor Core上，并高效地实施了带掩蔽的二进制权重和激活函数。这种设计利用了原生的二进制Tensor Core BMMA指令，从而在广泛可用的GPU上实现了高吞吐量和节能。 在三个分割基准测试中，MBU-Net实现了接近全精度（平均下降3%）的同时，相较于16位浮点数U-Net分别提高了2.04倍的速度和3.54倍的能量效率。
URL
https://arxiv.org/abs/2601.11660
