Abstract
Neural network accelerators have been widely applied to edge devices for complex tasks like object tracking, image recognition, etc. Previous works have explored the quantization technologies in related lightweight accelerator designs to reduce hardware resource consumption. However, low precision leads to high accuracy loss in inference. Therefore, mixed-precision quantization becomes an alternative solution by applying different precision in different layers to trade off resource consumption and accuracy. Because regular designs for multiplication on hardware cannot support the precision reconfiguration for a multi-precision Quantized Neural Network (QNN) model in runtime, we propose a runtime reconfigurable multi-precision multi-channel bitwise systolic array design for QNN accelerators. We have implemented and evaluated our work on the Ultra96 FPGA platform. Results show that our work can achieve 1.3185 to 3.5671 times speedup in inferring mixed-precision models and has less critical path delay, supporting a higher clock frequency (250MHz).
Abstract (translated)
神经网络加速器已被广泛应用于边缘设备，用于执行如目标跟踪、图像识别等复杂任务。此前的研究工作已经探索了在相关轻量级加速器设计中应用量化技术以减少硬件资源消耗的方法。然而，低精度会导致推理时的高准确性损失。因此，混合精度量化成为了一种替代方案，通过在不同的层采用不同的精度来权衡资源消耗和准确性的取舍。由于硬件上常规的设计无法支持多精度量化神经网络（QNN）模型在运行时重新配置精度，我们提出了一种针对QNN加速器的可重构多精度多通道位级阵列设计。我们在Ultra96 FPGA平台上实现了并评估了该设计方案。结果显示，我们的方案可以在推理混合精度模型上实现1.3185到3.5671倍的速度提升，并且减少了关键路径延迟，支持更高的时钟频率（250MHz）。
URL
https://arxiv.org/abs/2602.23334
