ProCon-V: A Programmable Tightly Coupled Convolution Accelerator Based on RISC-V Custom Instructions for Edge Devices

The increasing use of deep neural network (DNN) based applications on edge devices has driven the need for innovative computing solutions that can meet the high computational requirements of DNN workloads while ensuring low power consumption. Therefore, enhancing the general-purpose (GP) computing units of edge devices by integrating DNN-specific functions and operations into existing instruction set architectures (ISAs) provides the flexibility to support a wide variety of neural network models, thereby improving computing efficiency. This article proposes a programmable tightly coupled accelerator for convolution operations called ProCon-V. It is based on RISCV custom instructions, extending the standard ISA to support 2-D convolution operations. The accelerator is tightly coupled with a RISC-V-based pipeline through an open-source extension interface [21], offloading convolution operations from the pipeline’s main execution unit. The ProCon-V computing engine is based on a scalable systolic array architecture for partial-sum computation, with hardware-supported Im2Col transformation that supports multi-channel convolution layers with various configurations. Moreover, the tightly coupled accelerator enables the execution of convolution layers through a set of custom instructions handled by the RISC-V core without requiring extra modifications to the core pipeline. ProCon-V is implemented and evaluated on an AMD/Xilinx Virtex Ultrascale+ FPGA featuring dual clock domains for the RISC-V core and the convolution accelerator. Evaluation results demonstrate that the proposed accelerator features a maximum computing throughput of 174.4 GOPS for INT8-based operations with an energy efficiency of 181.7 GOPS/W.