Bandwidth and Energy Efficient CNN Acceleration for Next Generation Cameras and Displays

Speaker: Prof. Chao-Tsung Huang (Department of Electrical Engineering, National Tsing Hua University)
In the era of artificial intelligence, convolutional neural networks (CNNs) are emerging as a powerful technique for image processing, such as denoising, super-resolution, and even style transfer. They show great potential to bring next-generation cameras and displays to our daily life. However, it is difficult for conventional accelerators to generate ultra-high-resolution videos at the edge due to considerable DRAM bandwidth and power consumption. In this talk, I will introduce two of our recent works on tackling these challenges. The first work-CINE-is a computational imaging neural engine with overlapped stripe inference flow and structure-sparse convolution. Given only 0.9GB/s of DRAM bandwidth, it provides 4K-UHD applications with 4.6-8.3 TOPS/W of efficiency. The second work-VISTA-achieves video/image spatial/temporal interpolation acceleration with cuboid-based layer fusion and ring-algebra computation sparsity. It supports video CNNs for 4K-UHD displays while consuming only 704 mW of power and less than 3 GB/s of DRAM bandwidth.