A 2.17-mW Acoustic DSP Processor With CNN-FFT Accelerators for Intelligent Hearing Assistive Devices

Yu-Chi Lee, Tai-Shih Chi, Chia-Hsiang Yang*

*Corresponding author for this work

Research output: Contribution to journalArticle


This article presents an acoustic DSP processor containing a neural network core for intelligent hearing assistive devices. The processor includes the accelerators for convolutional neural networks (CNNs) and fast Fourier transform (FFT). The CNN-based speech enhancement algorithm predicts the desired mask for the Fourier spectrogram of the speech signal to enhance speech intelligibility. Several design techniques are applied to enable efficient hardware mapping. The computational complexity for the CNN can be reduced by 23.6% by frame sharing, and a fast mask generation + partial sums pre-computation technique further reduces output latency by up to 64%. The size of the memory for the model is reduced by 75% using weight quantization. FFT is implemented by leveraging the packing algorithm to reduce the computational complexity by 43%. Reconfigurable processing elements are shared to support both FFT and CNN, realizing a saving in the area of 42%. In addition, input sharing and output sharing are used to, respectively, reduce data movements by 94% and 75%. A reordered FFT structure also eliminates up to 256 multiplexers. Fabricated in a 40-nm CMOS technology, the chip's core area is 4.2 mm(2) and the power dissipation is 2.17 mW at a clock frequency of 5 MHz from a 0.6-V supply. The embedded CNN accelerator supports both convolutional and fully connected (FC) layers and achieves a comparable energy efficiency with state-of-the-art CNN accelerators, despite the flexibility for FFT. The speech intelligibility is enhanced by up to 41% in the low SNR regime.

Original languageEnglish
Pages (from-to)2247-2258
Number of pages12
JournalIEEE Journal of Solid-State Circuits
Issue number8
StatePublished - Aug 2020


  • Speech enhancement
  • Convolution
  • Discrete Fourier transforms
  • Spectrogram
  • Acoustics
  • Hardware
  • Computational modeling
  • CMOS integrated circuits
  • convolutional neural network (CNN)
  • fast Fourier transform (FFT)
  • reconfigurable architecture
  • speech enhancement

Cite this