Development and Validation of an EEG-Based Real-Time Emotion Recognition System Using Edge AI Computing Platform With Convolutional Neural Network System-on-Chip Design

Wai Chi Fang, Kai Yen Wang*, Nicolas Fahier, Yun Lung Ho, Yu De Huang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

This study proposed an electroencephalogram (EEG)-based real-time emotion recognition hardware system architecture based on multiphase convolutional neural network (CNN) algorithm implemented on a 28-nm technology chip and on field programmable gate array (FPGA) for binary and quaternary classification. Sample entropy, differential asymmetry, short-time Fourier transform, and a channel reconstruction method were used for emotion feature extraction. In this work, six EEG channels were selected (FP1, FP2, F3, F4, F7, and F8), and EEG images were generated from spectrogram fusions. The complete CNN architecture included training and acceleration for efficient artificial intelligence (AI) edge application, and we proposed a multiphase CNN execution method to accommodate hardware resource constraints. Datasets of 32 subjects from the DEAP database were used to validate the proposed design, exhibiting mean accuracies for valance binary classification and valance-arousal quaternary classification of 83.36% and 76.67%, respectively. The core area and total power consumption of the CNN chip were 1.83 1.83 mm2 , respectively, and 76.61 mW. The chip operation was validated using ADVANTEST V93000 PS1600, and the training process and real-time classification processing time took 0.12495 ms and 0.02634 ms for each EEG image, respectively. The proposed EEG-based real-time emotion recognition system included a dry electrode EEG headset, feature extraction processor, CNN chip platform, and graphical user interface, and the execution time costed 450 ms for each emotional state recognition.

Original languageEnglish
Article number8890664
Pages (from-to)645-657
Number of pages13
JournalIEEE Journal on Emerging and Selected Topics in Circuits and Systems
Volume9
Issue number4
DOIs
StatePublished - Dec 2019

Keywords

  • affective computing
  • convolutional neural network (CNN)
  • electroencephalography
  • Emotion recognition
  • system-on-chip

Fingerprint Dive into the research topics of 'Development and Validation of an EEG-Based Real-Time Emotion Recognition System Using Edge AI Computing Platform With Convolutional Neural Network System-on-Chip Design'. Together they form a unique fingerprint.

Cite this