A new information fusion method for SVM-based robotic audio-visual emotion recognition

Meng Ju Han*, Jing Huai Hsu, Kai-Tai Song, Fuh Yu Chang

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations

Abstract

Emotion recognition has become an important research area for advanced human-robot interaction. Through recognizing facial expressions, a robot can interact with a person with a more friendly manner. In this paper, we proposed a bimodal emotion recognition system by combining image and speech information. A novel information fusion strategy is proposed to set proper weights to two feature modalities based on their recognition reliability. The fusion weights are determined by the distance between test data and the classification hyperplane and the standard deviation of training samples. After normalization using the mean distance between training samples and the hyperplane, the fusion weight is set to represent the classification reliability of individual modality. In the latter bimodal SVM classification, the recognition result with higher weight is selected. The complete procedure has been implemented in a DSP-based system to recognize five facial expressions on-line in real time. The experimental results show that a recognition rate of 86.9% is achieved, an improvement of 5% compared with using only image information.

Original languageEnglish
Title of host publication2007 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2007
Pages2656-2661
Number of pages6
DOIs
StatePublished - 1 Dec 2007
Event2007 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2007 - Montreal, QC, Canada
Duration: 7 Oct 200710 Oct 2007

Publication series

NameConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
ISSN (Print)1062-922X

Conference

Conference2007 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2007
CountryCanada
CityMontreal, QC
Period7/10/0710/10/07

Fingerprint Dive into the research topics of 'A new information fusion method for SVM-based robotic audio-visual emotion recognition'. Together they form a unique fingerprint.

  • Cite this

    Han, M. J., Hsu, J. H., Song, K-T., & Chang, F. Y. (2007). A new information fusion method for SVM-based robotic audio-visual emotion recognition. In 2007 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2007 (pp. 2656-2661). [4413990] (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). https://doi.org/10.1109/ICSMC.2007.4413990