Contrastive feature learning and class-weighted loss for facial action unit detection

Bing Fei Wu, Yin Tse Wei, Bing Jhang Wu, Chun Hsien Lin

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Facial action unit detection (FAUD) is aimed to detect subtle facial motions, known as action units (AUs), induced by the traction or towing of facial muscles. In recent years, FAUD has become an attractive task since these slight changes on face reveal some cues of emotions which can then be utilized to infer people's affective. Many prior works are developed based on physical features of AUs, such as regional occurrence and temporal continuity, but few of them concern the negative impact of individual differences, causing an unrobust result in variant subjects and environments. To deal with the problem, a contrastive feature learning method is proposed to make a convolutional neural network (CNN) learn to extract the contrastive feature which is the difference between the features of a neutral face image and an AU-occurred face image. In this way, the individual information will be mitigated so that it becomes easier to detect AUs according to the features. A great disparity between the number of positive and negative samples, named as data imbalance, makes a serious interference in most detection problems, including FAUD. A lower occurrence rate of an AU leads a more severe data imbalance problem, which results in a biased result for the AU detection. Therefore, the class-weighted loss is proposed to change the weight-ratio between the positive and negative samples so as to encourage the learning of the positive samples and ease that of negative samples. Two widely used databases, BP4D and DISFA, are adopted to be the benchmark of the performance testing. In the experiment, it shows that the proposed method performs well both in accuracy and speed comparing with the state-of-the-art approaches.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2478-2483
Number of pages6
ISBN (Electronic)9781728145693
DOIs
StatePublished - Oct 2019
Event2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019 - Bari, Italy
Duration: 6 Oct 20199 Oct 2019

Publication series

NameConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
Volume2019-October
ISSN (Print)1062-922X

Conference

Conference2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019
CountryItaly
CityBari
Period6/10/199/10/19

Fingerprint Dive into the research topics of 'Contrastive feature learning and class-weighted loss for facial action unit detection'. Together they form a unique fingerprint.

Cite this