[1]WU Xiru,LING Xingyu.Facial expression recognition based on improved Faster RCNN[J].CAAI Transactions on Intelligent Systems,2021,16(2):210-217.[doi:10.11992/tis.201910020]
Copy

Facial expression recognition based on improved Faster RCNN

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 2 Page number: 210-217 Column: 学术论文—机器学习 Public date: 2021-03-05
Title:
Facial expression recognition based on improved Faster RCNN
Author(s):
WU Xiru;  LING Xingyu
College of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin 541004, China
Keywords:
target detection; deep learning; expression recognition; Faster RCNN; feature extraction; classification and recognition; multi-target recognition; multi-target location
CLC:
TP391.4
DOI:
10.11992/tis.201910020
Abstract:
To address the problem of the low accuracy rate of the multi-target facial expression classification and recognition algorithm in real environments, in this paper we propose a facial expression detection algorithm based on an improved faster region-based convolutional neural network (RCNN). The proposed algorithm uses a two-stage detection network to accomplish multi-target recognition and location in facial expression recognition. Instead of the original feature extraction module, densely connected convolutional networks are used, which can fuse multi-level feature information, increase network depth, and prevent network gradient disappearance. Soft non-maximum suppression (NMS) is used to improve the candidate-box merging strategy, and the attenuation function is designed to replace the traditional NMS greedy algorithm, thereby preventing the missed detection of adjacent or overlapping targets and improving the detection accuracy of the network under multi-target conditions. Through the construction of an expression data set in a real environment and an experiment based on the improved Faster RCNN, the facial expression of the target was detected in different scenes with a detection accuracy rate 5% higher than that of the original detection model. Therefore, good accuracy is achieved by the proposed algorithm.
References:
[1] CALVO R A, D’MELLO S. Affect detection:an interdisciplinary review of models, methods, and their applications[J]. IEEE transactions on affective computing, 2010, 1(1):18-37.
[2] IZARD C E. Innate and universal facial expressions:evidence from developmental and cross-cultural research[J]. Psychological bulletin, 1994, 115(2):288-299.
[3] SUN Zhe, HU Zhengping, ZHAO Mengyao. Automatically query active features based on pixel-level for facial expression recognition[J]. IEEE access, 2019, 7:104630-104641.
[4] ALPHONSE A S, STARVIN M S. A novel maximum and minimum response-based Gabor (MMRG) feature extraction method for facial expression recognition[J]. Multimedia tools and applications, 2019, 78(16):23369-23397.
[5] 童莹. 一种方向性的局部二值模式在人脸表情识别中的应用[J]. 智能系统学报, 2015, 10(3):422-428
TONG Ying. Local binary pattern based on the directions and its application in facial expression recognition[J]. CAAI transactions on intelligent systems, 2015, 10(3):422-428
[6] SADEGHI H, RAIE A A. Histogram distance metric learning for facial expression recognition[J]. Journal of visual communication and image representation, 2019, 62:152-165.
[7] HAPPY S L, ROUTRAY A. Automatic facial expression recognition using features of salient facial patches[J]. IEEE transactions on affective computing, 2015, 6(1):1-12.
[8] BEJAOUI H, GHAZOUANI H, BARHOUMI W. Sparse coding-based representation of LBP difference for 3D/4D facial expression recognition[J]. Multimedia tools and applications, 2019, 78(16):22773-22796.
[9] LI Ruiqi, TIAN Jing, CHUA M C H. Facial expressi on classification using salient pattern driven integrated geometric and textual features[J]. Multimedia tools and applications, 2019, 78(20):28971-28983.
[10] ZANGENEH E, MORADI A. Facial expression recognition by using differential geometric features[J]. The imaging science journal, 2018, 66(8):463-470.
[11] HAPPY S L, ROUTRAY A. Fuzzy histogram of optical flow orientations for micro-expression recognition[J]. IEEE transactions on affective computing, 2019, 10(3):394-406.
[12] LU Hua, KPALMA K, RONSIN J. Motion descriptors for micro-expression recognition[J]. Signal processing:image communication, 2018, 67:108-117.
[13] 刘帅师, 郭文燕, 张言, 等. 鲁棒的正则化编码随机遮挡表情识别[J]. 智能系统学报, 2018, 13(2):261-268
LIU Shuaishi, GUO Wenyan, ZHANG Yan, et al. Recognition of facial expression in case of random shielding based on ro-bust regularized coding[J]. CAAI transactions on intelligent systems, 2018, 13(2):261-268
[14] HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786):504-507.
[15] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural computation, 2006, 18(7):1527-1554.
[16] YANG Biao, CAO Jinmeng, NI Rongrong, et al. Facial expression recognition using weighted mixture deep neural network based on double-channel facial images[J]. IEEE access, 2017, 6:4630-4640.
[17] WU Min, SU Wanjuan, CHEN Luefeng, et al. Weight-adapted convolution neural network for facial expression recognition in Human-robot interaction[J]. IEEE transactions on systems, man, and cybernetics:systems, 2019, 51(3):1473-1484.
[18] SALMAM F Z, MADANI A, KISSI M. Fusing multi-stream deep neural networks for facial expression recognition[J]. Signal, image and video processing, 2019, 13(3):609-616.
[19] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, Ohio, USA, 2014:580-587
[20] GIRSHICK R. Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile, 2015:1440-1448.
[21] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6):1137-1149.
[22] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//Proceedings of the 13th European Conference on Computer Vision (ECCV). Zurich, Switzerland:Springer, 2014:818-833.
[23] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2019-12-12].http//:arXiv:1409.1556, 2014.
[24] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA, 2016:770-778.
[25] LYONS M, AKAMATSU S, KAMACHI M, et al. Coding facial expressions with Gabor wavelets[C/OL]. Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition. Nara, Japan:IEEE, 1998.[2012-12-05]. http://www.doc88.com/p-6921152816768.html.
Similar References:

Memo

-

Last Update: 2021-04-25

Copyright © CAAI Transactions on Intelligent Systems