[1]GAO Tao,YANG Zhaochen,CHEN Ting,et al.Deep multiscale fusion attention residual network for facial expression recognition[J].CAAI Transactions on Intelligent Systems,2022,17(2):393-401.[doi:10.11992/tis.202107028]
Copy

Deep multiscale fusion attention residual network for facial expression recognition

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 17 Number of periods: 2022 2 Page number: 393-401 Column: 学术论文—人工智能基础 Public date: 2022-03-05
Title:
Deep multiscale fusion attention residual network for facial expression recognition
Author(s):
GAO Tao1;  YANG Zhaochen1;  CHEN Ting1;  SHAO Qian1;  LEI Tao2
1. School of Information Engineering, Chang’an University, Xi’an 710000, China;
2. School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology, Xi’an 710021, China
Keywords:
facial expression recognition; residual network; multiscale features; attention mechanism; occlusion of human faces; convolution neural network; feature fusion; deep learning
CLC:
TP391
DOI:
10.11992/tis.202107028
Abstract:
This paper proposes a deep multiscale fusion attention residual network based on the ResNet-50 model to solve the problems of the diversification of facial expression presentation and the susceptibility of facial expression recognition to nonlinear factors, such as illumination, posture, and occlusion. A novel attention residual module consisting of seven attention residual learning units with three branches is designed to perform multiple convolution operations on the input image in parallel and obtain multiscale features. To highlight important local areas, the attention mechanism is introduced simultaneously, which is conducive to the feature learning of the occluded images. Furthermore, a novel transition layer is added between the attention residual modules to remove redundant information, simplify the network complexity, reduce the amount of calculation while ensuring the receptive field, and realize the anti-overfitting effect of the network. Experimental results on three datasets demonstrate that the proposed algorithm is superior to other advanced methods.
References:
[1] BEN Xianye, REN Yi, ZHANG Junping, et al. Video-based Facial micro-expression analysis: a survey of datasets, features and algorithms[EB/OL].(2021-03-19)[2021-05-01].https://arxiv.org/abs/2201.12728v1.
[2] CHEN Boyu, GUAN Wenlong, LI Peixia, et al. Residual multi-task learning for facial landmark localization and expression recognition[EB/OL].(2021-07-01)[2021-07-05].https://www.sciencedirect.com/science/article/pii/S0031320321000807.
[3] LI Shan, DENG Weihong. Deep facial expression recognition: a survey[EB/OL].（2020-03-17）[2021-05-01]. https://ieeexplore.ieee.org/document/9039580.
[4] ZHAO Guoying, PIETIKAINEN M. Dynamic texture recognition using local binary patterns with an application to facial expressions[J]. IEEE transactions on pattern analysis and machine intelligence, 2007, 29(6): 915–928.
[5] WHITEHILL J, OMLIN C W. Haar features for FACS AU recognition[C]//Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition. Southampton, UK, 2006: 5?101.
[6] BARTLETT M S, LITTLEWORT G, FRANK M, et al. Recognizing facial expression: machine learning and application to spontaneous behavior[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA, 2005: 568?573.
[7] LI Xiaobai, PFISTER T, HUANG Xiaohua, et al. A spontaneous micro-expression database: inducement, collection and baseline[C]//2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). Shanghai, China, 2013: 1?6.
[8] RIVERA A R, CASTILLO J R, CHA E O O. Local directional number pattern for face analysis: face and expression recognition[J]. IEEE transactions on image processing, 2013, 22(5): 1740–1752.
[9] KIM T H, YU C, LEE S W. Facial expression recognition using feature additive pooling and progressive fine-tuning of CNN[J]. Electronics letters, 2018, 54(23): 1326–1328.
[10] AN Fengping, LIU Zhiwen. Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM[J]. The visual computer, 2020, 36(3): 483–498.
[11] XIE Siyue, HU Haifeng, WU Yongbo. Deep multi-path convolutional neural network joint with salient region attention for facial expression recognition[J]. Pattern recognition, 2019, 92: 177–191.
[12] WANG Kai, PENG Xiaojiang, YANG Jianfei, et al. Suppressing uncertainties for large-scale facial expression recognition[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA, 2020: 6897?6906.
[13] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016: 770?778.
[14] LI Yong, ZENG Jiabei, SHAN Shiguang, et al. Occlusion aware facial expression recognition using CNN with attention mechanism[J]. IEEE transactions on image processing, 2019, 28(5): 2439–2450.
[15] LIU Yuanyuan, YUAN Xiaohui, GONG Xi, et al. Conditional convolution neural network enhanced random forest for facial expression recognition[J]. Pattern recognition, 2018, 84: 251–261.
[16] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 7132?7141.
[17] 江静, 邓伟洪. 持续学习改进的人脸表情识别[J]. 中国图象图形学报, 2020, 25(11): 2361–2369
JIANG Jing, DENG Weihong. Facial expression recognition improved by continual learning[J]. Journal of image and graphics, 2020, 25(11): 2361–2369
[18] 王善敏, 帅惠, 刘青山. 关键点深度特征驱动人脸表情识别[J]. 中国图象图形学报, 2020, 25(4): 813–823
WANG Shanmin, SHUAI Hui, LIU Qingshan. Facial expression recognition based on deep facial landmark features[J]. Journal of image and graphics, 2020, 25(4): 813–823
[19] 张文萍, 贾凯, 王宏玉, 等. 改进的Island损失函数在人脸表情识别上的应用[J]. 计算机辅助设计与图形学学报, 2020, 32(12): 1910–1917
ZHANG Wenping, JIA Kai, WANG Hongyu, et al. Application of improved Island loss in facial expression recognition[J]. Journal of computer-aided design & computer graphics, 2020, 32(12): 1910–1917
[20] LIU Xiaofeng, KUMAR B V K V, JIA Ping, et al. Hard negative generation for identity-disentangled facial expression recognition[J]. Pattern recognition, 2019, 88: 1–12.
[21] YANG Huiyuan, CIFTCI U, YIN Lijun. Facial expression recognition by de-expression residue learning[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018.
[22] XIE Siyue, HU Haifeng. Facial expression recognition using hierarchical features with deep comprehensive multipatches aggregation convolutional neural networks[J]. IEEE transactions on multimedia, 2019, 21(1): 211–220.
[23] LOPES A T, DE AGUIAR E, DE SOUZA A F, et al. Facial expression recognition with convolutional neural networks: coping with few data and the training sample order[J]. Pattern recognition, 2017, 61: 610–628.
[24] ZHANG Hepeng, HUANG Bin, TIAN Guohui. Facial expression recognition based on deep convolution long short-term memory networks of double-channel weighted mixture[J]. Pattern recognition letters, 2020, 131: 128–134.
[25] YANG Biao, CAO Jinmeng, NI Rongrong, et al. Facial expression recognition using weighted mixture deep neural network based on double-channel facial images[J]. IEEE access, 2017, 6: 4630–4640.
[26] KIM J H, KIM B G, ROY P P, et al. Efficient facial expression recognition algorithm based on hierarchical deep neural network structure[J]. IEEE access, 2019, 7: 41273–41285.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems