[1]XU Wei,ZHENG Hao,YANG Zhongxue.Apex frame microexpression recognition based on dual attention model and transfer learning[J].CAAI Transactions on Intelligent Systems,2021,16(6):1015-1020.[doi:10.11992/tis.202010031]
Copy

Apex frame microexpression recognition based on dual attention model and transfer learning

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 6 Page number: 1015-1020 Column: 学术论文—机器学习 Public date: 2021-11-05
Title:
Apex frame microexpression recognition based on dual attention model and transfer learning
Author(s):
XU Wei1;  ZHENG Hao2;  YANG Zhongxue2
1. School of Computer Science and Engineering/School of Software, Guangxi Normal University, Guilin 541004, China;
2. School of Information and Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China
Keywords:
microexpression recognition; deep learning; Apex frame; dual attention model; ResNet18 network; Focal Loss function; macroexpression; transfer learning
CLC:
TP391
DOI:
10.11992/tis.202010031
Abstract:
Microexpression is of short duration and low intensity, and its recognition accuracy is generally not high. To address this problem, an improved deep learning recognition method is proposed. This method takes Apex frames in the microexpression video sequence and adopts the ResNet18 network integrating spatial and channel dual attention modules. Moreover, the method introduces the Focal Loss function to solve the imbalance of microexpression data samples and transfers the prior knowledge in the field of macroexpression recognition to the field of microexpression recognition to improve the recognition effect. Experiments were performed on the CASME II microexpression dataset using the “leave one out-cross validation” method. The results show that the method presented in this paper has a higher recognition accuracy and F1 value than other existing methods.
References:
[1] SHEN Xunbing, WU Qi, FU Xiaolan. Effects of the duration of expressions on the recognition of microexpressions[J]. Journal of Zhejiang University SCIENCE B, 2012, 13(3): 221-230.
[2] ZHAO Guoying, PIETIKAINEN M. Dynamic texture recognition using local binary patterns with an application to facial expressions[J]. IEEE transactions on pattern analysis and machine intelligence, 2007, 29(6): 915-928.
[3] WANG Yandan, SEE J, PHAN R C W, et al. LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition[C]//Proceedings of the 12th Asian Conference on Computer Vision. Singapore, Singapore, 2014: 525-537.
[4] 卢官明, 杨成, 杨文娟, 等. 基于LBP-TOP特征的微表情识别[J]. 南京邮电大学学报(自然科学版), 2017, 37(6): 1-7
LU Guanming, YANG Cheng, YANG Wenjuan, et al. Micro-expression recognition based on LBP-TOP features[J]. Journal of Nanjing University of Posts and Telecommunications (natural science edition), 2017, 37(6): 1-7
[5] WANG Sujing, YAN Wenjing, LI Xiaobai, et al. Micro-expression recognition using dynamic textures on tensor independent color space[C]//Proceedings of the 22nd International Conference on Pattern Recognition. Stockholm, Sweden, 2014: 4678-4683.
[6] WANG Sujing, YAN Wenjing, LI Xiaobai, et al. Micro-expression recognition using color spaces[J]. IEEE transactions on image processing, 2015, 24(12): 6034-6047.
[7] XU Feng, ZHANG Junping, WANG J Z. Microexpression identification and categorization using a facial dynamics map[J]. IEEE transactions on affective computing, 2017, 8(2): 254-267.
[8] LIU Yongjin, ZHANG Jinkai, YAN Wenjing, et al. A main directional mean optical flow feature for spontaneous micro-expression recognition[J]. IEEE transactions on affective computing, 2016, 7(4): 299-310.
[9] 张轩阁, 田彦涛, 郭艳君, 等. 基于光流与LBP-TOP特征结合的微表情识别[J]. 吉林大学学报(信息科学版), 2015, 33(5): 516-523
ZHANG Xuange, TIAN Yantao, GUO Yanjun, et al. Micro-Expression recognition based on feature combination of optical flow and LBP-TOP[J]. Journal of Jilin University (information science edition), 2015, 33(5): 516-523
[10] OH Y H, LE NGO A C, SEE J, et al. Monogenic Riesz wavelet representation for micro-expression recognition[C]//Proceedings of 2015 IEEE International Conference on Digital Signal Processing. Singapore, Singapore, 2015: 1237-1241.
[11] WU Qi, SHEN Xunbing, FU Xiaolan. The machine knows what you are hiding: an automatic micro-expression recognition system[C]//Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction. Memphis, TN, USA, 2011: 152-162.
[12] ZHANG Peng, BEN Xianye, YAN Rui, et al. Micro-expression recognition system[J]. Optik, 2016, 127(3): 1395-1400.
[13] PATEL D, HONG Xiaopeng, ZHAO Guoying. Selective deep features for micro-expression recognition[C]//Proceedings of the 23rd International Conference on Pattern Recognition. Cancun, Mexico, 2016: 2258-2263.
[14] PENG Min, WANG Chongyang, CHEN Tong, et al. Dual temporal scale convolutional neural network for micro-expression recognition[J]. Frontiers in psychology, 2017, 8: 1745.
[15] VERMA M, VIPPARTHI S K, SINGH G, et al. LEARNet: dynamic imaging network for micro expression recognition[J]. IEEE transactions on image processing, 2019, 29: 1618-1627.
[16] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany, 2018: 3-19.
[17] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016: 770-778.
[18] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017: 2980-2988.
[19] LIONG S T, SEE J, WONG K S, et al. Less is more: micro-expression recognition from video using apex frame[J]. Signal processing: image communication, 2018, 62: 82-92.
[20] GAN Y S, LIONG S T, YAU W C, et al. OFF-ApexNet on micro-expression recognition system[J]. Signal processing: image communication, 2019, 74: 129-139.
[21] YAN Wenjing, LI Xiaobai, WANG Sujing, et al. CASME II: an improved spontaneous micro-expression database and the baseline evaluation[J]. PLoS One, 2014, 9(1): e86041.
[22] LUCEY P, COHN J F, KANADE T, et al. The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. San Francisco, CA, USA, 2010: 94-101.
[23] LIONG S T, SEE J, PHAN R C W, et al. Subtle expression recognition using optical strain weighted features[C]//Proceedings of the Asian Conference on Computer Vision. Singapore, Singapore, 2014: 644-657.
[24] 孔慧芳, 钱世超, 闫嘉鹏. 基于不均衡数据与迁移学习的面部微表情识别[J]. 合肥工业大学学报(自然科学版), 2020, 43(7): 895-900
KONG Huifang, QIAN Shichao, YAN Jiapeng. Image based facial micro-expression recognition using transfer learning on imbalanced data[J]. Journal of Hefei University of Technology (natural science edition), 2020, 43(7): 895-900
Similar References:

Memo

-

Last Update: 2021-12-25

Copyright © CAAI Transactions on Intelligent Systems