[1]徐玮,郑豪,杨种学.基于双注意力模型和迁移学习的Apex帧微表情识别[J].智能系统学报,2021,16(6):1015-1020.[doi:10.11992/tis.202010031]
 XU Wei,ZHENG Hao,YANG Zhongxue.Apex frame microexpression recognition based on dual attention model and transfer learning[J].CAAI Transactions on Intelligent Systems,2021,16(6):1015-1020.[doi:10.11992/tis.202010031]
点击复制

基于双注意力模型和迁移学习的Apex帧微表情识别(/HTML)
分享到:

《智能系统学报》[ISSN:1673-4785/CN:23-1538/TP]

卷:
第16卷
期数:
2021年6期
页码:
1015-1020
栏目:
学术论文—机器学习
出版日期:
2021-11-05

文章信息/Info

Title:
Apex frame microexpression recognition based on dual attention model and transfer learning
作者:
徐玮1 郑豪2 杨种学2
1. 广西师范大学 计算机科学与工程学院/软件学院,广西 桂林 541004;
2. 南京晓庄学院 信息工程学院,江苏 南京 211171
Author(s):
XU Wei1 ZHENG Hao2 YANG Zhongxue2
1. School of Computer Science and Engineering/School of Software, Guangxi Normal University, Guilin 541004, China;
2. School of Information and Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China
关键词:
微表情识别深度学习Apex帧双注意力模型ResNet18网络Focal Loss函数宏表情迁移学习
Keywords:
microexpression recognitiondeep learningApex framedual attention modelResNet18 networkFocal Loss functionmacroexpressiontransfer learning
分类号:
TP391
DOI:
10.11992/tis.202010031
摘要:
微表情具有持续时间短、强度低的特点,其识别准确率普遍不高。针对该问题提出了一种改进的深度学习识别方法,该方法取微表情视频序列中的Apex帧,采用集成空间、通道双注意力模块的ResNet18网络,引入Focal Loss函数解决微表情数据样本不平衡的问题,并将宏表情识别领域的先验知识迁移到微表情识别领域,以提高识别效果。在CASME II微表情数据集上使用“留一交叉验证法”进行实验,结果表明本文方法相比一些现有的方法识别准确率及 ${F_1}$ 值更高。
Abstract:
Microexpression is of short duration and low intensity, and its recognition accuracy is generally not high. To address this problem, an improved deep learning recognition method is proposed. This method takes Apex frames in the microexpression video sequence and adopts the ResNet18 network integrating spatial and channel dual attention modules. Moreover, the method introduces the Focal Loss function to solve the imbalance of microexpression data samples and transfers the prior knowledge in the field of macroexpression recognition to the field of microexpression recognition to improve the recognition effect. Experiments were performed on the CASME II microexpression dataset using the “leave one out-cross validation” method. The results show that the method presented in this paper has a higher recognition accuracy and F1 value than other existing methods.

参考文献/References:

[1] SHEN Xunbing, WU Qi, FU Xiaolan. Effects of the duration of expressions on the recognition of microexpressions[J]. Journal of Zhejiang University SCIENCE B, 2012, 13(3): 221-230.
[2] ZHAO Guoying, PIETIKAINEN M. Dynamic texture recognition using local binary patterns with an application to facial expressions[J]. IEEE transactions on pattern analysis and machine intelligence, 2007, 29(6): 915-928.
[3] WANG Yandan, SEE J, PHAN R C W, et al. LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition[C]//Proceedings of the 12th Asian Conference on Computer Vision. Singapore, Singapore, 2014: 525-537.
[4] 卢官明, 杨成, 杨文娟, 等. 基于LBP-TOP特征的微表情识别[J]. 南京邮电大学学报(自然科学版), 2017, 37(6): 1-7
LU Guanming, YANG Cheng, YANG Wenjuan, et al. Micro-expression recognition based on LBP-TOP features[J]. Journal of Nanjing University of Posts and Telecommunications (natural science edition), 2017, 37(6): 1-7
[5] WANG Sujing, YAN Wenjing, LI Xiaobai, et al. Micro-expression recognition using dynamic textures on tensor independent color space[C]//Proceedings of the 22nd International Conference on Pattern Recognition. Stockholm, Sweden, 2014: 4678-4683.
[6] WANG Sujing, YAN Wenjing, LI Xiaobai, et al. Micro-expression recognition using color spaces[J]. IEEE transactions on image processing, 2015, 24(12): 6034-6047.
[7] XU Feng, ZHANG Junping, WANG J Z. Microexpression identification and categorization using a facial dynamics map[J]. IEEE transactions on affective computing, 2017, 8(2): 254-267.
[8] LIU Yongjin, ZHANG Jinkai, YAN Wenjing, et al. A main directional mean optical flow feature for spontaneous micro-expression recognition[J]. IEEE transactions on affective computing, 2016, 7(4): 299-310.
[9] 张轩阁, 田彦涛, 郭艳君, 等. 基于光流与LBP-TOP特征结合的微表情识别[J]. 吉林大学学报(信息科学版), 2015, 33(5): 516-523
ZHANG Xuange, TIAN Yantao, GUO Yanjun, et al. Micro-Expression recognition based on feature combination of optical flow and LBP-TOP[J]. Journal of Jilin University (information science edition), 2015, 33(5): 516-523
[10] OH Y H, LE NGO A C, SEE J, et al. Monogenic Riesz wavelet representation for micro-expression recognition[C]//Proceedings of 2015 IEEE International Conference on Digital Signal Processing. Singapore, Singapore, 2015: 1237-1241.
[11] WU Qi, SHEN Xunbing, FU Xiaolan. The machine knows what you are hiding: an automatic micro-expression recognition system[C]//Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction. Memphis, TN, USA, 2011: 152-162.
[12] ZHANG Peng, BEN Xianye, YAN Rui, et al. Micro-expression recognition system[J]. Optik, 2016, 127(3): 1395-1400.
[13] PATEL D, HONG Xiaopeng, ZHAO Guoying. Selective deep features for micro-expression recognition[C]//Proceedings of the 23rd International Conference on Pattern Recognition. Cancun, Mexico, 2016: 2258-2263.
[14] PENG Min, WANG Chongyang, CHEN Tong, et al. Dual temporal scale convolutional neural network for micro-expression recognition[J]. Frontiers in psychology, 2017, 8: 1745.
[15] VERMA M, VIPPARTHI S K, SINGH G, et al. LEARNet: dynamic imaging network for micro expression recognition[J]. IEEE transactions on image processing, 2019, 29: 1618-1627.
[16] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany, 2018: 3-19.
[17] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016: 770-778.
[18] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017: 2980-2988.
[19] LIONG S T, SEE J, WONG K S, et al. Less is more: micro-expression recognition from video using apex frame[J]. Signal processing: image communication, 2018, 62: 82-92.
[20] GAN Y S, LIONG S T, YAU W C, et al. OFF-ApexNet on micro-expression recognition system[J]. Signal processing: image communication, 2019, 74: 129-139.
[21] YAN Wenjing, LI Xiaobai, WANG Sujing, et al. CASME II: an improved spontaneous micro-expression database and the baseline evaluation[J]. PLoS One, 2014, 9(1): e86041.
[22] LUCEY P, COHN J F, KANADE T, et al. The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. San Francisco, CA, USA, 2010: 94-101.
[23] LIONG S T, SEE J, PHAN R C W, et al. Subtle expression recognition using optical strain weighted features[C]//Proceedings of the Asian Conference on Computer Vision. Singapore, Singapore, 2014: 644-657.
[24] 孔慧芳, 钱世超, 闫嘉鹏. 基于不均衡数据与迁移学习的面部微表情识别[J]. 合肥工业大学学报(自然科学版), 2020, 43(7): 895-900
KONG Huifang, QIAN Shichao, YAN Jiapeng. Image based facial micro-expression recognition using transfer learning on imbalanced data[J]. Journal of Hefei University of Technology (natural science edition), 2020, 43(7): 895-900

相似文献/References:

[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(02):193.[doi:10.3969/j.issn.1673-4785.201405060]
 ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10(6):193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(01):1.[doi:10.3969/j.issn.1673-4785.201403072]
 DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10(6):1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
 MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11(6):279.[doi:10.11992/tis.201603026]
[4]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
 LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11(6):567.[doi:10.11992/tis.201511028]
[5]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
 MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11(6):728.[doi:10.11992/tis.201611021]
[6]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
 WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11(6):788.[doi:10.11992/tis.201609006]
[7]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
 HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11(6):835.[doi:10.11992/tis.201605022]
[8]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(06):770.[doi:10.11992/tis.201706084]
 SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12(6):770.[doi:10.11992/tis.201706084]
[9]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(06):894.[doi:10.11992/tis.201706037]
 YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12(6):894.[doi:10.11992/tis.201706037]
[10]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(01):55.[doi:10.11992/tis.201609029]
 WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13(6):55.[doi:10.11992/tis.201609029]

备注/Memo

备注/Memo:
收稿日期:2020-10-26。
基金项目:国家自然科学基金项目(61976118);江苏省“六大人才高峰”高层次人才项目(RJFW-038)
作者简介:徐玮,硕士研究生,主要研究方向为机器学习、微表情识别;郑豪,教授,南京晓庄学院信息工程学院副院长,江苏省“333”工程中青年学术技术带头人,江苏省“六大人才高峰”高层次人才,主要研究方向为人工智能、模式识别。主持及参与国家、省级科学基金项目10余项。发表学术论文30余篇;杨种学,教授,南京晓庄学院副校长,江苏省青蓝工程优秀青年骨干,主要研究方向为人工智能、机器学习。发表学术论文10余篇
通讯作者:郑豪.E-mail:zhh710@163.com
更新日期/Last Update: 2021-12-25