[1]梁艳,温兴,潘家辉.融合全局与局部特征的跨数据集表情识别方法[J].智能系统学报,2023,18(6):1205-1212.[doi:10.11992/tis.202212030]
LIANG Yan,WEN Xing,PAN Jiahui.Cross-dataset facial expression recognition method fusing global and local features[J].CAAI Transactions on Intelligent Systems,2023,18(6):1205-1212.[doi:10.11992/tis.202212030]
点击复制
LIANG Yan,WEN Xing,PAN Jiahui.Cross-dataset facial expression recognition method fusing global and local features[J].CAAI Transactions on Intelligent Systems,2023,18(6):1205-1212.[doi:10.11992/tis.202212030]
融合全局与局部特征的跨数据集表情识别方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第6期
页码:
1205-1212
栏目:
学术论文—机器感知与模式识别
出版日期:
2023-11-05
- Title:
- Cross-dataset facial expression recognition method fusing global and local features
- Keywords:
- cross-dataset; facial expression recognition; domain adaptation; feature fusion; self-attention mechanism; transfer learning; fine-grained domain discriminator; residual network
- 分类号:
- TP391
- 摘要:
- 人脸表情数据集在收集过程中存在主观的标注差异和客观的条件差异，导致表情识别模型在不同数据集间呈现明显的性能差异。为了提高跨数据集表情识别精度、减少表情识别在实际应用中进行样本打标重训练的过程，本文提出了一种基于表情融合特征的域对抗网络模型，用于跨数据集人脸表情识别。采用残差神经网络提取人脸表情的全局特征与局部特征。利用Encoder模块对全局特征与局部特征进行融合，学习更深层次的表情信息。使用细粒度的域鉴别器进行源数据集与目标数据集对抗，对齐数据集的边缘分布和条件分布，使模型能迁移到无标签的目标数据集中。以RAF-DB为源数据集，以CK+、JAFFE、SFEW2.0、FER2013、Expw分别作为目标数据集进行跨数据集人脸表情识别实验。与其他跨数据集人脸表情识别算法相比，所提方法获得了最高的平均识别率。实验结果表明，所提方法能有效提高跨数据集人脸表情识别的性能。
- Abstract:
- The expression recognition model shows significant performance differences between datasets due to subjective annotation and objective condition differences in the collection of facial expression datasets. A domain adversarial network model based on expression fusion features is proposed for cross-dataset facial expression recognition. This model aims to improve the accuracy of cross-dataset expression recognition and reduce the sample marking and retraining processes for expression recognition in practical applications. Residual neural networks are used to extract the global and local features of facial expressions. An encoder module is then employed to fuse global and local features to learn deep expression information. A fine-grained domain discriminator is adopted to antagonize the source dataset against the target dataset, aligning the edge and conditional distributions of the dataset and facilitating the migration of the model to the unlabeled target dataset. RAF-DB is used as the source dataset, and CK+, JAFFE, SFEW2.0, FER2013, and Expw are used as the target datasets for cross-dataset facial expression recognition experiments. Compared with other cross-dataset facial expression recognition algorithms, the proposed method achieves the highest average recognition rate. Experimental results show that the proposed method can effectively improve the performance of cross-dataset facial expression recognition.
- 参考文献/References:
-
[1] MEHRABIAN A. Communication without words[M]//Communication theory. [S.l.]: Routledge, 2017: 193-200.
[2] LI Shan, DENG Weihong. Deep facial expression recognition: a survey[J]. IEEE transactions on affective computing, 2022, 13(3): 1195–1215.
[3] XU Ruijia, LI Guanbin, YANG Jihan, et al. Larger norm more transferable: an adaptive feature norm approach for unsupervised domain adaptation[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2020: 1426-1435.
[4] LEE Chenyu, BATRA T, BAIG M H, et al. Sliced Wasserstein discrepancy for unsupervised domain adaptation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 10277-10287.
[5] 莫宏伟, 傅智杰. 基于迁移学习的无监督跨域人脸表情识别[J]. 智能系统学报, 2021, 16(3): 397–406
MO Hongwei, FU Zhijie. Unsupervised cross-domain expression recognition based on transfer learning[J]. CAAI transactions on intelligent systems, 2021, 16(3): 397–406
[6] LONG Mingsheng, CAO Yue, WANG Jianmin, et al. Learning transferable features with deep adaptation networks[C]//32nd International Conference on Machine Learning. Lille: ICML, 2015, 1: 97-105.
[7] LI Shan, DENG Weihong. A deeper look at facial expression dataset bias[J]. IEEE transactions on affective computing, 2022, 13(2): 881–893.
[8] XU Xiaolin, ZHENG Wenming, ZONG Yuan, et al. Sample self-revised network for cross-dataset facial expression recognition[C]//2022 International Joint Conference on Neural Networks. Padua: IEEE, 2022: 1-8.
[9] XU Xiaolin, ZONG Yuan, LU Cheng, et al. Enhanced sample self-revised network for cross-dataset facial expression recognition[J]. Entropy, 2022, 24(10): 1475.
[10] CHEN Tianshui, PU Tao, WU Hefeng, et al. Cross-domain facial expression recognition: a unified evaluation benchmark and adversarial graph learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 44(12): 9887–9903.
[11] GANIN Y, LEMPITSKY V. Unsupervised domain adaptation by backpropagation[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning. New York: ACM, 2015: 1180-1189.
[12] LONG Mingsheng, CAO Zhangjie, WANG Jianmin, et al. Conditional adversarial domain adaptation[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal: ACM, 2018: 1647-1657.
[13] WANG Chao, DING Jundi, YAN Hui, et al. A prototype-oriented contrastive adaption network for cross-domain facial expression recognition[C]//Asian Conference on Computer Vision. Cham: Springer, 2023: 324-340.
[14] XIE Yuan, CHEN Tianshui, PU Tao, et al. Adversarial graph representation adaptation for cross-domain facial expression recognition[C]//Proceedings of the 28th ACM International Conference on Multimedia. Seattle: ACM, 2020: 1255-1264.
[15] TIAN Yingli, KANADE T, COHN J F. Recognizing action units for facial expression analysis[J]. IEEE transactions on pattern analysis and machine intelligence, 2001, 23(2): 97–115.
[16] ZHANG Kaipeng, ZHANG Zhanpeng, LI Zhifeng, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE signal processing letters, 2016, 23(10): 1499–1503.
[17] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[18] ARNAB A, DEHGHANI M, HEIGOLD G, et al. ViViT: a video vision transformer[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 6816-6826.
[19] KHAN S, NASEER M, HAYAT M, et al. Transformers in vision: a survey[J]. ACM computing surveys, 2022, 54(10s): 1–41.
[20] LUCEY P, COHN J F, KANADE T, et al. The extended cohn-kanade dataset (CK): a complete dataset for action unit and emotion-specified expression[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE, 2010: 94-101.
[21] LYONS M, AKAMATSU S, KAMACHI M, et al. Coding facial expressions with Gabor wavelets[C]//Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition. Nara: IEEE, 2002: 200-205.
[22] DHALL A, GOECKE R, LUCEY S, et al. Static facial expression analysis in tough conditions: data, evaluation protocol and benchmark[C]//2011 IEEE International Conference on Computer Vision Workshops. Barcelona: IEEE, 2012: 2106-2112.
[23] GOODFELLOW I J, ERHAN D, LUC CARRIER P, et al. Challenges in representation learning: a report on three machine learning contests[J]. Neural networks, 2015, 64: 59–63.
[24] ZHANG Zhanpeng, LUO Ping, LOY C C, et al. Learning social relation traits from face images[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2016: 3631-3639.
[25] LI Shan, DENG Weihong, DU Junping. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2584-2593.
[26] LI Shan, DENG Weihong. Deep emotion transfer network for cross-database facial expression recognition[C]//2018 24th International Conference on Pattern Recognition. Beijing: IEEE, 2018: 3092-3099.
[27] LAURENS V D M, HINTON G. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(2605): 2579–2605.
- 相似文献/References:
-
[1]孙正兴,徐文晖.基于局部SVM分类器的表情识别方法[J].智能系统学报,2008,3(5):455.
SUN Zheng-xing,XU Wen-hui.Fac ial expression recogn ition based on local SVM classif iers[J].CAAI Transactions on Intelligent Systems,2008,3():455.
[2]彭程,刘帅师,万川,等.基于局部纹理ASM模型的人脸表情识别[J].智能系统学报,2011,6(3):231.
PENG Cheng,LIU Shuaishi,WAN Chuan,et al.An active shape model for facial expression recognition based on a local texture model[J].CAAI Transactions on Intelligent Systems,2011,6():231.
[3]童莹.一种方向性的局部二值模式在人脸表情识别中的应用[J].智能系统学报,2015,10(3):422.[doi:10.3969/j.issn.1673-4785.201405016]
TONG Ying.Local binary pattern based on the directions and its application in facial expression recognition[J].CAAI Transactions on Intelligent Systems,2015,10():422.[doi:10.3969/j.issn.1673-4785.201405016]
[4]高涛,杨朝晨,陈婷,等.深度多尺度融合注意力残差人脸表情识别网络[J].智能系统学报,2022,17(2):393.[doi:10.11992/tis.202107028]
GAO Tao,YANG Zhaochen,CHEN Ting,et al.Deep multiscale fusion attention residual network for facial expression recognition[J].CAAI Transactions on Intelligent Systems,2022,17():393.[doi:10.11992/tis.202107028]
备注/Memo
收稿日期:2022-12-29。
基金项目:国家科技创新2030重点项目（2022ZD0208900）；国家自然科学基金项目（62076103）.
作者简介:梁艳,讲师,博士,主要研究方向为计算机视觉、模式识别与智能系统等。发表学术论文20余篇;温兴,硕士研究生,主要研究方向为深度学习、计算机视觉、迁移学习;潘家辉,教授,博士,中国人工智能学会脑机融合与生物机器智能专业委员会委员,主要研究方向为模式识别与智能系统、脑机交互。主持3项国家自然科学基金项目,2项广东省自然科学基金项目,发表学术论文60余篇
通讯作者:梁艳.E-mail:liangyan@m.scnu.edu.cn
更新日期/Last Update:
1900-01-01