[1]莫宏伟,傅智杰.基于迁移学习的无监督跨域人脸表情识别[J].智能系统学报,2021,16(3):397-406.[doi:10.11992/tis.202008034]
MO Hongwei,FU Zhijie.Unsupervised cross-domain expression recognition based on transfer learning[J].CAAI Transactions on Intelligent Systems,2021,16(3):397-406.[doi:10.11992/tis.202008034]
点击复制
MO Hongwei,FU Zhijie.Unsupervised cross-domain expression recognition based on transfer learning[J].CAAI Transactions on Intelligent Systems,2021,16(3):397-406.[doi:10.11992/tis.202008034]
基于迁移学习的无监督跨域人脸表情识别
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第3期
页码:
397-406
栏目:
学术论文—机器学习
出版日期:
2021-05-05
- Title:
- Unsupervised cross-domain expression recognition based on transfer learning
- Keywords:
- expression recognition; unsupervised; cross-domain; transfer learning; feature transformation; joint alignment; public subspace; domain adaptive
- 分类号:
- TP181
- 摘要:
- 本文主要研究了基于迁移学习的无监督跨域人脸表情识别。在过去的几年里，提出的许多方法在人脸表情识别方面取得了令人满意的识别效果。但这些方法通常认为训练和测试数据来自同一个数据集，因此其具有相同的分布。而在实际应用中，这一假设通常并不成立，特别当训练集和测试集来自不同的数据集时，即跨域人脸表情识别问题。为了解决这一问题，本文提出将一种基于联合分布对齐的迁移学习方法(domain align learning)应用于跨域人脸表情识别，该方法通过找到一个特征变换，将源域和目标域数据映射到一个公共子空间中，在该子空间中联合对齐边缘分布和条件分布来减小域之间的分布差异，然后对变换后的特征进行训练得到一个域适应分类器来预测目标域样本标签。为了验证提出算法的有效性，在CK+、Oulu-CASIA NIR和Oulu-CASIA VIS这3个不同的数据库上做了大量实验，实验结果证明所提算法在跨域表情识别上是有效性的。
- Abstract:
- This paper primarily studies unsupervised cross-domain facial expression recognition based on transfer learning. In recent years, many proposed methods have achieved satisfactory results in facial expression recognition. However, these methods usually assume that the training and test data come from the same data set and therefore have the same distribution. In practical applications, this assumption is usually untrue, especially when the training and test sets come from different data sets, also known as the cross-domain facial expression recognition problem. To solve this problem, we propose a migration learning method (domain align learning) based on joint distributed alignment for cross-domain facial expression recognition. By determining a feature transform, the source and target domain data are mapped onto a common subspace, wherein edge distribution and conditional distribution are aligned jointly to reduce the distribution difference between domains, and then a domain adaptive classifier is obtained by training the transformed features to predict the target domain sample label. To verify the effectiveness of the proposed algorithm, many experiments are performed on three databases, CK+, Oulu-CASIA NIR, and Oulu-CASIA VIS. The experimental results show the effectiveness of the proposed algorithm in cross-domain facial expression recognition.
- 参考文献/References:
-
[1] ZHENG Wenming, TANG Hao, HUANG T S. Emotion recognition from non-frontal facial images[M]//KONAR A, CHAKRABORTY A. Emotion Recognition:A Pattern Analysis Approach. Hoboken, USA:John Wiley & Sons, 2015:183-213.
[2] WANG Yongjin, GUAN Ling, VENETSANOPOULOS A N. Kernel cross-modal factor analysis for information fusion with application to bimodal emotion recognition[J]. IEEE transactions on multimedia, 2012, 14(3):597-607.
[3] YAN Jingjie, ZHENG Wenming, XU Guanming, et al. Sparse kernel reduced-rank regression for bimodal emotion recognition from facial expression and speech[J]. IEEE transactions on multimedia, 2016, 18(7):1319-1329.
[4] ZENG Zhihong, PANTIC M, ROISMAN G I, et al. A survey of affect recognition methods:audio, visual, and spontaneous expressions[J]. IEEE transactions on pattern analysis and machine intelligence, 2009, 31(1):39-58.
[5] CHU W S, DE LA TORRE F, COHN J F. Selective transfer machine for personalized facial expression analysis[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(3):529-545.
[6] ZHENG Wenming, ZHOU Xiaoyan, ZOU Cairong, et al. Facial expression recognition using kernel canonical correlation analysis (KCCA)[J]. IEEE transactions on neural networks, 2006, 17(1):233-238.
[7] UDDIN M Z. A local feature-based facial expression recognition system from depth video[M]//DELIGIANNIDIS L, ARABNIA H R. Emerging Trends in Image Processing, Computer Vision and Pattern Recognition. Amsterdam:Elsevier, 2015:407-419.
[8] YU Zhiding, ZHANG Cha. Image based static facial expression recognition with multiple deep network learning[C]//Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. New York, USA:ACM, 2015:433?442.
[9] BACCOUCHE M, MAMALET F, WOLF C, et al. Spatio-temporal convolutional sparse auto-encoder for sequence classification[C]//Proceedings of the of British Machine Vision Conference. Guildford, UK:BMVA Press, 2012:12.
[10] WANG Hua, NIE Feiping, HUANG Heng, et al. Dyadic transfer learning for cross-domain image classification[C]//IEEE International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011:551?556.
[11] LUO Jie, TOMMASI T, CAPUTO B. Multiclass transfer learning from unconstrained priors[C]//IEEE International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011:1863?1870.
[12] ROY S D, MEI Tao, ZENG Wenjun, et al. SocialTransfer:cross-domain transfer learning from social streams for media applications[C]//Proceedings of the 20th ACM international conference on Multimedia. Nara, Japan:ACM, 2012:649?658.
[13] WANG Shuhui, JIANG Shuqiang, HUANG Qingming, et al. Multi-feature metric learning with knowledge transfer among semantics and social tagging[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence:IEEE, 2012:2240?2247.
[14] AYTAR Y, ZISSERMAN A. Tabula rasa:model transfer for object category detection[C]//Proceedings of 2011 International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011:2252?2259.
[15] GOPALAN R, LI Ruonan, CHELLAPPA R. Domain adaptation for object recognition:an unsupervised approach[C]//2011 International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011:999?1006.
[16] GUILLAUMIN M, FERRARI V. Large-scale knowledge transfer for object localization in ImageNet[C]//Proceedings of 2012 IEEE Computer Vision and Pattern Recognition. Providence, USA:IEEE, 2012:3202?3209.
[17] LAMPERT C H, NICKISCH H, HARMELING S. Learning to detect unseen object classes by between-class attribute transfer[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA:IEEE, 2009:951?958.
[18] JHUO I H, LIU D, LEE D T, et al. Robust visual domain adaptation with low-rank reconstruction[C]//2012 IEEE Computer Vision and Pattern Recognition. Providence, USA:IEEE, 2012:2168?2175.
[19] LAMPERT C H, KR?MER O. Weakly-paired maximum covariance analysis for multimodal dimensionality reduction and transfer learning[C]//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece:Springer-Verlag, 2010:566?579.
[20] QIU Qiang, PATEL V M, TURAGA P, et al. Domain adaptive dictionary learning[C]//Proceedings of the 12th European Conference on Computer Vision. Florence, Italy:Springer-Verlag, 2012:631?645.
[21] DAI Wenyuan, YANG Qiang, XUE Guirong, et al. Boosting for transfer learning[C]//Proceedings of the 24th International Conference on Machine Learning. Corvalis, USA:ACM, 2007:193?200.
[22] PAN S J, TSANG I W, KWOK J T, et al. Domain adaptation via transfer component analysis[J]. IEEE transactions on neural networks, 2011, 22(2):199-210.
[23] GONG Boqing, SHI Yuan, SHA Fei, et al. Geodesic flow kernel for unsupervised domain adaptation[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA:IEEE, 2012:2066?2073.
[24] ZHONG Erheng, FAN Wei, PENG Jing, et al. Cross domain distribution adaptation via kernel mapping[C]//Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Paris, France:ACM, 2009.
[25] BRUZZONE L, MARCONCINI M. Domain adaptation problems:A DASVM classification technique and a circular validation strategy[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 32(5):770-787.
[26] QUANZ B, HUAN J, MISHRA M. Knowledge transfer with low-quality data:A feature extraction issue[J]. IEEE transactions on knowledge and data engineering, 2012, 24(10):1789-1802.
[27] LUCEY P, COHN J F, KANADE T, et al. The extended Cohn-Kanade dataset (CK+):a complete dataset for action unit and emotion-specified expression[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. San Francisco, USA:IEEE, 2010:94-101.
[28] KANADE T, COHN J F, TIAN Yingli. Comprehensive database for facial expression analysis[C]//Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition. Grenoble, France:IEEE, 2020:46-53.
[29] ZHAO Guoying, HUANG Xiaohua, TAINI M, et al. Facial expression recognition from near-infrared videos[J]. Image and vision computing, 2011, 29(9):607-619.
[30] HUANG J, GRETTON A, BORGWARDT K, et al. Correcting sample selection bias by unlabeled data[J]. Advances in neural information processing systems, 2006, 19:601-608.
[31] SUGIYAMA M, NAKAJIMA S, KASHIMA H, et al. Direct importance estimation with model selection and its application to covariate shift adaptation[C]//Proceedings of 21st Annual Conference on Advances in Neural Information Processing Systems. Vancouver, Canada:NIPS, 2008:1433-1440.
[32] 刘斌, 米强, 徐岩. LBP和MB-LBP加权融合的人脸识别[J]. 计算机工程与设计, 2018, 39(2):551-556
LIU Bin, MI Qiang, XU Yan. Face recognition based on weighted fusion of LBP and MB-LBP features[J]. Computer engineering and design, 2018, 39(2):551-556
- 相似文献/References:
-
[1]王立国,赵亮,石瑶.基于最大最小距离的高光谱遥感图像波段选择[J].智能系统学报,2018,13(1):131.[doi:10.11992/tis.201703023]
WANG Liguo,ZHAO Liang,SHI Yao.Maximin distance algorithm-based band selection for hyperspectral imagery[J].CAAI Transactions on Intelligent Systems,2018,13():131.[doi:10.11992/tis.201703023]
[2]刘帅师,郭文燕,张言,等.鲁棒的正则化编码随机遮挡表情识别[J].智能系统学报,2018,13(2):261.[doi:10.11992/tis.201609002]
LIU Shuaishi,GUO Wenyan,ZHANG Yan,et al.Recognition of facial expression in case of random shielding based on robust regularized coding[J].CAAI Transactions on Intelligent Systems,2018,13():261.[doi:10.11992/tis.201609002]
[3]张智,毕晓君.基于风格转换的无监督聚类行人重识别[J].智能系统学报,2021,16(1):48.[doi:10.11992/tis.202012014]
ZHANG Zhi,BI Xiaojun.Clustering approach based on style transfer for unsupervised person re-identification[J].CAAI Transactions on Intelligent Systems,2021,16():48.[doi:10.11992/tis.202012014]
[4]伍锡如,凌星雨.基于改进的Faster RCNN面部表情检测算法[J].智能系统学报,2021,16(2):210.[doi:10.11992/tis.201910020]
WU Xiru,LING Xingyu.Facial expression recognition based on improved Faster RCNN[J].CAAI Transactions on Intelligent Systems,2021,16():210.[doi:10.11992/tis.201910020]
[5]陈彤,陈秀宏.特征自表达和图正则化的鲁棒无监督特征选择[J].智能系统学报,2022,17(2):286.[doi:10.11992/tis.202012043]
CHEN Tong,CHEN Xiuhong.Feature self-representation and graph regularization for robust unsupervised feature selection[J].CAAI Transactions on Intelligent Systems,2022,17():286.[doi:10.11992/tis.202012043]
[6]杨梦茵,陈俊芬,翟俊海.非对称卷积编码器的聚类算法[J].智能系统学报,2022,17(5):900.[doi:10.11992/tis.202107021]
YANG Mengyin,CHEN Junfen,ZHAI Junhai.A clustering method based on the asymmetric convolutional autoencoder[J].CAAI Transactions on Intelligent Systems,2022,17():900.[doi:10.11992/tis.202107021]
[7]闫河,李梦雪,张宇宁,等.面向表情识别的重影非对称残差注意力网络模型[J].智能系统学报,2023,18(2):333.[doi:10.11992/tis.202201003]
YAN He,LI Mengxue,ZHANG Yuning,et al.A ghost asymmetric residual attention network model for facial expression recognition[J].CAAI Transactions on Intelligent Systems,2023,18():333.[doi:10.11992/tis.202201003]
[8]过伶俐,陈秀宏.潜在多步马尔可夫概率的鲁棒无监督特征选择[J].智能系统学报,2023,18(5):1017.[doi:10.11992/tis.202208013]
GUO Lingli,CHEN Xiuhong.Robust unsupervised feature selection via multistep Markov probability and latent representation[J].CAAI Transactions on Intelligent Systems,2023,18():1017.[doi:10.11992/tis.202208013]
备注/Memo
收稿日期:2020-08-28。
基金项目:国家自然科学基金项目(60035117)
作者简介:莫宏伟，教授，博士生导师，博士，主要研究方向为类脑计算与人工智能、机器视觉与机器认知、人机混合智能。主持国家自然科学基金等项目20余项。出版专著6部，发表学术论文80余篇;傅智杰，硕士研究生，主要研究方向为深度学习、计算机视觉、医学影像
通讯作者:莫宏伟.E-mail:honwei2004@126.com
更新日期/Last Update:
2021-06-25