FANG Tao,CHEN Zhiguo,FU Yi.Face recognition method based on neural network multi-layer feature information fusion[J].CAAI Transactions on Intelligent Systems,2021,16(2):279-285.[doi:10.11992/tis.201907053]





Face recognition method based on neural network multi-layer feature information fusion
方涛1 陈志国1 傅毅12
1. 江南大学 物联网工程学院,江苏 无锡 214122;
2. 无锡环境科学与工程研究中心,江苏 无锡 214153
FANG Tao1 CHEN Zhiguo1 FU Yi12
1. School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China;
2. Wuxi Research Center of Environmental Science and Engineering, Wuxi 214153, China
face recognitionfacial featureneural networkinformation fusionfeature fusiondecision fusionfeature extractionsimilarity fusion
由于人脸面部结构复杂,不同人脸之间结构特征相似,导致难以提取到十分适合用于分类的人脸特征,虽然神经网络具有良好效果,并且有很多改进的损失函数能够帮助提取需要的特征,但是单一的深度特征没有充分利用多层特征之间的互补性,针对这些问题提出了一种基于神经网络多层特征信息融合的人脸识别方法。首先选择ResNet网络结构进行改进,提取神经网络中的多层特征,然后将多层特征映射到子空间,在各自子空间内通过定义的中心变量进行自适应加权融合;为进一步提升效果,将所有特征送入Softmax分类器,同时对分类结果通过相同方式进行自适应加权决策融合;训练网络学习适合的中心变量,应用中心变量计算加权融合相似度。在同样的有限条件下,在使用AM-Softmax损失函数的基础上,融合特征在LFW(Labeled Faces in the Wild)上的识别效果了提升1.6%,使用融合相似度提升了2.2%。能够有效地提升人脸识别率,提取更合适的人脸特征。
Because the structure of the face is complex and the structural features of different faces are similar, it is difficult to extract facial features that are suitable for classification. Neural networks generate good results, and the recent improvements in many loss functions can help extract the required features. However, a single depth feature does not make full use of the complementarity of multi-layer features. To solve these problems, we propose a face recognition method based on the fusion of neural-network multi-layer feature information. First, we select the ResNet network structure to improve the outcome, then we extract the multi-layer features in the neural network. These features are then mapped onto the sub-spaces. Next, adaptive weighted fusion is performed of the defined central variables in the respective sub-spaces. To realize further improvement, all the features are sent to the Softmax classifier, and the classification results are fused in the same way by adaptive weighted decision-making. The training network learns the appropriate central variable, which is applied to calculate the weighted fusion similarity. Under the same conditions, based on the AM-Softmax loss function, the recognition of the fusion feature on the Labeled Faces in the Wild database increased by 1.6%, and the fusion similarity increased by 2.2%. We conclude that the proposed method effectively improves the face recognition rate and extracts more suitable facial features.


[1] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6):1229-1251
ZHOU Feiyan, JIN Linpeng, DONG Jun. Review of convolutional neural network[J]. Chinese journal of computers, 2017, 40(6):1229-1251
[2] WU X, HE R, SUN Z, et al. A light CNN for deep face representation with noisy labels[J]. IEEE transactions on information forensics and security, 2018, 13(11):2884-2896.
[3] CHEN Juncheng, PATEL V M, CHELLAPPA R. Unconstrained face verification using deep CNN features[C]//Proceedings of 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). Lake Placid, USA, 2016:1-9.
[4] SUN Y, LIANG D, WANG X, et al. Deepid3:Face recognition with very deep neural networks[J/OL].(2015-02-03).[2019-01-01]. https://www.docin.com/p-1571349020.html.
[5] SCHROFF F, KALENICHENKO D, PHILBIN J, et al. FaceNet:a unified embedding for face recognition and clustering[J]. Computer vision and pattern recognition, 2015:815-823.
[6] 景晨凯, 宋涛, 庄雷, 等. 基于深度卷积神经网络的人脸识别技术综述[J]. 计算机应用与软件, 2018, 35(1):223-231
JING Chenkai, SONG Tao, ZHUANG Lei, et al. A survey of face recognition technology based on deep convolutional neural networks[J]. Computer applications and software, 2018, 35(1):223-231
[7] 孙劲光, 孟凡宇. 基于深度神经网络的特征加权融合人脸识别方法[J]. 计算机应用, 2016, 36(2):437-443
SUN Jinguang, MENG Fanyu. Face recognition based on deep neural network and weighted fusion of face features[J]. Journal of computer applications, 2016, 36(2):437-443
[8] TAIGMAN Y, YANG Ming, RANZATO MARC’AURELIO, et al. DeepFace:closing the gap to human-level performance in face verification[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA, 2014:1701-1708.
[9] SUN Yi, WANG Xiaogang, TANG Xiaoou. Deep learning face representation from predicting 10, 000 classes[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA, 2014:1891-1898.
[10] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//Proceedings of the 13th European Conference on Computer Vision. Cham, Germany, 2014:818-833.
[11] HOFFER E, AILON N. Deep metric learning using triplet network[C]//Proceedings of the 3rd International Workshop no Similarity-Based Pattern Recognition. Copenhagen, Denmark, 2015:84-92.
[12] TADMOR O, ROSENWEIN T, SHALEV-SHWARTZ S, et al. Learning a metric embedding for face recognition using the multibatch method[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook, United States, 2016:1396-1397.
[13] 吕璐, 蔡晓东, 曾燕, 等. 一种基于融合深度卷积神经网络与度量学习的人脸识别方法[J]. 现代电子技术, 2018, 41(9):58-61, 67
LYU Lu, CAI Xiaodong, ZENG Yan, et al. A face recognition method based on fusion of deep CNN and metric learning[J]. Modern electronics technique, 2018, 41(9):58-61, 67
[14] WEN Yandong, ZHANG Kaipeng, LI Zhifeng, et al. A discriminative feature learning approach for deep face recognition[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands, 2016:499-515.
[15] LIU Weiyang, WEN Yandong, YU Zandong, et al. SphereFace:deep hypersphere embedding for face recognition[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017:212-220.
[16] WANG Hao, WANG Yitong, ZHOU Zheng, et al. CosFace:Large margin cosine loss for deep face recognition[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018:5265-5274.
[17] WANG Feng, CHENG Jian, LIU Weiyang, et al. Additive margin softmax for face verification[J]. IEEE signal processing letters, 2018, 25(7):926-930.
[18] 任克强, 胡慧. 角度空间三元组损失微调的人脸识别[J]. 液晶与显示, 2019, 34(1):110-117
REN Keqiang, HU Hui. Face recognition of triple loss fine-tuning in angular space[J]. Chinese journal of liquid crystals and displays, 2019, 34(1):110-117
[19] GUNTHER M, CRUZ S, RUDD E M, et al. Toward open-set face recognition[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA, 2017:71-80.
[20] HUANG G B, MATTAR M, BERG T, et al. Labeled faces in the wild:a database forstudying face recognition in unconstrained environments[C]//Workshop on Faces in ‘Real-Life’ Images:Detection, Alignment, and Recognition. Marseille, Francem, 2008.


 JIN Yi,RUAN Qiu-qi.A partially weighted twodimensional PCA for face recognition[J].CAAI Transactions on Intelligent Systems,2007,2(2):25.
[2]任小龙,苏光大,相 燕.使用第2代身份证的人脸识别身份认证系统[J].智能系统学报,2009,4(03):213.
 REN Xiao-long,SU Guang-da,XIANG Yan.Face authentication system using the Chinese second generation identity card[J].CAAI Transactions on Intelligent Systems,2009,4(2):213.
 SU Guang-da.Face recognition system designed to integrate multiple techniques[J].CAAI Transactions on Intelligent Systems,2009,4(2):471.[doi:doi:10.3969/j.issn.1673-4785.2009.06.001]
 WANG Kejun,ZOU Guofeng,ZHANG Jie.Analysis of the influence of SPCA parameters on the recognition of a single sample face[J].CAAI Transactions on Intelligent Systems,2011,6(2):531.
 BEN Xianye,WANG Kejun,MA Hui.Videobased automatic frontview human identification[J].CAAI Transactions on Intelligent Systems,2012,7(2):69.
 DU Jixiang,ZHAI Chuanmin,YE Yongqing.An agespan face recognition method based on an NMF algorithm with sparseness constraints[J].CAAI Transactions on Intelligent Systems,2012,7(2):271.
 RUAN Xiaohu,LI Weijun,QIN Hong,et al.An assessment method for face alignment based on feature matching[J].CAAI Transactions on Intelligent Systems,2015,10(2):12.[doi:10.3969/j.issn.1673-4785.201312064]
 SUN Jinguang,MENG Fanyu.Face recognition by weighted fusion of facial features[J].CAAI Transactions on Intelligent Systems,2015,10(2):912.[doi:10.11992/tis.201509025]
 MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11(2):279.[doi:10.11992/tis.201603026]
 LIU Xunli,GONG Xun,WANG Guoyin.Face recognition based on the linear representation model without residual estimation[J].CAAI Transactions on Intelligent Systems,2014,9(2):285.[doi:10.3969/j.issn.1673-4785.201309065]
 YANG Huixian,LIU Jian,ZHANG Mengjuan,et al.Face recognition with double difference local directional pattern[J].CAAI Transactions on Intelligent Systems,2018,13(2):751.[doi:10.11992/tis.201706032]


更新日期/Last Update: 2021-04-25