[1]JIANG Yi,LYU Rongzhen,LIU Mingzhu,et al.Masked face image synthesis based on a generative adversarial network[J].CAAI Transactions on Intelligent Systems,2021,16(6):1073-1080.[doi:10.11992/tis.202012010]
Copy

Masked face image synthesis based on a generative adversarial network

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 6 Page number: 1073-1080 Column: 学术论文—智能系统 Public date: 2021-11-05
Title:
Masked face image synthesis based on a generative adversarial network
Author(s):
JIANG Yi;  LYU Rongzhen;  LIU Mingzhu;  HAN Chuang
School of Measurement-Control Technology and Communications Engineering, Harbin University of Science and Technology, Harbin 150080, China
Keywords:
deep learning; generative adversarial networks; spatial transformation; convolution neural network; image fusion; face mask; human face dataset; face recognition
CLC:
TP391
DOI:
10.11992/tis.202012010
Abstract:
This paper proposes a method for generating masked face images using a generative adversarial network (GAN) and spatial transformer networks. The proposed method is used to solve the present problem of lacking face datasets of people wearing masks. Based on the GAN, the proposed method introduces a multiscale convolution kernel to extract image characteristics in various dimensions. This method introduces the Wasserstein divergence to measure the distance between an authentic specimen and a synthetic specimen so that generator’s performance can be optimized. Experiments show that the proposed method can add a mask to a face image effectively without any annotations on the original image, and the synthesized image has high fidelity.
References:
[1] 国家卫生健康委员会.新型冠状病毒肺炎诊疗方案(试行第八版)[EB/OL].(2020-08-18)[2020-12-08]http://www.gov.cn/zhengce/zhengceku/2020-08/19/content_5535757.htm.
National Health Commission of the People’s Republic of China. Diagnosis and treatment protocol for novel coronavirus pneumonia (trial version 8) [EB/OL].(2020-08-18)[2020-12-01]http://www.gov.cn/zhengce/zhengceku/2020-08/19/content_5535757.htm.
[2] TAIGMAN Y, YANG Ming, RANZATO M A, et al. DeepFace: closing the gap to human-level performance in face verification[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA, 2014: 1701-1708.
[3] SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015: 815-823.
[4] 李小薪, 梁荣华. 有遮挡人脸识别综述: 从子空间回归到深度学习[J]. 计算机学报, 2018, 41(1): 177-207
LI Xiaoxin, LIANG Ronghua. A review for face recognition with occlusion: from subspace regression to deep learning[J]. Chinese journal of computers, 2018, 41(1): 177-207
[5] ANWAR A, RAYCHOWDHURY A. Masked face recognition for secure authentication[EB/OL].(2020-08-25)[2020-12-01] https://arxiv.org/abs/2008.11104.
[6] CABANI A, HAMMOUNDI K, BENHABILES H, et al. Masked-Face-Net—a dataset of correctly/incorrectly masked face images in the context of covid-19[EB/OL].(2020-08-18)[2020-12-01] https://arxiv.org/abs/2008.08016.
[7] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada, 2014: 2672-2680.
[8] 胡铭菲,刘建伟,左信.深度生成模型综述[EB/OL].（2021-10-28）[2021-10-30] https://doi.org/10.16383/j.aas.c190866.
HU Mingfei, LIU Jianwei, ZUO Xin. Survey on deep generative model[EB/OL].（2021-10-28）[2021-10-30] https://doi.org/10.16383/j.aas.c190866.
[9] MIRZA M, OSINDERO S. Conditional generative adversarial nets[EB/OL].(2014-11-06)[2020-12-01]https://arxiv.org/abs/1411.1784.
[10] RADFORD A, METZ L, CHINTALA S. Unsupervised repress-enttation learning with deep convolutional generative adversarial networks[EB/OL]. (2016-01-07)[2020-12-01] https://arxiv.org/abs/1511.06434.
[11] ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN [EB/OL].(2017-12-06)[2020-12-01] https://arxiv.org/abs/1701.07875.
[12] GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of Wasserstein GANs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA, 2017: 5769-5779.
[13] WU Jiqing, HUANG Zhiwu, THOMA J, et al. Wasserstein divergence for GANs[C]//Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany, 2018: 673-688.
[14] JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial transformer networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada, 2015: 2017-2025.
[15] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015: 1-9.
[16] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251
ZHOU Feiyan, JIN Linpeng, DONG Jun. Review of convolutional neural network[J]. Chinese journal of computers, 2017, 40(6): 1229-1251
[17] DUTA I C, LIU L, ZHU F, et al. Pyramidal convolutio-n: rethinking convolutional neural networks for vis-ual recognition[EB/OL].(2020-06-20)[2020-12-01]https://arxiv.org/abs/2006.11538.
[18] ARJOVSKY M, BOTTOU L. Towards principled methods for training generative adversarial networks[EB/OL]. (2017-01-17)[2020-12-01] https://arxiv.org/abs/1701.04862.
[19] MIYATO T, KATAOKA T, KOYAMA M, et al. Spectral normalization for generative adversarial networks[C]//6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[20] WANG Z, WANG G, HUANG B, et al. Masked face re-cognition dataset and application[EB/OL].(2020-03-23)[2020-12-01] https://arxiv.org/abs/2003.09093.
[21] LIU Ziwei, LUO Ping, WANG Xiaogang, et al. Deep learning face attributes in the wild[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile, 2015: 3730-3738.
[22] HAN Xintong, WU Zuxuan, WU Zhe, et al. VITON: an image-based virtual try-on network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 7543-7552.
[23] HAN Xintong, WU Zuxuan, HUANG Weilin, et al. Compatible and diverse fashion image inpainting[EB/OL]. (2019-04-24)[2020-12-01] https://arxiv.org/abs/1902.01096.
[24] PANDEY N, SAVAKIS A. Poly-GAN: multi-conditioned GAN for fashion synthesis[J]. Neurocomputing, 2020, 414: 356-364.
[25] DONG Haoye, LIANG Xiaodan, SHEN Xiaohui, et al. Towards multi-pose guided virtual try-on network[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South), 2019: 9025-9034.
Similar References:

Memo

-

Last Update: 2021-12-25

Copyright © CAAI Transactions on Intelligent Systems