[1]姜义,吕荣镇,刘明珠,等.基于生成对抗网络的人脸口罩图像合成[J].智能系统学报,2021,16(6):1073-1080.[doi:10.11992/tis.202012010]
JIANG Yi,LYU Rongzhen,LIU Mingzhu,et al.Masked face image synthesis based on a generative adversarial network[J].CAAI Transactions on Intelligent Systems,2021,16(6):1073-1080.[doi:10.11992/tis.202012010]
点击复制
JIANG Yi,LYU Rongzhen,LIU Mingzhu,et al.Masked face image synthesis based on a generative adversarial network[J].CAAI Transactions on Intelligent Systems,2021,16(6):1073-1080.[doi:10.11992/tis.202012010]
基于生成对抗网络的人脸口罩图像合成
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第6期
页码:
1073-1080
栏目:
学术论文—智能系统
出版日期:
2021-11-05
- Title:
- Masked face image synthesis based on a generative adversarial network
- Keywords:
- deep learning; generative adversarial networks; spatial transformation; convolution neural network; image fusion; face mask; human face dataset; face recognition
- 分类号:
- TP391
- 摘要:
- 为了解决现阶段缺乏被口罩遮挡的人脸数据集的问题，本文提出了基于生成对抗网络与空间变换网络相结合生成戴口罩的人脸图像的方法。本文的方法以生成对抗网络为基础，结合了多尺度卷积核对图像进行不同尺度的特征提取，并引入了沃瑟斯坦散度作为度量真实样本和合成样本之间的距离，并以此来优化生成器的性能。实验表明，所提方法能够在没有对原始图像进行任何标注的情况下有效地对人脸图像进行口罩佩戴，且合成的图像具有较高的真实性。
- Abstract:
- This paper proposes a method for generating masked face images using a generative adversarial network (GAN) and spatial transformer networks. The proposed method is used to solve the present problem of lacking face datasets of people wearing masks. Based on the GAN, the proposed method introduces a multiscale convolution kernel to extract image characteristics in various dimensions. This method introduces the Wasserstein divergence to measure the distance between an authentic specimen and a synthetic specimen so that generator’s performance can be optimized. Experiments show that the proposed method can add a mask to a face image effectively without any annotations on the original image, and the synthesized image has high fidelity.
- 参考文献/References:
-
[1] 国家卫生健康委员会.新型冠状病毒肺炎诊疗方案(试行第八版)[EB/OL].(2020-08-18)[2020-12-08]http://www.gov.cn/zhengce/zhengceku/2020-08/19/content_5535757.htm.
National Health Commission of the People’s Republic of China. Diagnosis and treatment protocol for novel coronavirus pneumonia (trial version 8) [EB/OL].(2020-08-18)[2020-12-01]http://www.gov.cn/zhengce/zhengceku/2020-08/19/content_5535757.htm.
[2] TAIGMAN Y, YANG Ming, RANZATO M A, et al. DeepFace: closing the gap to human-level performance in face verification[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA, 2014: 1701-1708.
[3] SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015: 815-823.
[4] 李小薪, 梁荣华. 有遮挡人脸识别综述: 从子空间回归到深度学习[J]. 计算机学报, 2018, 41(1): 177-207
LI Xiaoxin, LIANG Ronghua. A review for face recognition with occlusion: from subspace regression to deep learning[J]. Chinese journal of computers, 2018, 41(1): 177-207
[5] ANWAR A, RAYCHOWDHURY A. Masked face recognition for secure authentication[EB/OL].(2020-08-25)[2020-12-01] https://arxiv.org/abs/2008.11104.
[6] CABANI A, HAMMOUNDI K, BENHABILES H, et al. Masked-Face-Net—a dataset of correctly/incorrectly masked face images in the context of covid-19[EB/OL].(2020-08-18)[2020-12-01] https://arxiv.org/abs/2008.08016.
[7] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada, 2014: 2672-2680.
[8] 胡铭菲,刘建伟,左信.深度生成模型综述[EB/OL].（2021-10-28）[2021-10-30] https://doi.org/10.16383/j.aas.c190866.
HU Mingfei, LIU Jianwei, ZUO Xin. Survey on deep generative model[EB/OL].（2021-10-28）[2021-10-30] https://doi.org/10.16383/j.aas.c190866.
[9] MIRZA M, OSINDERO S. Conditional generative adversarial nets[EB/OL].(2014-11-06)[2020-12-01]https://arxiv.org/abs/1411.1784.
[10] RADFORD A, METZ L, CHINTALA S. Unsupervised repress-enttation learning with deep convolutional generative adversarial networks[EB/OL]. (2016-01-07)[2020-12-01] https://arxiv.org/abs/1511.06434.
[11] ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN [EB/OL].(2017-12-06)[2020-12-01] https://arxiv.org/abs/1701.07875.
[12] GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of Wasserstein GANs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA, 2017: 5769-5779.
[13] WU Jiqing, HUANG Zhiwu, THOMA J, et al. Wasserstein divergence for GANs[C]//Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany, 2018: 673-688.
[14] JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial transformer networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada, 2015: 2017-2025.
[15] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015: 1-9.
[16] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251
ZHOU Feiyan, JIN Linpeng, DONG Jun. Review of convolutional neural network[J]. Chinese journal of computers, 2017, 40(6): 1229-1251
[17] DUTA I C, LIU L, ZHU F, et al. Pyramidal convolutio-n: rethinking convolutional neural networks for vis-ual recognition[EB/OL].(2020-06-20)[2020-12-01]https://arxiv.org/abs/2006.11538.
[18] ARJOVSKY M, BOTTOU L. Towards principled methods for training generative adversarial networks[EB/OL]. (2017-01-17)[2020-12-01] https://arxiv.org/abs/1701.04862.
[19] MIYATO T, KATAOKA T, KOYAMA M, et al. Spectral normalization for generative adversarial networks[C]//6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[20] WANG Z, WANG G, HUANG B, et al. Masked face re-cognition dataset and application[EB/OL].(2020-03-23)[2020-12-01] https://arxiv.org/abs/2003.09093.
[21] LIU Ziwei, LUO Ping, WANG Xiaogang, et al. Deep learning face attributes in the wild[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile, 2015: 3730-3738.
[22] HAN Xintong, WU Zuxuan, WU Zhe, et al. VITON: an image-based virtual try-on network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 7543-7552.
[23] HAN Xintong, WU Zuxuan, HUANG Weilin, et al. Compatible and diverse fashion image inpainting[EB/OL]. (2019-04-24)[2020-12-01] https://arxiv.org/abs/1902.01096.
[24] PANDEY N, SAVAKIS A. Poly-GAN: multi-conditioned GAN for fashion synthesis[J]. Neurocomputing, 2020, 414: 356-364.
[25] DONG Haoye, LIANG Xiaodan, SHEN Xiaohui, et al. Towards multi-pose guided virtual try-on network[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South), 2019: 9025-9034.
- 相似文献/References:
-
[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[4]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[5]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[6]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[7]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
[8]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[9]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(6):894.[doi:10.11992/tis.201706037]
YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12():894.[doi:10.11992/tis.201706037]
[10]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
[11]朱文霖,刘华平,王博文,等.基于视-触跨模态感知的智能导盲系统[J].智能系统学报,2020,15(1):33.[doi:10.11992/tis.201908015]
ZHU Wenlin,LIU Huaping,WANG Bowen,et al.An intelligent blind guidance system based on visual-touch cross-modal perception[J].CAAI Transactions on Intelligent Systems,2020,15():33.[doi:10.11992/tis.201908015]
[12]毕晓君,潘梦迪.基于生成对抗网络的机载遥感图像超分辨率重建[J].智能系统学报,2020,15(1):74.[doi:10.11992/tis.202002002]
BI Xiaojun,PAN Mengdi.Super-resolution reconstruction of airborne remote sensing images based on the generative adversarial networks[J].CAAI Transactions on Intelligent Systems,2020,15():74.[doi:10.11992/tis.202002002]
[13]王昌安,田金文.生成对抗网络辅助学习的舰船目标精细识别[J].智能系统学报,2020,15(2):296.[doi:10.11992/tis.201901004]
WANG Changan,TIAN Jinwen.Fine-grained inshore ship recognition assisted by deep-learning generative adversarial networks[J].CAAI Transactions on Intelligent Systems,2020,15():296.[doi:10.11992/tis.201901004]
[14]张冀,曹艺,王亚茹,等.融合VAE和StackGAN的零样本图像分类方法[J].智能系统学报,2022,17(3):593.[doi:10.11992/tis.202107012]
ZHANG Ji,CAO Yi,WANG Yaru,et al.Zero-shot image classification method combining VAE and StackGAN[J].CAAI Transactions on Intelligent Systems,2022,17():593.[doi:10.11992/tis.202107012]
[15]翟永杰,张智柏,王亚茹.基于改进TransGAN的零样本图像识别方法[J].智能系统学报,2023,18(2):352.[doi:10.11992/tis.202111002]
ZHAI Yongjie,ZHANG Zhibai,WANG Yaru.An image recognition method of zero -shot learning based on an improved TransGAN[J].CAAI Transactions on Intelligent Systems,2023,18():352.[doi:10.11992/tis.202111002]
[16]卢娟,陈纯毅.融合递归自编解码器的蒙卡画面重构降噪方法[J].智能系统学报,2023,18(3):459.[doi:10.11992/tis.202112011]
LU Juan,CHEN Chunyi.A denoised method by fusing recursive auto-encoder decorder for Monte Carlo rendering image reconstruction[J].CAAI Transactions on Intelligent Systems,2023,18():459.[doi:10.11992/tis.202112011]
[17]管凤旭,路斯棋,郑岩.多尺度特征融合的双判别器残差生成对抗网络[J].智能系统学报,2023,18(5):917.[doi:10.11992/tis.202207005]
GUAN Fengxu,LU Siqi,ZHENG Yan.Dual discriminator residual generation adversarial network with multiscale feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18():917.[doi:10.11992/tis.202207005]
备注/Memo
收稿日期:2020-12-03。
基金项目:国家自然科学基金项目(61601149)；黑龙江省科学基金项目(QC2017074)；黑龙江省普通本科高等学校青年创新人才培养计划项目(UNPYSCT-2018199)
作者简介:姜义，讲师，主要研究方向为人工智能、传感器网络、分布式系统;吕荣镇，硕士研究生，主要研究方向为人工智能;刘明珠，副教授，主要研究方向为通信与信息系统。发表学术论文10余篇
通讯作者:姜义.E-mail:jasonj@hrbust.edu.cn
更新日期/Last Update:
2021-12-25