[1]管凤旭,路斯棋,郑岩.多尺度特征融合的双判别器残差生成对抗网络[J].智能系统学报,2023,18(5):917-925.[doi:10.11992/tis.202207005]
GUAN Fengxu,LU Siqi,ZHENG Yan.Dual discriminator residual generation adversarial network with multiscale feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18(5):917-925.[doi:10.11992/tis.202207005]
点击复制
GUAN Fengxu,LU Siqi,ZHENG Yan.Dual discriminator residual generation adversarial network with multiscale feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18(5):917-925.[doi:10.11992/tis.202207005]
多尺度特征融合的双判别器残差生成对抗网络
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第5期
页码:
917-925
栏目:
学术论文—机器学习
出版日期:
2023-09-05
- Title:
- Dual discriminator residual generation adversarial network with multiscale feature fusion
- Keywords:
- generative adversarial networks; deep learning; unsupervised model; mode collapse; gradient explosion; gradient disappearance; multiscale feature fusion; training stability
- 分类号:
- TP18
- 摘要:
- 生成对抗网络(generative adversarial networks, GANs)作为一类基于深度学习的无监督生成模型，无需对数据分布进行建模便可以生成真实且质量较高的图像。标准的GANs往往训练困难，常出现梯度消失、梯度爆炸或者模式崩溃等问题，限制模型的性能。为解决模式崩溃问题，本文提出一种双判别器结构来提高模型生成图像的多样性。另外，本文改进了生成器模型和判别器模型，提出一种基于残差网络和多尺度特征融合的生成器和基于多尺度特征融合的判别器，在提高生成图像质量的前提下解决深层网络出现的梯度消失、梯度爆炸的问题。将其应用于MNIST、LSUN、CelebA数据集上，训练结果稳定且生成图像质量较高，取得了令人满意的FID和IS值。
- Abstract:
- Generative adversarial networks (GANs), a class of unsupervised generative models based on deep learning, have shown promising results in generating realistic and high-quality images without modeling the data distribution. However, standard GANs are often difficult to train and often suffer from gradient disappearance, gradient explosion, or mode collapse, which can limit the overall performance of the model. To address the mode collapse problem and improve the variety of the generated images, this paper proposes a dual discriminator structure. Furthermore, this paper improves the generator and discriminator model and proposes a generator based on residual network and multiscale feature fusion and a discriminator based on multiscale feature fusion, which effectively solves the problem of gradient disappearance and gradient explosion that occurs in deep networks while improving the quality of generated images. The proposed approach is applied to various datasets, including MNIST, LSUN, and CelebA. The training results reveal that the stability and quality of the generated images are high, achieving satisfying FID and IS values.
- 参考文献/References:
-
[1] KINGMA D P, WELLING M. Auto-encoding variational Bayes[C]//2nd International Conference on Learning Representations. Banff: ICLR, 2014: 1-14.
[2] REZENDE D J, MOHAMED S, WIERSTRA D. Stochastic backpropagation and approximate inference in deep generative models[C]//2014 International Conference on Machine Learning. SAN DIEGO: JMLR, 2014: 1278?1286.
[3] HINTON G E, SEJNOWSKI T J, ACKLEY D H. Boltzmann machines: constraint satisfaction networks that learn[J]. Cognitive science, 1994, 9: 1-40.
[4] ACKLEY D H, HINTON G E, SEJNOWSKI T J. A learning algorithm for boltzmann machines[J]. Cognitive science, 1985, 9(1): 147-169.
[5] GOODFELLOW I, POUGET-ABADIE J, MIRZA M. Generative adversarial nets[C]//Proceedings of the 2014 Conference on Advances in Neural Information Processing Systems 27. Montreal: Curran Associates, 2014. 2672?2680.
[6] RUAN Congcong, YUAN Liuchun, HU Haifeng, et al. Image translation with dual-directional generative adversarial networks[J]. IET computer vision, 2021, 15(1): 73-83.
[7] CHEN Xiaocong, LI Yun, YAO Lina, et al. Generative adversarial U-net for domain-free medical image augmentation[EB/OL]. (2021?01?12)[2022?07?06]. https://arxiv.org/abs/2101.04793.
[8] KONG Jiahui, SHEN Haibin, HUANG Kejie. DualPathGAN: facial reenacted emotion synthesis[J]. IET computer vision, 2021, 15(7): 501-513.
[9] 张冀, 曹艺, 王亚茹, 等. 融合VAE和StackGAN的零样本图像分类方法[J]. 智能系统学报, 2022, 17(3): 593-601
ZHANG Ji, CAO Yi, WANG Yaru, et al. Zero-shot image classification method combining VAE and StackGAN[J]. CAAI transactions on intelligent systems, 2022, 17(3): 593-601
[10] 王凯旋, 任福继, 倪红军, 等. 基于循环互相关系数的CGAN温度值图像扩增[J]. 智能系统学报, 2022, 17(1): 32-40
WANG Kaixuan, REN Fuji, NI Hongjun, et al. Image amplification for temperature value image based on cyclic cross-correlation coefficient CGAN[J]. CAAI transactions on intelligent systems, 2022, 17(1): 32-40
[11] 姜义, 吕荣镇, 刘明珠, 等. 基于生成对抗网络的人脸口罩图像合成[J]. 智能系统学报, 2021, 16(6): 1073-1080
JIANG Yi, LYU Rongzhen, LIU Mingzhu, et al. Masked face image synthesis based on a generative adversarial network[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1073-1080
[12] 尹诗, 侯国莲, 胡晓东, 等. 基于AC-GAN数据重构的风电机组主轴承温度监测方法[J]. 智能系统学报, 2021, 16(6): 1106-1116
YIN Shi, HOU Guolian, HU Xiaodong, et al. Temperature monitoring method of the main bearing of wind turbine based on AC-GAN data reconstruction[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1106-1116
[13] MIYATO T, KATAOKA T, KOYAMA M, et al. Spectral normalization for generative adversarial networks[EB/OL]. (2018?02?16)[2022?07?06]. https://arxiv.org/abs/1802.05957.
[14] CASANOVA A, CAREIL M, VERBEEK J, et al. Instance-conditioned GAN[C]//35th Conference on Neural Information Processing Systems. North torrey: NIPS, 2021.
[15] CHAVDAROVA T, FLEURET F. SGAN: an alternative training of generative adversarial networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 9407-9415.
[16] KANG M, SHIM W J, CHO M, et al. Rebooting ACGAN: auxiliary classifier GANs with stable training[C]// Neural Information Processing Systems. Virtual: NeurlPS, 2021: 1-13.
[17] 史彩娟, 涂冬景, 刘靖祎. Re-GAN: 残差生成式对抗网络算法[J]. 中国图象图形学报, 2021, 26(3): 594-604
SHI Caijuan, TU Dongjing, LIU Jingyi. Re-GAN: residual generative adversarial network algorithm[J]. Journal of image and graphics, 2021, 26(3): 594-604
[18] RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. (2016?01?07)[2022?07?06]. https://arxiv.org/abs/1511.06434.
[19] KARRAS T, AILA, LAINE S, et al. Progressive growing of GANs for improved quality, stability, and variation[EB/OL]. (2018?02?26)[2022?07?06]. https://arxiv.org/abs/1710.10196.
[20] MIRZA M, OSINDERO S. Conditional generative adversarial nets[EB/OL]. (2014?11?06)[2022?07?06]. https://arxiv.org/abs/1411.1784.
[21] ODENA A, OLAH C, SHLENS J. Conditional image synthesis with auxiliary classifier GANs[C]// 34th International Conference on Machine Learning. San Diego: JMLR, 2017.
[22] ZHOU Peng, XIE Lingxi, NI Bingbing, et al. Omni-GAN: on the secrets of cGANs and beyond[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 14041-14051.
[23] NGUYEN T D, LE Trung, VU Hung, et al. Dual discriminator generative adversarial nets[J]. Advances in neural information processing systems, 2017, 30(12): 2671-2681.
[24] ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein Gan[EB/OL]. (2017?12?06)[2022?07?06]. https://arxiv.org/abs/1701.07875.
[25] Gulrajani I, Ahmed F, Arjovsky M, et al. Improved training of wasserstein GANs[J]. Advances in neural information processing systems, 2017, 30(12): 5768-5778.
[26] BHASKARA V S, AUMENTADO-ARMSTRONG T, JEPSON A, et al. GraN-GAN: piecewise gradient normalization for generative adversarial networks[C]//2022 IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2022: 2432-2441.
[27] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[28] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261-2269.
- 相似文献/References:
-
[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[4]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[5]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[6]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[7]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
[8]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[9]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(6):894.[doi:10.11992/tis.201706037]
YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12():894.[doi:10.11992/tis.201706037]
[10]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
[11]朱文霖,刘华平,王博文,等.基于视-触跨模态感知的智能导盲系统[J].智能系统学报,2020,15(1):33.[doi:10.11992/tis.201908015]
ZHU Wenlin,LIU Huaping,WANG Bowen,et al.An intelligent blind guidance system based on visual-touch cross-modal perception[J].CAAI Transactions on Intelligent Systems,2020,15():33.[doi:10.11992/tis.201908015]
[12]毕晓君,潘梦迪.基于生成对抗网络的机载遥感图像超分辨率重建[J].智能系统学报,2020,15(1):74.[doi:10.11992/tis.202002002]
BI Xiaojun,PAN Mengdi.Super-resolution reconstruction of airborne remote sensing images based on the generative adversarial networks[J].CAAI Transactions on Intelligent Systems,2020,15():74.[doi:10.11992/tis.202002002]
[13]王昌安,田金文.生成对抗网络辅助学习的舰船目标精细识别[J].智能系统学报,2020,15(2):296.[doi:10.11992/tis.201901004]
WANG Changan,TIAN Jinwen.Fine-grained inshore ship recognition assisted by deep-learning generative adversarial networks[J].CAAI Transactions on Intelligent Systems,2020,15():296.[doi:10.11992/tis.201901004]
[14]姜义,吕荣镇,刘明珠,等.基于生成对抗网络的人脸口罩图像合成[J].智能系统学报,2021,16(6):1073.[doi:10.11992/tis.202012010]
JIANG Yi,LYU Rongzhen,LIU Mingzhu,et al.Masked face image synthesis based on a generative adversarial network[J].CAAI Transactions on Intelligent Systems,2021,16():1073.[doi:10.11992/tis.202012010]
[15]张冀,曹艺,王亚茹,等.融合VAE和StackGAN的零样本图像分类方法[J].智能系统学报,2022,17(3):593.[doi:10.11992/tis.202107012]
ZHANG Ji,CAO Yi,WANG Yaru,et al.Zero-shot image classification method combining VAE and StackGAN[J].CAAI Transactions on Intelligent Systems,2022,17():593.[doi:10.11992/tis.202107012]
[16]翟永杰,张智柏,王亚茹.基于改进TransGAN的零样本图像识别方法[J].智能系统学报,2023,18(2):352.[doi:10.11992/tis.202111002]
ZHAI Yongjie,ZHANG Zhibai,WANG Yaru.An image recognition method of zero -shot learning based on an improved TransGAN[J].CAAI Transactions on Intelligent Systems,2023,18():352.[doi:10.11992/tis.202111002]
[17]卢娟,陈纯毅.融合递归自编解码器的蒙卡画面重构降噪方法[J].智能系统学报,2023,18(3):459.[doi:10.11992/tis.202112011]
LU Juan,CHEN Chunyi.A denoised method by fusing recursive auto-encoder decorder for Monte Carlo rendering image reconstruction[J].CAAI Transactions on Intelligent Systems,2023,18():459.[doi:10.11992/tis.202112011]
备注/Memo
收稿日期:2022-7-6。
基金项目:国家自然科学基金项目（62101156）.
作者简介:管凤旭，副教授，博士，主要研究方向为无人系统自主控制、机器视觉目标检测与跟踪、计算机控制及应用。获授权发明专利近20项，出版教材5部，发表学术论文40余篇;路斯棋，硕士研究生，主要研究方向为水下图像处理、计算机视觉;郑岩，讲师，博士，主要研究方向为计算机视觉和机器学习
通讯作者:管凤旭.E-mail：guanfengxu@hrbeu.edu.cn
更新日期/Last Update:
1900-01-01