[1]ZHANG Ji,CAO Yi,WANG Yaru,et al.Zero-shot image classification method combining VAE and StackGAN[J].CAAI Transactions on Intelligent Systems,2022,17(3):593-601.[doi:10.11992/tis.202107012]
Copy

Zero-shot image classification method combining VAE and StackGAN

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 17 Number of periods: 2022 3 Page number: 593-601 Column: 学术论文—人工智能基础 Public date: 2022-05-05
Title:
Zero-shot image classification method combining VAE and StackGAN
Author(s):
ZHANG Ji1;  CAO Yi1;  WANG Yaru2;  ZHAO Wenqing1;  ZHAI Yongjie2
1. Department of Computer, North China Electric Power University, Baoding 071003, China;
2. Department of Automation, North China Electric Power University, Baoding 071003, China
Keywords:
Deep learning; Zero-shot learning; Image classification; Variational autoencoder; Generative adversarial network; Staged network; Sentence vector; Auxiliary information
CLC:
TP18
DOI:
10.11992/tis.202107012
Abstract:
The zero-shot classification algorithm is designed to solve the classification problem in case of a few samples or even missing categories. With the development of deep learning, the application of the generation model in zero-shot classification has made a breakthrough. By generating images of missing categories, the zero-shot image classification is transformed into a traditional image classification problem based on supervised learning. However, the generated samples are unstable in quality, including missing details and color distortion, thus affecting the accuracy of image classification. To this end, the zero-shot image classification method combining variational auto-encoding (VAE) and stack generative adversarial networks (StackGAN) is proposed. Based on the VAE/GAN model, StackGAN is introduced to generate the data of missing categories. Meanwhile, the deep learning method is used to train and obtain the sentence vectors of each category as auxiliary information and build a new generation model stc–CLS–VAEStackGAN to improve the quality of generated images and subsequently improve the classification accuracy of the zero-shot images. A comparative experiment was conducted on the public dataset, and the experimental results verified the effectiveness and superiority of the method proposed herein.
References:
[1] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60: 84–90.
[2] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436–444.
[3] BIEDERMAN I. Recognition-by-components: a theory of human image understanding[J]. Psychological review, 1987, 94(2): 115–147.
[4] PALATUCCI M, POMERLEAU D, HINTON G E, et al. Zero-shot Learning with semantic output codes[C]// Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Vancouver: NIPS, 2009: 1410–1418.
[5] LAMPERT C H, NICKISCH H, HARMELING S. Learning to detect unseen object classes by between-class attribute transfer[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 951–958.
[6] ROHRBACH M, STARK M, SCHIELE B. Evaluating knowledge transfer and zero-shot learning in a large-scale setting[C]// Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 2011: 1641–1648.
[7] HABIBIAN A, MENSINK T, SNOEK C G M. Video2vec embeddings recognize events when examples are scarce[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(10): 2089–2103.
[8] 冀中, 汪浩然, 于云龙, 等. 零样本图像分类综述: 十年进展[J]. 中国科学:信息科学, 2019, 49(10): 1299–1320
JI Zhong, WANG Haoran, YU Yunlong, et al. A decadal survey of zero-shot image classification[J]. Scientia sinica (informationis), 2019, 49(10): 1299–1320
[9] 张鲁宁, 左信, 刘建伟. 零样本学习研究进展[J]. 自动化学报, 2020, 46(1): 1–23
ZHANG Luning, ZUO Xin, LIU Jianwei. Research and development on zero-shot learning[J]. Acta automatica sinica, 2020, 46(1): 1–23
[10] 冀中, 孙涛, 于云龙. 一种基于直推判别字典学习的零样本分类方法[J]. 软件学报, 2017, 28(11): 2961–2970
JI Zhong, SUN Tao, YU Yunlong. Transductive discriminative dictionary learning approach for zero-shot classification[J]. Journal of software, 2017, 28(11): 2961–2970
[11] WANG Xuesong, CHEN Chen, CHENG Yuhu. Zero-shot learning by exploiting class-related and attribute-related prior knowledge[J]. IET computer vision, 2016, 10(6): 483–492.
[12] 赵鹏, 汪纯燕, 张思颖, 等. 一种基于融合重构的子空间学习的零样本图像分类方法[J]. 计算机学报, 2021, 44(2): 409–421
ZHAO Peng, WANG Chunyan, ZHANG Siying, et al. A zero-shot image classification method based on subspace learning with the fusion of reconstruction[J]. Chinese journal of computers, 2021, 44(2): 409–421
[13] DU Yujiao, XIAO Bo, XU Wenchao, et al. Destination prediction for sharing-bikes’ trips[C]//2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC). Guiyang: IEEE, 2018: 198–202.
[14] 陈祥凤, 陈雯柏. 度量学习改进语义自编码零样本分类算法[J]. 北京邮电大学学报, 2018, 41(4): 69–75
CHEN Xiangfeng, CHEN Wenbai. Improving semantic autoencoder zero-shot classification algorithm by metric learning[J]. Journal of Beijing university of posts and telecommunications, 2018, 41(4): 69–75
[15] 吴晨, 袁昱纬, 王宏伟, 等. 基于词向量融合的遥感场景零样本分类算法[J]. 计算机科学, 2019, 46(12): 286–291
WU Chen, YUAN Yuwei, WANG Hongwei, et al. Word vectors fusion based remote sensing scenes zero-shot classification algorithm[J]. Computer science, 2019, 46(12): 286–291
[16] 冀中, 谢于中, 庞彦伟. 基于典型相关分析和距离度量学习的零样本学习[J]. 天津大学学报(自然科学与工程技术版), 2017, 50(8): 813–820
JI Zhong, XIE Yuzhong, PANG Yanwei. Zero-shot learning based on canonical correlation analysis and distance metric learning[J]. Journal of Tianjin University (science and technology edition), 2017, 50(8): 813–820
[17] XIAN Yongqin, LAMPERT C H, SCHIELE B, et al. Zero-shot learning-A comprehensive evaluation of the good, the bad and the ugly[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(9): 2251–2265.
[18] XIAN Yongqin, LORENZ T, SCHIELE B, et al. Feature generating networks for zero-shot learning[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 5542–5551.
[19] SARIYILDIZ M B, CINBIS R G. Gradient matching generative networks for zero-shot learning[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 2163–2173.
[20] MANDAL D, NARAYAN S, DWIVEDI S K, et al. Out-of-distribution detection for generalized zero-shot action recognition[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach : IEEE, 2019: 9977–9985.
[21] XIAN Yongqin, SHARMA S, SCHIELE B, et al. F-VAEGAN-D2: a feature generating framework for any-shot learning[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach : IEEE, 2019: 10267–10276.
[22] KIM H, LEE J, BYUN H. Unseen image generating domain-free networks for generalized zero-shot learning[J]. Neurocomputing, 2020, 411: 67–77.
[23] VERMA V K, BRAHMA D, RAI P. Meta-learning for generalized zero-shot learning[J]. Proceedings of the AAAI conference on artificial intelligence, 2020, 34(4): 6062–6069.
[24] MA Yuanbo, XU Xing, SHEN Fumin, et al. Similarity preserving feature generating networks for zero-shot learning[J]. Neurocomputing, 2020, 406: 333–342.
[25] LIU Huan, YAO Lina, ZHENG Qinghua, et al. Dual-stream generative adversarial networks for distributionally robust zero-shot learning[J]. Information sciences, 2020, 519: 407–422.
[26] LIU Jinlu, ZHANG Zhaocheng, YANG Gang. Cross-class generative network for zero-shot learning[J]. Information sciences, 2021, 555: 147–163.
[27] TANG C, HE Z, LI Y, ET Al. Zero-shot learning via structure-aligned generative adversarial network[J]. IEEE transactions on neural networks and learning systems, 2021(99): 1–14.
[28] LI Zhiqun, CHEN Qiong, LIU Qingfa. Augmented semantic feature based generative network for generalized zero-shot learning[J]. Neural networks:the official journal of the international neural network society, 2021, 143: 1–11.
[29] GAO RUI, HOU XINGSONG, QIN JIE, et al. Zero-VAE-GAN: generating unseen features for generalized and transductive zero-shot learning[J]. IEEE transactions on image processing, 2020, 29: 3665–3680.
[30] KINGMA D P, WELLING M. Auto-Encoding Variational Bayes[EB/OL]. (2014–05–01)[2022–03–10]https://arxiv.org/abs/1312.6114v2.
[31] LARSEN A B L, S?NDERBY S K, LAROCHELLE H, et al. Autoencoding beyond pixels using a learned similarity metric[EB/OL]. (2016–02–10)[2022–03–10]https://arxiv.org/abs/1512.09300.
[32] ZHANG Han, XU Tao, LI Hongsheng, et al. StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks[EB/OL]. (2017–08–05)[2022-03-10]https://arxiv.org/abs/1612.03242v2.
[33] Wah C, Branson S, Welinder P, et al. The Caltech-UCSD Birds-200-2011 Dataset[J]. California institute of technology, 2011.
[34] LAMPERT C H, NICKISCH H, HARMELING S. Attribute-based classification for zero-shot visual object categorization[J]. IEEE transactions on pattern analysis and machine intelligence, 2014, 36(3): 453–465.
[35] CHE TONG, LI YANRAN, JACOB A P, et al. Mode regularized generative adversarial networks[EB/OL]. (2017–03–02)[2022–03–10]https://arxiv.org/abs/1612.02136 .
[36] 魏宏喜, 张越. 基于生成对抗网络的零样本图像分类[J]. 北京航空航天大学学报, 2019, 45(12): 2345–2350
WEI Hongxi, ZHANG Yue. Zero-shot image classification based on generative adversarial network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(12): 2345–2350
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems