[1]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(6):894-898.[doi:10.11992/tis.201706037]
YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12(6):894-898.[doi:10.11992/tis.201706037]
点击复制
YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12(6):894-898.[doi:10.11992/tis.201706037]
基于自编码器的特征迁移算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
12
期数:
2017年第6期
页码:
894-898
栏目:
学术论文—机器学习
出版日期:
2017-12-25
- Title:
- Feature transfer algorithm based on an auto-encoder
- Keywords:
- auto-encoder; feature transfer; deep network; deep learning; image classification; mid-level image representation; visual recognition; large-scale datasets
- 分类号:
- TP181
- 摘要:
- 近年来，栈式自编码网络（stacked auto-encoder，SAE）在大规模数据集上表现出优异的图像分类性能。相对于其他图像分类方法中手工设计的低级特征，SAE的成功归因于深度网络能够学习到丰富的中级图像特征。然而，估计上百万个网络参数需要非常庞大的带标签的图像样本数据集。这样的性质阻止了SAE在小规模训练数据上的许多应用。在这篇文章中，提出的算法展示如何将SAE在大规模数据集上学习到的图像表示有效地迁移到只有有限训练数据的视觉识别任务中。实验部分设计了一个方法来复用在MNIST数据集上训练得到的隐藏层，以此计算在MNIST-variations数据集上的中级图像表示。实验结果展示了尽管两个数据集之间存在差异，但是被迁移的图像特征能够使得模型的分类性能得到极大的提升。
- Abstract:
- The stacked auto-encoder (SAE) has recently shown outstanding image classification performance in large-scale datasets. Relative to the low-level features of artificial design in other image classification methods, the success of SAE is its deep network that can learn rich mid-level image features. However, estimating millions of parameters requires a very large number of annotated image samples, and this prevents many SAE applications to small-scale training data. In this paper, the proposed algorithm shows how to efficiently transfer image representation learned by SAE on a large-scale dataset to other visual recognition tasks with limited training data. In the experimental section, a method is designed to reuse the hidden layers trained on MNIST datasets to compute the mid-level image representation of MNIST-variation datasets. Experimental results show that, despite differences in image datasets, the transferred image features can significantly improve the classification performance of the model.
- 参考文献/References:
-
[1] HINTON G E, ZEMEL R S. Autoencoder minimum description length and helmholtz free energy[C]//Conference on Neural Information Processing Systems(NIPS). Denver, USA, 1993: 3-10.
[2] SOCHER R, HUANG E H, PENNINGTON J, et al. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection[C]//Proc Neural Information and Processing Systems. Granada, Spain, 2011: 801-809.
[3] SWERSKY K, RANZATO M, BUCHMAN D, et al. On score matching for energy based models: generalizing autoencoders and simplifying deep learning[C]//Proc Int’l Conf Machine Learning Bellevue. Washington, USA, 2011: 1201-1208.
[4] FISCHER A, IGEL C. An introduction to restricted Boltzmann machines[C]//Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. Guadalajara, Mexico, 2012: 14-36.
[5] BREULEUX O, BENGIO Y, VINCENT P. Quickly generating representative samples from an RBM-derived process[J]. Neural computation, 2011, 23(8): 2053-2073.
[6] COURVILLE A, BERGSTRA J, BENGIO Y. Unsupervised models of images by spike-and-slab RBMs[C]//Proc Int’l Conf Machine Learning. Bellevue, Washington, USA, 2011: 1145-1152.
[7] SCHMAH T, HINTON G E, ZEMEL R, et al. Generative versus discriminative training of RBMs for classification of fMRI images[C]//Proc Neural Information and Processing Systems. Vancouver, Canada, 2008: 1409-1416.
[8] ERHAN D, BENGIO Y, COURVILLE A, et al. Why does unsupervised pre-training help deep learning?[J]. Machine learning research, 2010, 11: 625-660.
[9] VINCENT P. Extracting and composing robust features with denoising auto-encoders[C]//International Conference on Machine Learning(ICML). Helsinki, Finland, 2008: 1096-1103.
[10] VINCENT P, LAROCHELLE H, LAJOIE I, et al. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion[J]. Machine learning research, 2010, 11: 3371-3408.
[11] CHEN M, XU Z, WINBERGER K Q, et al. Marginalized denoising autoencoders for domain adaptation[C]//International Conference on Machine Learning. Edinburgh, Scotland, 2012: 1206-1214.
[12] VINCENT P. A connection between score matching and denoising autoencoders[J]. Neural computation, 2011, 23(7): 1661-1674.
[13] RIFAI S. Contractive auto-encoders: explicit invariance during feature extraction[C]//Proceedings of the Twenty-eight International Conference on Machine Learning. Bellevue, USA, 2011: 833-840
[14] RIFAI S, BENGIO Y, DAUPHIN Y, et al. A generative process for sampling contractive auto-encoders[C]//Proc Int’l Conf Machine Learning. Edinburgh, Scotland, UK, 2012: 1206-1214.
[15] LECUN Y. Neural networks: tricks of the trade (2nd ed.) [M]. Germany: Springer, 2012: 9-48
[16] ZEILER M D, TAYLOR G W, FERGUS R. Adaptive deconvolutional networks for mid and high level feature learning[C]//IEEE International Conference on Computer Vision. Barcelona, Spain, 2011: 2013-2025.
- 相似文献/References:
-
[1]莫凌飞,蒋红亮,李煊鹏.基于深度学习的视频预测研究综述[J].智能系统学报,2018,13(1):85.[doi:10.11992/tis.201707032]
MO Lingfei,JIANG Hongliang,LI Xuanpeng.Review of deep learning-based video prediction[J].CAAI Transactions on Intelligent Systems,2018,13():85.[doi:10.11992/tis.201707032]
[2]杨文元.多标记学习自编码网络无监督维数约简[J].智能系统学报,2018,13(5):808.[doi:10.11992/tis.201804051]
YANG Wenyuan.Unsupervised dimensionality reduction of multi-label learning via autoencoder networks[J].CAAI Transactions on Intelligent Systems,2018,13():808.[doi:10.11992/tis.201804051]
[3]张潇鲲,刘琰,陈静.引入外部词向量的文本信息网络表示学习[J].智能系统学报,2019,14(5):1056.[doi:10.11992/tis.201809037]
ZHANG Xiaokun,LIU Yan,CHEN Jing.Representation learning using network embedding based on external word vectors[J].CAAI Transactions on Intelligent Systems,2019,14():1056.[doi:10.11992/tis.201809037]
[4]王丽娟,丁世飞.一种基于ELM-AE特征表示的谱聚类算法[J].智能系统学报,2021,16(3):560.[doi:10.11992/tis.202005021]
WANG Lijuan,DING Shifei.A spectral clustering algorithm based on ELM-AE feature representation[J].CAAI Transactions on Intelligent Systems,2021,16():560.[doi:10.11992/tis.202005021]
[5]马睿,周治平.结合地标点与自编码的快速多视图聚类网络[J].智能系统学报,2022,17(2):333.[doi:10.11992/tis.202101011]
MA Rui,ZHOU Zhiping.Fast multiview clustering network combining landmark points and autoencoder[J].CAAI Transactions on Intelligent Systems,2022,17():333.[doi:10.11992/tis.202101011]
[6]吴晓东,熊伟丽.采用编码输入的生成对抗网络故障检测方法及应用[J].智能系统学报,2022,17(3):496.[doi:10.11992/tis.202102003]
WU Xiaodong,XIONG Weili.Fault detection method and its application using GAN with an encoded input[J].CAAI Transactions on Intelligent Systems,2022,17():496.[doi:10.11992/tis.202102003]
[7]杨梦茵,陈俊芬,翟俊海.非对称卷积编码器的聚类算法[J].智能系统学报,2022,17(5):900.[doi:10.11992/tis.202107021]
YANG Mengyin,CHEN Junfen,ZHAI Junhai.A clustering method based on the asymmetric convolutional autoencoder[J].CAAI Transactions on Intelligent Systems,2022,17():900.[doi:10.11992/tis.202107021]
[8]张智慧,杨燕,张熠玲.面向不完整多视图聚类的深度互信息最大化方法[J].智能系统学报,2023,18(1):12.[doi:10.11992/tis.202203051]
ZHANG Zhihui,YANG Yan,ZHANG Yiling.Deep mutual information maximization method for incomplete multi-view clustering[J].CAAI Transactions on Intelligent Systems,2023,18():12.[doi:10.11992/tis.202203051]
[9]戚银城,耿劭锋,赵振兵,等.基于特征迁移的螺栓图像超分辨率处理方法[J].智能系统学报,2023,18(4):858.[doi:10.11992/tis.202201009]
QI Yincheng,GENG Shaofeng,ZHAO Zhenbing,et al.A method for super resolution processing of bolt image based on feature transfer[J].CAAI Transactions on Intelligent Systems,2023,18():858.[doi:10.11992/tis.202201009]
[10]邢天祎,郭茂祖,陈加栋,等.基于空时对抗变分自编码器的人群异常行为检测[J].智能系统学报,2023,18(5):994.[doi:10.11992/tis.202303002]
XING Tianyi,GUO Maozu,CHEN Jiadong,et al.Detection of abnormal crowd behavior based on spatial-temporal adversarial variational autoencoder[J].CAAI Transactions on Intelligent Systems,2023,18():994.[doi:10.11992/tis.202303002]
备注/Memo
收稿日期:2017-06-10;改回日期:。
基金项目:国家自然科学基金项目（61672364）.
作者简介:杨梦铎,女,1989年生,讲师,博士,主要研究方向为模式识别与机器学习;栾咏红,女,1968年生,副教授,主要研究方向为强化学习;刘文军,男,1981年生,讲师,博士,主要研究方向为无线传感网络与算法分析。
通讯作者:杨梦铎.E-mail:mengduoyang@163.com.
更新日期/Last Update:
2018-01-03