[1]YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12(6):894-898.[doi:10.11992/tis.201706037]
Copy

Feature transfer algorithm based on an auto-encoder

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 12 Number of periods: 2017 6 Page number: 894-898 Column: 学术论文—机器学习 Public date: 2017-12-25
Title:
Feature transfer algorithm based on an auto-encoder
Author(s):
YANG Mengduo1;  LUAN Yonghong1;  LIU Wenjun1;  LI Fanzhang2
1. Department of Software and Service Outsourcing, Suzhou Vocational Institute of Industrial Technology, Suzhou 215104, China;
2. School of Computer Science and Technology, Soochow University, Suzhou 215006, China
Keywords:
auto-encoder; feature transfer; deep network; deep learning; image classification; mid-level image representation; visual recognition; large-scale datasets
CLC:
TP181
DOI:
10.11992/tis.201706037
Abstract:
The stacked auto-encoder (SAE) has recently shown outstanding image classification performance in large-scale datasets. Relative to the low-level features of artificial design in other image classification methods, the success of SAE is its deep network that can learn rich mid-level image features. However, estimating millions of parameters requires a very large number of annotated image samples, and this prevents many SAE applications to small-scale training data. In this paper, the proposed algorithm shows how to efficiently transfer image representation learned by SAE on a large-scale dataset to other visual recognition tasks with limited training data. In the experimental section, a method is designed to reuse the hidden layers trained on MNIST datasets to compute the mid-level image representation of MNIST-variation datasets. Experimental results show that, despite differences in image datasets, the transferred image features can significantly improve the classification performance of the model.
References:
[1] HINTON G E, ZEMEL R S. Autoencoder minimum description length and helmholtz free energy[C]//Conference on Neural Information Processing Systems(NIPS). Denver, USA, 1993: 3-10.
[2] SOCHER R, HUANG E H, PENNINGTON J, et al. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection[C]//Proc Neural Information and Processing Systems. Granada, Spain, 2011: 801-809.
[3] SWERSKY K, RANZATO M, BUCHMAN D, et al. On score matching for energy based models: generalizing autoencoders and simplifying deep learning[C]//Proc Int’l Conf Machine Learning Bellevue. Washington, USA, 2011: 1201-1208.
[4] FISCHER A, IGEL C. An introduction to restricted Boltzmann machines[C]//Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. Guadalajara, Mexico, 2012: 14-36.
[5] BREULEUX O, BENGIO Y, VINCENT P. Quickly generating representative samples from an RBM-derived process[J]. Neural computation, 2011, 23(8): 2053-2073.
[6] COURVILLE A, BERGSTRA J, BENGIO Y. Unsupervised models of images by spike-and-slab RBMs[C]//Proc Int’l Conf Machine Learning. Bellevue, Washington, USA, 2011: 1145-1152.
[7] SCHMAH T, HINTON G E, ZEMEL R, et al. Generative versus discriminative training of RBMs for classification of fMRI images[C]//Proc Neural Information and Processing Systems. Vancouver, Canada, 2008: 1409-1416.
[8] ERHAN D, BENGIO Y, COURVILLE A, et al. Why does unsupervised pre-training help deep learning?[J]. Machine learning research, 2010, 11: 625-660.
[9] VINCENT P. Extracting and composing robust features with denoising auto-encoders[C]//International Conference on Machine Learning(ICML). Helsinki, Finland, 2008: 1096-1103.
[10] VINCENT P, LAROCHELLE H, LAJOIE I, et al. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion[J]. Machine learning research, 2010, 11: 3371-3408.
[11] CHEN M, XU Z, WINBERGER K Q, et al. Marginalized denoising autoencoders for domain adaptation[C]//International Conference on Machine Learning. Edinburgh, Scotland, 2012: 1206-1214.
[12] VINCENT P. A connection between score matching and denoising autoencoders[J]. Neural computation, 2011, 23(7): 1661-1674.
[13] RIFAI S. Contractive auto-encoders: explicit invariance during feature extraction[C]//Proceedings of the Twenty-eight International Conference on Machine Learning. Bellevue, USA, 2011: 833-840
[14] RIFAI S, BENGIO Y, DAUPHIN Y, et al. A generative process for sampling contractive auto-encoders[C]//Proc Int’l Conf Machine Learning. Edinburgh, Scotland, UK, 2012: 1206-1214.
[15] LECUN Y. Neural networks: tricks of the trade (2nd ed.) [M]. Germany: Springer, 2012: 9-48
[16] ZEILER M D, TAYLOR G W, FERGUS R. Adaptive deconvolutional networks for mid and high level feature learning[C]//IEEE International Conference on Computer Vision. Barcelona, Spain, 2011: 2013-2025.
Similar References:

Memo

-

Last Update: 2018-01-03

Copyright © CAAI Transactions on Intelligent Systems