[1]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(6):894-898.[doi:10.11992/tis.201706037]
　YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12(6):894-898.[doi:10.11992/tis.201706037]
点击复制

基于自编码器的特征迁移算法

《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷: 12 期数: 2017年第6期 页码: 894-898 栏目: 学术论文—智能系统 出版日期: 2017-12-25
Title:
Feature transfer algorithm based on an auto-encoder
作者:
杨梦铎1, 栾咏红1, 刘文军1, 李凡长2
1. 苏州工业职业技术学院 软件与服务外包学院, 江苏 苏州 215104;
2. 苏州大学 计算机科学与技术学院, 江苏 苏州 215006
Author(s):
YANG Mengduo1, LUAN Yonghong1, LIU Wenjun1, LI Fanzhang2
1. Department of Software and Service Outsourcing, Suzhou Vocational Institute of Industrial Technology, Suzhou 215104, China;
2. School of Computer Science and Technology, Soochow University, Suzhou 215006, China
关键词:
自编码器; 特征迁移; 深度网络; 深度学习; 图像分类; 中级图像特征; 视觉识别; 大规模数据集
Keywords:
auto-encoder; feature transfer; deep network; deep learning; image classification; mid-level image representation; visual recognition; large-scale datasets
分类号:
TP181
DOI:
10.11992/tis.201706037
摘要:
近年来，栈式自编码网络（stacked auto-encoder，SAE）在大规模数据集上表现出优异的图像分类性能。相对于其他图像分类方法中手工设计的低级特征，SAE的成功归因于深度网络能够学习到丰富的中级图像特征。然而，估计上百万个网络参数需要非常庞大的带标签的图像样本数据集。这样的性质阻止了SAE在小规模训练数据上的许多应用。在这篇文章中，提出的算法展示如何将SAE在大规模数据集上学习到的图像表示有效地迁移到只有有限训练数据的视觉识别任务中。实验部分设计了一个方法来复用在MNIST数据集上训练得到的隐藏层，以此计算在MNIST-variations数据集上的中级图像表示。实验结果展示了尽管两个数据集之间存在差异，但是被迁移的图像特征能够使得模型的分类性能得到极大的提升。
Abstract:
The stacked auto-encoder (SAE) has recently shown outstanding image classification performance in large-scale datasets. Relative to the low-level features of artificial design in other image classification methods, the success of SAE is its deep network that can learn rich mid-level image features. However, estimating millions of parameters requires a very large number of annotated image samples, and this prevents many SAE applications to small-scale training data. In this paper, the proposed algorithm shows how to efficiently transfer image representation learned by SAE on a large-scale dataset to other visual recognition tasks with limited training data. In the experimental section, a method is designed to reuse the hidden layers trained on MNIST datasets to compute the mid-level image representation of MNIST-variation datasets. Experimental results show that, despite differences in image datasets, the transferred image features can significantly improve the classification performance of the model.
参考文献/References:
[1] HINTON G E, ZEMEL R S. Autoencoder minimum description length and helmholtz free energy[C]//Conference on Neural Information Processing Systems(NIPS). Denver, USA, 1993: 3-10.
[2] SOCHER R, HUANG E H, PENNINGTON J, et al. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection[C]//Proc Neural Information and Processing Systems. Granada, Spain, 2011: 801-809.
[3] SWERSKY K, RANZATO M, BUCHMAN D, et al. On score matching for energy based models: generalizing autoencoders and simplifying deep learning[C]//Proc Int’l Conf Machine Learning Bellevue. Washington, USA, 2011: 1201-1208.
[4] FISCHER A, IGEL C. An introduction to restricted Boltzmann machines[C]//Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. Guadalajara, Mexico, 2012: 14-36.
[5] BREULEUX O, BENGIO Y, VINCENT P. Quickly generating representative samples from an RBM-derived process[J]. Neural computation, 2011, 23(8): 2053-2073.
[6] COURVILLE A, BERGSTRA J, BENGIO Y. Unsupervised models of images by spike-and-slab RBMs[C]//Proc Int’l Conf Machine Learning. Bellevue, Washington, USA, 2011: 1145-1152.
[7] SCHMAH T, HINTON G E, ZEMEL R, et al. Generative versus discriminative training of RBMs for classification of fMRI images[C]//Proc Neural Information and Processing Systems. Vancouver, Canada, 2008: 1409-1416.
[8] ERHAN D, BENGIO Y, COURVILLE A, et al. Why does unsupervised pre-training help deep learning?[J]. Machine learning research, 2010, 11: 625-660.
[9] VINCENT P. Extracting and composing robust features with denoising auto-encoders[C]//International Conference on Machine Learning(ICML). Helsinki, Finland, 2008: 1096-1103.
[10] VINCENT P, LAROCHELLE H, LAJOIE I, et al. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion[J]. Machine learning research, 2010, 11: 3371-3408.
[11] CHEN M, XU Z, WINBERGER K Q, et al. Marginalized denoising autoencoders for domain adaptation[C]//International Conference on Machine Learning. Edinburgh, Scotland, 2012: 1206-1214.
[12] VINCENT P. A connection between score matching and denoising autoencoders[J]. Neural computation, 2011, 23(7): 1661-1674.
[13] RIFAI S. Contractive auto-encoders: explicit invariance during feature extraction[C]//Proceedings of the Twenty-eight International Conference on Machine Learning. Bellevue, USA, 2011: 833-840
[14] RIFAI S, BENGIO Y, DAUPHIN Y, et al. A generative process for sampling contractive auto-encoders[C]//Proc Int’l Conf Machine Learning. Edinburgh, Scotland, UK, 2012: 1206-1214.
[15] LECUN Y. Neural networks: tricks of the trade (2nd ed.) [M]. Germany: Springer, 2012: 9-48
[16] ZEILER M D, TAYLOR G W, FERGUS R. Adaptive deconvolutional networks for mid and high level feature learning[C]//IEEE International Conference on Computer Vision. Barcelona, Spain, 2011: 2013-2025.
相似文献/References:
[1]莫凌飞,蒋红亮,李煊鹏.基于深度学习的视频预测研究综述[J].智能系统学报,2018,13(1):85.[doi:10.11992/tis.201707032]
　MO Lingfei,JIANG Hongliang,LI Xuanpeng.Review of deep learning-based video prediction[J].CAAI Transactions on Intelligent Systems,2018,13():85.[doi:10.11992/tis.201707032]
[2]杨文元.多标记学习自编码网络无监督维数约简[J].智能系统学报,2018,13(5):808.[doi:10.11992/tis.201804051]
　YANG Wenyuan.Unsupervised dimensionality reduction of multi-label learning via autoencoder networks[J].CAAI Transactions on Intelligent Systems,2018,13():808.[doi:10.11992/tis.201804051]
[3]张潇鲲,刘琰,陈静.引入外部词向量的文本信息网络表示学习[J].智能系统学报,2019,14(5):1056.[doi:10.11992/tis.201809037]
　ZHANG Xiaokun,LIU Yan,CHEN Jing.Representation learning using network embedding based on external word vectors[J].CAAI Transactions on Intelligent Systems,2019,14():1056.[doi:10.11992/tis.201809037]
[4]王丽娟,丁世飞.一种基于ELM-AE特征表示的谱聚类算法[J].智能系统学报,2021,16(3):560.[doi:10.11992/tis.202005021]
　WANG Lijuan,DING Shifei.A spectral clustering algorithm based on ELM-AE feature representation[J].CAAI Transactions on Intelligent Systems,2021,16():560.[doi:10.11992/tis.202005021]
[5]马睿,周治平.结合地标点与自编码的快速多视图聚类网络[J].智能系统学报,2022,17(2):333.[doi:10.11992/tis.202101011]
　MA Rui,ZHOU Zhiping.Fast multiview clustering network combining landmark points and autoencoder[J].CAAI Transactions on Intelligent Systems,2022,17():333.[doi:10.11992/tis.202101011]
[6]吴晓东,熊伟丽.采用编码输入的生成对抗网络故障检测方法及应用[J].智能系统学报,2022,17(3):496.[doi:10.11992/tis.202102003]
　WU Xiaodong,XIONG Weili.Fault detection method and its application using GAN with an encoded input[J].CAAI Transactions on Intelligent Systems,2022,17():496.[doi:10.11992/tis.202102003]

备注/Memo

收稿日期:2017-06-10;改回日期:。
基金项目:国家自然科学基金项目（61672364）.
作者简介:杨梦铎,女,1989年生,讲师,博士,主要研究方向为模式识别与机器学习;栾咏红,女,1968年生,副教授,主要研究方向为强化学习;刘文军,男,1981年生,讲师,博士,主要研究方向为无线传感网络与算法分析。
通讯作者:杨梦铎.E-mail:mengduoyang@163.com.

更新日期/Last Update: 2018-01-03
Copyright @ 《 智能系统学报》 编辑部
地址：（150001）黑龙江省哈尔滨市南岗区南通大街145-1号楼 电话：0451- 82534001、82518134