[1]杨梦茵,陈俊芬,翟俊海.非对称卷积编码器的聚类算法[J].智能系统学报,2022,17(5):900-907.[doi:10.11992/tis.202107021]
YANG Mengyin,CHEN Junfen,ZHAI Junhai.A clustering method based on the asymmetric convolutional autoencoder[J].CAAI Transactions on Intelligent Systems,2022,17(5):900-907.[doi:10.11992/tis.202107021]
点击复制
YANG Mengyin,CHEN Junfen,ZHAI Junhai.A clustering method based on the asymmetric convolutional autoencoder[J].CAAI Transactions on Intelligent Systems,2022,17(5):900-907.[doi:10.11992/tis.202107021]
非对称卷积编码器的聚类算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第5期
页码:
900-907
栏目:
学术论文—机器学习
出版日期:
2022-09-05
- Title:
- A clustering method based on the asymmetric convolutional autoencoder
- Keywords:
- unsupervised; clustering; deep neural network; convolutional neural network; autoencoder; feature learning; feature representation; algorithm complexity
- 分类号:
- TP181
- 文献标志码:
- 2022-06-20
- 摘要:
- 基于深度神经网络的非监督学习方法通过联合优化特征表示和聚类指派，大大提升了聚类任务的性能。但大量的参数降低了运行速度，另外，深度模型提取的特征的区分能力也影响聚类性能。为此，提出一种新的聚类算法(asymmetric fully-connected layers convolutional auto-encoder, AFCAE)，其中卷积编码器结合非对称全连接进行无监督的特征提取，然后K-means算法对所得特征执行聚类。网络采用3×3和2×2的小卷积核，大大减少了参数个数，降低了算法复杂性。在MNIST上AFCAE获得0.960的聚类精度，比联合训练的DEC(deep embedding clustering)方法(0.840)提高了12个百分点。在6个图像数据集上实验结果表明AFCAE网络有优异的特征表示能力，能出色完成下游的聚类任务。
- Abstract:
- Unsupervised learning methods based on deep neural networks have synergistically optimized the feature representation and clustering assignment, thus greatly improving the clustering performance. However, numerous parameters slow down the running speed, and the discriminative ability of the features extracted by deep models also influences their clustering performance. To address these two issues, a new clustering algorithm is proposed (asymmetric fully-connected layers convolutional autoencoder, AFCAE), where a convolutional autoencoder combined with several asymmetric fully-connected layers is used to extract the features, and the K-means algorithm is subsequently applied to perform clustering on the obtained features. AFCAE adopts 3×3 and 2×2 convolutional kernels, thereby considerably reducing the number of parameters and the computational complexity. The clustering accuracy of AFCAE on MNIST reaches 0.960, almost 12% higher than that of the jointly trained DEC method (0.840). Experimental results on six image data sets show that the AFCAE network has excellent feature representation ability and can finish the subsequent clustering tasks well.
- 参考文献/References:
-
[1] LLOYD S. Least squares quantization in PCM[J]. IEEE transactions on information theory, 1982, 28(2): 129–137.
[2] HAEUSSER P, PLAPP J, GOLKOV V, et al. Associative deep clustering: training a classification network with no labels[C]//German Conference on Pattern Recognition. Cham: Springer, 2019: 18?32.
[3] REYNOLDS D. Gaussian mixture models[M]//Encyclopedia of Biometrics. Boston, MA: Springer US, 2009: 659?663.
[4] LIU Peng, ZHOU Dong, WU Naijun. VDBSCAN: varied density based spatial clustering of applications with noise[C]//2007 International Conference on Service Systems and Service Management. Chengdu, China. IEEE, 2007: 1?4.
[5] ABDI H, WILLIAMS L J. Principal component analysis[J]. Wiley interdisciplinary reviews:computational statistics, 2010, 2(4): 433–459.
[6] ALQAHTANI A, XIE X, DENG J, et al. A deep convolutional auto-encoder with embedded clustering[C]//2018 25th IEEE International Conference on Image Processing. Athens, Greece. IEEE, 2018: 4058?4062.
[7] YU Tianqi, WANG Xianbin, SHAMI A. UAV-enabled spatial data sampling in large-scale IoT systems using denoising autoencoder neural network[J]. IEEE Internet of things journal, 2019, 6(2): 1856–1865.
[8] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278–2324.
[9] MASCI J, MEIER U, CIRE?AN D, et al. Stacked convolutional auto-encoders for hierarchical feature extraction[M]//Lecture Notes in Computer Science. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011: 52?59.
[10] LEE Honglak, EKANADHAM C, NG A Y. Sparse deep belief net model for visual area V2[C]//Proc of Conf on Advances in Neural Information Processing Systems. Washington D. C. , USA: MIT Press, 2007: 873?880.
[11] VINCENT P, LAROCHELLE H, BENGIO Y, et al. Extracting and composing robust features with denoising autoencoders[C]//Proceedings of the 25th international conference on Machine learning-ICML ’08. Helsinki, Finland. New York: ACM Press, 2008: 1096?1103.
[12] BENGIO Y, LAMBLIN P, POPOVICI D, et al. Greedy layer-wise training of deep networks[C]//Proc of Advances in Neural Information Processing Systems. Washington, USA: MIT Press, 2006: 153?160.
[13] MA Xiaolei, DAI Zhuang, HE Zhengbing, et al. Learning traffic as images: a deep convolutional neural network for large-scale transportation network speed prediction[J]. Sensors, 2017, 17(4): 818.
[14] XIE Junyuan, ROSS G, ALI F. Unsupervised deep embedding for clustering analysis[C]//Proc of ICML’16 Proc of the 33rd Int Conf on Int Conf on Machine Learning. New York City, NY: Semantic Scholar, 2016: 478?487.
[15] GUO Xifeng, GAO Long, LIU Xinwang, et al. Improved deep embedded clustering with local structure preservation[C]//IJCAI’17: Proceedings of the 26th International Joint Conference on Artificial Intelligence. New York: ACM, 2017: 1753?1759.
[16] YANG Bo, FU Xiao, NICHOLAS D S, et al. Towards K-means-friendly spaces: simultaneous deep learning and clustering[C]//Proc of ICML’17 Proc of the 34th Int Conf on Machine Learning. Sydney, Australia: TonyJebara, 2016: 3861?3870.
[17] HUANG Peihao, HUANG Yan, WANG Wei, et al. Deep embedding network for clustering[C]//2014 22nd International Conference on Pattern Recognition. Stockholm, Sweden. IEEE, 2014: 1532?1537.
[18] LI Fengfu, QIAO Hong, ZHANG Bo. Discriminatively boosted image clustering with fully convolutional auto-encoders[J]. Pattern recognition, 2018, 83: 161–173.
[19] VAN L, MAATEN D, GEOFFREY H. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(2605): 2579–2605.
[20] 陈俊芬, 赵佳成, 韩洁, 等. 基于深度特征表示的Softmax聚类算法[J]. 南京大学学报(自然科学版), 2020, 56(4): 533–540
CHEN Junfen, ZHAO Jiacheng, HAN Jie, et al. Softmax clustering algorithm based on deep features representation[J]. Journal of Nanjing university (natural science edition), 2020, 56(4): 533–540
[21] HE Kaiming, SUN Jian. Convolutional neural networks at constrained time cost[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA. IEEE, 2015: 5353?5360.
[22] SONG Chunfeng, LIU Feng, HUANG Yongzhen, et al. Auto-encoder based Data clustering[C]//Iberoamerican Congress on Pattern Recognition. Berlin, Heidelberg: Springer, 2013: 117?124.
[23] LIU Hongfu, SHAO Ming, LI Sheng, et al. Infinite ensemble for image clustering[C]//KDD ’16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 1745?1754.
[24] YANG Jianwei, PARIKH D, BATRA D. Joint unsupervised learning of deep representations and image clusters[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA. IEEE, 2016: 5147?5156.
- 相似文献/References:
-
[1]杨小兵,何灵敏,孔繁胜.切换回归模型的抗噪音聚类算法[J].智能系统学报,2009,4(6):497.[doi:10.3969/j.issn.1673-4785.2009.06.005]
YANG Xiao-bing,HE Ling-min,KONG Fan-sheng.A noise-resistant clustering algorithm for switching regression models[J].CAAI Transactions on Intelligent Systems,2009,4():497.[doi:10.3969/j.issn.1673-4785.2009.06.005]
[2]季瑞瑞,刘?? 丁.支持向量数据描述的基因表达数据聚类方法[J].智能系统学报,2009,4(6):544.[doi:10.3969/j.issn.1673-4785.2009.06.013]
JI Rui-rui,LIU Ding.Improved gene expression data clustering using a support vector domain description algorithm[J].CAAI Transactions on Intelligent Systems,2009,4():544.[doi:10.3969/j.issn.1673-4785.2009.06.013]
[3]张秀玲,逄宗鹏,李少清,等.ANFIS的板形控制动态影响矩阵方法[J].智能系统学报,2010,5(4):360.
ZHANG Xiu-ling,PANG Zong-peng,LI Shao-qing,et al.A dynamic influence matrix method for flatness control based on adaptivenetworkbased fuzzy inference systems[J].CAAI Transactions on Intelligent Systems,2010,5():360.
[4]李伟,杨晓峰,张重阳,等.复杂网络社团的投影聚类划分[J].智能系统学报,2011,6(1):57.
LI Wei,YANG Xiaofeng,ZHANG Chongyang,et al.A clustering method for community detection on complex networks[J].CAAI Transactions on Intelligent Systems,2011,6():57.
[5]陈岳峰,苗夺谦,李文,等.基于概念的词汇情感倾向识别方法[J].智能系统学报,2011,6(6):489.
CHEN Yuefeng,MIAO Duoqian,LI Wen,et al.Semantic orientation computing based on concepts[J].CAAI Transactions on Intelligent Systems,2011,6():489.
[6]方然,苗夺谦,张志飞.一种基于情感的中文微博话题检测方法[J].智能系统学报,2013,8(3):208.
FANG Ran,MIAO Duoqian,ZHANG Zhifei.An emotion-based method of topic detection from Chinese microblogs[J].CAAI Transactions on Intelligent Systems,2013,8():208.
[7]刘恋,常冬霞,邓勇.动态小生境人工鱼群算法的图像分割[J].智能系统学报,2015,10(5):669.[doi:10.11992/tis.201501001]
LIU lian,CHANG Dongxia,DENG Yong.An image segmentation method based on dynamic niche artificial fish-swarm algorithm[J].CAAI Transactions on Intelligent Systems,2015,10():669.[doi:10.11992/tis.201501001]
[8]刘贝贝,马儒宁,丁军娣.基于密度的统计合并聚类算法[J].智能系统学报,2015,10(5):712.[doi:10.11992/tis.201410028]
LIU Beibei,MA Runing,DING Jundi.Density-based statistical merging clustering algorithm[J].CAAI Transactions on Intelligent Systems,2015,10():712.[doi:10.11992/tis.201410028]
[9]朱书伟,周治平,张道文.融合并行混沌萤火虫算法的K-调和均值聚类[J].智能系统学报,2015,10(6):872.[doi:10.11992/tis.201505043]
ZHU Shuwei,ZHOU Zhiping,ZHANG Daowen.K-harmonic means clustering merged with parallel chaotic firefly algorithm[J].CAAI Transactions on Intelligent Systems,2015,10():872.[doi:10.11992/tis.201505043]
[10]谷飞洋,田博,张思萌,等.基于置换检验的聚类结果评估[J].智能系统学报,2016,11(3):301.[doi:10.11992/tis.201603038]
GU Feiyang,TIAN Bo,ZHANG Simeng,et al.Statistical evaluation of the clustering results based on permutation test[J].CAAI Transactions on Intelligent Systems,2016,11():301.[doi:10.11992/tis.201603038]
[11]张智,毕晓君.基于风格转换的无监督聚类行人重识别[J].智能系统学报,2021,16(1):48.[doi:10.11992/tis.202012014]
ZHANG Zhi,BI Xiaojun.Clustering approach based on style transfer for unsupervised person re-identification[J].CAAI Transactions on Intelligent Systems,2021,16():48.[doi:10.11992/tis.202012014]
备注/Memo
收稿日期:2021-07-09。
基金项目:河北省引进留学人员资助项目(C20200302); 河北省机器学习与计算智能重点实验室自主立项项目（ZZ201909-202109-1）；河北省科技计划重点研发项目（19210310D）；河北省自然科学基金项目（F2021201020）；河北省社会科学基金项目（HB20TQ005）.
作者简介:杨梦茵，硕士研究生，主要研究方向为图像聚类和机器学习;陈俊芬，副教授，博士，CCF会员，主要研究方向为数据挖掘、机器学习和图像处理。主持河北省留学回国基金1项。发表学术论文10余篇;翟俊海，教授，博士生导师，博士，河北大学学术委员会委员，中国人工智能学会知识工程与分布智能专业委员会委员、粒计算与知识发现专业委员会委员，主要研究方向为大数据处理、机器学习、深度学习。主持省重点自然科学基金项目1项和省自然科学 基金项目2项，近3年发表学术论文 10余篇。
通讯作者:陈俊芬. E-mail: chenjunfen2010@126.com
更新日期/Last Update:
1900-01-01