[1]YANG Mengyin,CHEN Junfen,ZHAI Junhai.A clustering method based on the asymmetric convolutional autoencoder[J].CAAI Transactions on Intelligent Systems,2022,17(5):900-907.[doi:10.11992/tis.202107021]
Copy

A clustering method based on the asymmetric convolutional autoencoder

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 17 Number of periods: 2022 5 Page number: 900-907 Column: 学术论文—机器学习 Public date: 2022-09-05
Title:
A clustering method based on the asymmetric convolutional autoencoder
Author(s):
YANG Mengyin1; 2;  CHEN Junfen1; 2;  ZHAI Junhai1; 2
1. College of Mathematics and Information Science, Hebei University, Baoding 071002, China;
2. Hebei Key Laboratory of Machine Learning and Computational Intelligence, Baoding 071002, China
Keywords:
unsupervised; clustering; deep neural network; convolutional neural network; autoencoder; feature learning; feature representation; algorithm complexity
CLC:
TP181
DOI:
10.11992/tis.202107021
Abstract:
Unsupervised learning methods based on deep neural networks have synergistically optimized the feature representation and clustering assignment, thus greatly improving the clustering performance. However, numerous parameters slow down the running speed, and the discriminative ability of the features extracted by deep models also influences their clustering performance. To address these two issues, a new clustering algorithm is proposed (asymmetric fully-connected layers convolutional autoencoder, AFCAE), where a convolutional autoencoder combined with several asymmetric fully-connected layers is used to extract the features, and the K-means algorithm is subsequently applied to perform clustering on the obtained features. AFCAE adopts 3×3 and 2×2 convolutional kernels, thereby considerably reducing the number of parameters and the computational complexity. The clustering accuracy of AFCAE on MNIST reaches 0.960, almost 12% higher than that of the jointly trained DEC method (0.840). Experimental results on six image data sets show that the AFCAE network has excellent feature representation ability and can finish the subsequent clustering tasks well.
References:
[1] LLOYD S. Least squares quantization in PCM[J]. IEEE transactions on information theory, 1982, 28(2): 129–137.
[2] HAEUSSER P, PLAPP J, GOLKOV V, et al. Associative deep clustering: training a classification network with no labels[C]//German Conference on Pattern Recognition. Cham: Springer, 2019: 18?32.
[3] REYNOLDS D. Gaussian mixture models[M]//Encyclopedia of Biometrics. Boston, MA: Springer US, 2009: 659?663.
[4] LIU Peng, ZHOU Dong, WU Naijun. VDBSCAN: varied density based spatial clustering of applications with noise[C]//2007 International Conference on Service Systems and Service Management. Chengdu, China. IEEE, 2007: 1?4.
[5] ABDI H, WILLIAMS L J. Principal component analysis[J]. Wiley interdisciplinary reviews:computational statistics, 2010, 2(4): 433–459.
[6] ALQAHTANI A, XIE X, DENG J, et al. A deep convolutional auto-encoder with embedded clustering[C]//2018 25th IEEE International Conference on Image Processing. Athens, Greece. IEEE, 2018: 4058?4062.
[7] YU Tianqi, WANG Xianbin, SHAMI A. UAV-enabled spatial data sampling in large-scale IoT systems using denoising autoencoder neural network[J]. IEEE Internet of things journal, 2019, 6(2): 1856–1865.
[8] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278–2324.
[9] MASCI J, MEIER U, CIRE?AN D, et al. Stacked convolutional auto-encoders for hierarchical feature extraction[M]//Lecture Notes in Computer Science. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011: 52?59.
[10] LEE Honglak, EKANADHAM C, NG A Y. Sparse deep belief net model for visual area V2[C]//Proc of Conf on Advances in Neural Information Processing Systems. Washington D. C. , USA: MIT Press, 2007: 873?880.
[11] VINCENT P, LAROCHELLE H, BENGIO Y, et al. Extracting and composing robust features with denoising autoencoders[C]//Proceedings of the 25th international conference on Machine learning-ICML ’08. Helsinki, Finland. New York: ACM Press, 2008: 1096?1103.
[12] BENGIO Y, LAMBLIN P, POPOVICI D, et al. Greedy layer-wise training of deep networks[C]//Proc of Advances in Neural Information Processing Systems. Washington, USA: MIT Press, 2006: 153?160.
[13] MA Xiaolei, DAI Zhuang, HE Zhengbing, et al. Learning traffic as images: a deep convolutional neural network for large-scale transportation network speed prediction[J]. Sensors, 2017, 17(4): 818.
[14] XIE Junyuan, ROSS G, ALI F. Unsupervised deep embedding for clustering analysis[C]//Proc of ICML’16 Proc of the 33rd Int Conf on Int Conf on Machine Learning. New York City, NY: Semantic Scholar, 2016: 478?487.
[15] GUO Xifeng, GAO Long, LIU Xinwang, et al. Improved deep embedded clustering with local structure preservation[C]//IJCAI’17: Proceedings of the 26th International Joint Conference on Artificial Intelligence. New York: ACM, 2017: 1753?1759.
[16] YANG Bo, FU Xiao, NICHOLAS D S, et al. Towards K-means-friendly spaces: simultaneous deep learning and clustering[C]//Proc of ICML’17 Proc of the 34th Int Conf on Machine Learning. Sydney, Australia: TonyJebara, 2016: 3861?3870.
[17] HUANG Peihao, HUANG Yan, WANG Wei, et al. Deep embedding network for clustering[C]//2014 22nd International Conference on Pattern Recognition. Stockholm, Sweden. IEEE, 2014: 1532?1537.
[18] LI Fengfu, QIAO Hong, ZHANG Bo. Discriminatively boosted image clustering with fully convolutional auto-encoders[J]. Pattern recognition, 2018, 83: 161–173.
[19] VAN L, MAATEN D, GEOFFREY H. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(2605): 2579–2605.
[20] 陈俊芬, 赵佳成, 韩洁, 等. 基于深度特征表示的Softmax聚类算法[J]. 南京大学学报(自然科学版), 2020, 56(4): 533–540
CHEN Junfen, ZHAO Jiacheng, HAN Jie, et al. Softmax clustering algorithm based on deep features representation[J]. Journal of Nanjing university (natural science edition), 2020, 56(4): 533–540
[21] HE Kaiming, SUN Jian. Convolutional neural networks at constrained time cost[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA. IEEE, 2015: 5353?5360.
[22] SONG Chunfeng, LIU Feng, HUANG Yongzhen, et al. Auto-encoder based Data clustering[C]//Iberoamerican Congress on Pattern Recognition. Berlin, Heidelberg: Springer, 2013: 117?124.
[23] LIU Hongfu, SHAO Ming, LI Sheng, et al. Infinite ensemble for image clustering[C]//KDD ’16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 1745?1754.
[24] YANG Jianwei, PARIKH D, BATRA D. Joint unsupervised learning of deep representations and image clusters[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA. IEEE, 2016: 5147?5156.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems