[1]ZHANG Zhihui,YANG Yan,ZHANG Yiling.Deep mutual information maximization method for incomplete multi-view clustering[J].CAAI Transactions on Intelligent Systems,2023,18(1):12-22.[doi:10.11992/tis.202203051]
Copy

Deep mutual information maximization method for incomplete multi-view clustering

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 1 Page number: 12-22 Column: 学术论文—机器学习 Public date: 2023-01-05
Title:
Deep mutual information maximization method for incomplete multi-view clustering
Author(s):
ZHANG Zhihui;  YANG Yan;  ZHANG Yiling
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China
Keywords:
data mining; clustering; incomplete multi-view clustering; multi-view representation learning; deep learning; autoencoder; mutual information; self-paced learning
CLC:
TP391
DOI:
10.11992/tis.202203051
Abstract:
Multi-view clustering is a research hotspot in the field of unsupervised learning. Of the many excellent multi-view clustering studies that have recently arisen, most assume that each view is complete. However, in a real scene, the data are extremely easily missed in the collection process, resulting in partially incomplete views. Simultaneously, many methods use traditional machine learning, i.e., the shallow-layer model, to learn data features, which makes it difficult for the model to mine the complex information of high-dimensional data. To solve these problems, in this paper, a novel deep mutual information maximization method is proposed for incomplete multi-view clustering. First, a deep autoencoder is used to learn the rich complex information of each view, and the knowledge of consistency among views is learned by the mutual information between potential representations. Then, the missing data are fixed up by the common latent representation of multi-view data. Additionally, this paper uses a self-paced strategy to fine-tune the model as it learns the samples from easy to difficult, obtaining a more clustering-friendly representation. Experiments performed on several real datasets show the effectiveness of our proposed method.
References:
[1] 赵博宇, 张长青, 陈蕾, 等. 生成式不完整多视图数据聚类[J]. 自动化学报, 2021, 47(8): 1867–1875
ZHAO Boyu, ZHANG Changqing, CHEN Lei, et al. Generative model for partial multi-view clustering[J]. Acta automatica sinica, 2021, 47(8): 1867–1875
[2] 张熠玲, 杨燕, 周威, 等. CMvSC: 知识迁移下的深度一致性多视图谱聚类网络[J]. 软件学报, 2022, 33(4): 1373–1389
ZHANG Yiling, YANG Yan, ZHOU Wei, et al. CMvSC: knowledge transferring based deep consensus network for multi-view spectral clustering[J]. Journal of software, 2022, 33(4): 1373–1389
[3] SHAO Weixiang, HE Lifang, YU P S. Multiple incomplete views clustering via weighted nonnegative matrix factorization with L2, 1 regularization[M]//Machine Learning and Knowledge Discovery in Databases. Cham: Springer International Publishing, 2015: 318?334.
[4] YIN Qiyue, WU Shu, WANG Liang. Incomplete multi-view clustering via subspace learning[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. Melbourne: ACM, 2015: 383?392.
[5] WANG HAO, ZONG LINLIN, LIU BING, et al. Spectral perturbation meets incomplete multi-view data[EB/OL]. (2019?03?31)[2022?03?23].https://arxiv.org/abs/1906.00098.
[6] HU MENGLEI, CHEN SONGCAN. Doubly aligned incomplete multi-view clustering[EB/OL]. (2019?03?07)[2022?03?23].https://arxiv.org/abs/1903.02785.
[7] LI Shaoyuan, JIANG Yuan, ZHOU Zhihua. Partial multi-view clustering[C]//Proceedings of the AAAI conference on artificial intelligence. Québec: AAAI Press, 2014, 28(1): 1968?1974.
[8] HUANG Zhenyu, ZHOU Tianyi, PENG Xi, et al. Multi-view spectral clustering network[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao: International Joint Conferences on Artificial Intelligence Organization, 2019: 2563?2569.
[9] SUN Xiukun, CHENG Miaomiao, CHEN Min, et al. Self-supervised deep multi-view subspace clustering [C]//Proceedings of the 8th Asian Conference on Machine Learning. Nagoya: PMLR, 2019: 1001?1016.
[10] ZHANG Changqing, FU Huazhu, HU Qinghua, et al. Generalized latent multi-view subspace clustering[J]. IEEE transactions on pattern analysis and machine intelligence, 2020, 42(1): 86–99.
[11] ZHANG Changqing, LIU Yeqing, FU Huazhu. AE2-nets: autoencoder in autoencoder networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 2572?2580.
[12] ZHANG Changqing, CUI Yajie, HAN Zongbo, et al. Deep partial multi-view learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 44(5): 2402–2415.
[13] PAN Erlin, ZHAO Kang. Multi-view Contrastive Graph Clustering[J]. Advances in neural information processing systems, 2021, 34: 2148–2159.
[14] WAN Zhibin, ZHANG Changqing, ZHU Pengfei, et al. Multi-view information-bottleneck representation learning[C]//Proceedings of the AAAI conference on artificial intelligence. Van couver: AAAI Press, 2021, 35(11): 10085?10092.
[15] SHAHAM U, STANTON K, LI H, et al. Spectralnet: Spectral clustering using deep neural networks[EB/OL]. (2018?01?04) [2022?03?23].https://arxiv.org/abs/18012021.
[16] PAN Ji, ZHANG Tong, LI Hongdong, et al. Deep subspace clustering networks[J]. Advances in neural information processing systems, 2017: 30.
[17] ZHONG HUASONG, CHEN CHONG, JIN ZHONGMING, et al. Deep robust clustering by contrastive learning[EB/OL]. (2020?08?07)[2022?03?23].https://arxiv.org/abs/2008.03030.
[18] MACQUEEN J. Classification and analysis of multivariate observations[C]//Fifth Berkeley Symposium on Mathematical Statistics and Probability. Berkeley: The Regents of the University of California, 1967: 281?297.
[19] KUMAR M P, PACKER B, KOLLER D. Self-paced learning for latent variable models[C]// Advances in neural information processing systems. NIPS, 2010.
[20] BENGIO Y, LOURADOUR J, COLLOBERT R, et al. Curriculum learning[C]//ICML’09: Proceedings of the 26th Annual International Conference on Machine Learning. New York: ACM, 2009: 41?48.
[21] HU Menglei, CHEN Songcan. One-pass incomplete multi-view clustering[C]//Proceedings of the AAAI conference on artificial intelligence. Honolulu, AAAI Press: 2019, 33: 3838?3845.
[22] ZHAO Liang, CHEN Zhikui, YANG Yi, et al. Incomplete multi-view clustering via deep semantic mapping[J]. Neurocomputing, 2018, 275: 1053–1062.
[23] 刘相男, 丁世飞, 王丽娟. 基于深度图正则化矩阵分解的多视图聚类算法[J]. 智能系统学报, 2022, 17(1): 158–169
LIU Xiangnan, DING Shifei, WANG Lijuan. A multi-view clustering algorithm based on deep matrix factorization with graph regularization[J]. CAAI transactions on intelligent systems, 2022, 17(1): 158–169
[24] 郭圣, 仲兆满, 李存华. 基于深度自编码的多视图子空间聚类网络[J]. 计算机工程与应用, 2020, 56(17): 60–68
GUO Sheng, ZHONG Zhaoman, LI Cunhua. Multi-view subspace clustering network based on deep autoencoder[J]. Computer engineering and applications, 2020, 56(17): 60–68
[25] LIN Yijie, GOU Yuanbiao, LIU Zitao, et al. COMPLETER: incomplete multi-view clustering via contrastive prediction[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 11169?11178.
[26] GE Yixiao, ZHU Feng, CHEN Dapeng, et al. Self-paced contrastive learning with hybrid memory for domain adaptive object re-id[J]. Advances in neural information processing systems, 2020, 33: 11309?11321.
[27] JIANG Lu, MENG Deyu, ZHAO Qian, et al. Self-paced curriculum learning[C]//Proceedings of the AAAI conference on artificial intelligence. Buenos Aires: AAAI Press, 2015, 29(1).
[28] CHU Guanyi, WANG Xiao, SHI Chuan, et al. CuCo: graph representation with curriculum contrastive learning[C]//Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 2021: 2300?2306.
[29] PENG Jizong, WANG Ping, DESROSIERS C, et al. Self-paced contrastive learning for semi-supervised medical image segmentation with meta-labels[J]. Advances in neural information processing systems, 2021, 34: 16686–16699.
[30] 许子微, 陈秀宏. 自步稀疏最优均值主成分分析[J]. 智能系统学报, 2021, 16(3): 416–424
XU Ziwei, CHEN Xiuhong. Sparse optimal mean principal component analysis based on self-paced learning[J]. CAAI transactions on intelligent systems, 2021, 16(3): 416–424
[31] WANG Weiran, RAMAN Arora, LIVESCU, K, et al. On deep multi-view representation learning[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille: PMLR, 2015: 1083?1092.
[32] ANDREW G, ARORA R, BILMES J, et al. Deep canonical correlation analysis[C]//ICML’13: Proceedings of the 30th International Conference on International Conference on Machine Learning-Volume 28. New York: ACM, 2013: 1247–1255.
[33] ZHANG Zheng, LIU Li, SHEN Fumin, et al. Binary multi-view clustering[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(7): 1774–1782.
[34] LIU Xinwang, LI Miaomiao, TANG Chang, et al. Efficient and effective regularized incomplete multi-view clustering[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(8): 2634–2646.
[35] ZHANG Handong, LIU Hongfu, FU Yun. Incomplete multi-modal visual data grouping[C]// International Joint Conference on Artificial Intelligence. New York: AAAI, 2016: 2392?2398.
[36] WEN Jie, ZHANG Zheng, XU Yong, et al. Unified embedding alignment with missing views inferring for incomplete multi-view clustering[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Honolulu: AAAI, 2019: 5393?5400.
[37] VAN der Maaten L, HINTON G. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(11): 2579?2605.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems