[1]张智慧,杨燕,张熠玲.面向不完整多视图聚类的深度互信息最大化方法[J].智能系统学报,2023,18(1):12-22.[doi:10.11992/tis.202203051]
ZHANG Zhihui,YANG Yan,ZHANG Yiling.Deep mutual information maximization method for incomplete multi-view clustering[J].CAAI Transactions on Intelligent Systems,2023,18(1):12-22.[doi:10.11992/tis.202203051]
点击复制
ZHANG Zhihui,YANG Yan,ZHANG Yiling.Deep mutual information maximization method for incomplete multi-view clustering[J].CAAI Transactions on Intelligent Systems,2023,18(1):12-22.[doi:10.11992/tis.202203051]
面向不完整多视图聚类的深度互信息最大化方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第1期
页码:
12-22
栏目:
学术论文—机器学习
出版日期:
2023-01-05
- Title:
- Deep mutual information maximization method for incomplete multi-view clustering
- Keywords:
- data mining; clustering; incomplete multi-view clustering; multi-view representation learning; deep learning; autoencoder; mutual information; self-paced learning
- 分类号:
- TP391
- 摘要:
- 多视图聚类是无监督学习领域研究热点之一，近年来涌现出许多优秀的多视图聚类工作，但其中大多数方法均假设各视图是完整的，然而真实场景下数据收集过程极容易发生缺失，造成部分视图不完整。同时，很多方法采取传统机器学习方法（即浅层模型）对数据进行特征学习，这导致模型难以挖掘高维数据内的复杂信息。针对以上问题，本文提出一种面向不完整多视图聚类的深度互信息最大化方法。首先利用深度自编码器挖掘各视图深层次的隐含特征，并通过最大化潜在表示间的互信息来学习各视图间的一致性知识。然后，对于不完整视图中的缺失数据，利用多视图的公共潜在表示进行补全。此外，本文采用一种自步学习策略对网络进行微调，从易到难地学习数据集中的样本，得到更加宜于聚类的特征表示。最后，在多个真实数据集上进行实验，验证了本文方法的有效性。
- Abstract:
- Multi-view clustering is a research hotspot in the field of unsupervised learning. Of the many excellent multi-view clustering studies that have recently arisen, most assume that each view is complete. However, in a real scene, the data are extremely easily missed in the collection process, resulting in partially incomplete views. Simultaneously, many methods use traditional machine learning, i.e., the shallow-layer model, to learn data features, which makes it difficult for the model to mine the complex information of high-dimensional data. To solve these problems, in this paper, a novel deep mutual information maximization method is proposed for incomplete multi-view clustering. First, a deep autoencoder is used to learn the rich complex information of each view, and the knowledge of consistency among views is learned by the mutual information between potential representations. Then, the missing data are fixed up by the common latent representation of multi-view data. Additionally, this paper uses a self-paced strategy to fine-tune the model as it learns the samples from easy to difficult, obtaining a more clustering-friendly representation. Experiments performed on several real datasets show the effectiveness of our proposed method.
- 参考文献/References:
-
[1] 赵博宇, 张长青, 陈蕾, 等. 生成式不完整多视图数据聚类[J]. 自动化学报, 2021, 47(8): 1867–1875
ZHAO Boyu, ZHANG Changqing, CHEN Lei, et al. Generative model for partial multi-view clustering[J]. Acta automatica sinica, 2021, 47(8): 1867–1875
[2] 张熠玲, 杨燕, 周威, 等. CMvSC: 知识迁移下的深度一致性多视图谱聚类网络[J]. 软件学报, 2022, 33(4): 1373–1389
ZHANG Yiling, YANG Yan, ZHOU Wei, et al. CMvSC: knowledge transferring based deep consensus network for multi-view spectral clustering[J]. Journal of software, 2022, 33(4): 1373–1389
[3] SHAO Weixiang, HE Lifang, YU P S. Multiple incomplete views clustering via weighted nonnegative matrix factorization with L2, 1 regularization[M]//Machine Learning and Knowledge Discovery in Databases. Cham: Springer International Publishing, 2015: 318?334.
[4] YIN Qiyue, WU Shu, WANG Liang. Incomplete multi-view clustering via subspace learning[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. Melbourne: ACM, 2015: 383?392.
[5] WANG HAO, ZONG LINLIN, LIU BING, et al. Spectral perturbation meets incomplete multi-view data[EB/OL]. (2019?03?31)[2022?03?23].https://arxiv.org/abs/1906.00098.
[6] HU MENGLEI, CHEN SONGCAN. Doubly aligned incomplete multi-view clustering[EB/OL]. (2019?03?07)[2022?03?23].https://arxiv.org/abs/1903.02785.
[7] LI Shaoyuan, JIANG Yuan, ZHOU Zhihua. Partial multi-view clustering[C]//Proceedings of the AAAI conference on artificial intelligence. Québec: AAAI Press, 2014, 28(1): 1968?1974.
[8] HUANG Zhenyu, ZHOU Tianyi, PENG Xi, et al. Multi-view spectral clustering network[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao: International Joint Conferences on Artificial Intelligence Organization, 2019: 2563?2569.
[9] SUN Xiukun, CHENG Miaomiao, CHEN Min, et al. Self-supervised deep multi-view subspace clustering [C]//Proceedings of the 8th Asian Conference on Machine Learning. Nagoya: PMLR, 2019: 1001?1016.
[10] ZHANG Changqing, FU Huazhu, HU Qinghua, et al. Generalized latent multi-view subspace clustering[J]. IEEE transactions on pattern analysis and machine intelligence, 2020, 42(1): 86–99.
[11] ZHANG Changqing, LIU Yeqing, FU Huazhu. AE2-nets: autoencoder in autoencoder networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 2572?2580.
[12] ZHANG Changqing, CUI Yajie, HAN Zongbo, et al. Deep partial multi-view learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 44(5): 2402–2415.
[13] PAN Erlin, ZHAO Kang. Multi-view Contrastive Graph Clustering[J]. Advances in neural information processing systems, 2021, 34: 2148–2159.
[14] WAN Zhibin, ZHANG Changqing, ZHU Pengfei, et al. Multi-view information-bottleneck representation learning[C]//Proceedings of the AAAI conference on artificial intelligence. Van couver: AAAI Press, 2021, 35(11): 10085?10092.
[15] SHAHAM U, STANTON K, LI H, et al. Spectralnet: Spectral clustering using deep neural networks[EB/OL]. (2018?01?04) [2022?03?23].https://arxiv.org/abs/18012021.
[16] PAN Ji, ZHANG Tong, LI Hongdong, et al. Deep subspace clustering networks[J]. Advances in neural information processing systems, 2017: 30.
[17] ZHONG HUASONG, CHEN CHONG, JIN ZHONGMING, et al. Deep robust clustering by contrastive learning[EB/OL]. (2020?08?07)[2022?03?23].https://arxiv.org/abs/2008.03030.
[18] MACQUEEN J. Classification and analysis of multivariate observations[C]//Fifth Berkeley Symposium on Mathematical Statistics and Probability. Berkeley: The Regents of the University of California, 1967: 281?297.
[19] KUMAR M P, PACKER B, KOLLER D. Self-paced learning for latent variable models[C]// Advances in neural information processing systems. NIPS, 2010.
[20] BENGIO Y, LOURADOUR J, COLLOBERT R, et al. Curriculum learning[C]//ICML’09: Proceedings of the 26th Annual International Conference on Machine Learning. New York: ACM, 2009: 41?48.
[21] HU Menglei, CHEN Songcan. One-pass incomplete multi-view clustering[C]//Proceedings of the AAAI conference on artificial intelligence. Honolulu, AAAI Press: 2019, 33: 3838?3845.
[22] ZHAO Liang, CHEN Zhikui, YANG Yi, et al. Incomplete multi-view clustering via deep semantic mapping[J]. Neurocomputing, 2018, 275: 1053–1062.
[23] 刘相男, 丁世飞, 王丽娟. 基于深度图正则化矩阵分解的多视图聚类算法[J]. 智能系统学报, 2022, 17(1): 158–169
LIU Xiangnan, DING Shifei, WANG Lijuan. A multi-view clustering algorithm based on deep matrix factorization with graph regularization[J]. CAAI transactions on intelligent systems, 2022, 17(1): 158–169
[24] 郭圣, 仲兆满, 李存华. 基于深度自编码的多视图子空间聚类网络[J]. 计算机工程与应用, 2020, 56(17): 60–68
GUO Sheng, ZHONG Zhaoman, LI Cunhua. Multi-view subspace clustering network based on deep autoencoder[J]. Computer engineering and applications, 2020, 56(17): 60–68
[25] LIN Yijie, GOU Yuanbiao, LIU Zitao, et al. COMPLETER: incomplete multi-view clustering via contrastive prediction[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 11169?11178.
[26] GE Yixiao, ZHU Feng, CHEN Dapeng, et al. Self-paced contrastive learning with hybrid memory for domain adaptive object re-id[J]. Advances in neural information processing systems, 2020, 33: 11309?11321.
[27] JIANG Lu, MENG Deyu, ZHAO Qian, et al. Self-paced curriculum learning[C]//Proceedings of the AAAI conference on artificial intelligence. Buenos Aires: AAAI Press, 2015, 29(1).
[28] CHU Guanyi, WANG Xiao, SHI Chuan, et al. CuCo: graph representation with curriculum contrastive learning[C]//Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 2021: 2300?2306.
[29] PENG Jizong, WANG Ping, DESROSIERS C, et al. Self-paced contrastive learning for semi-supervised medical image segmentation with meta-labels[J]. Advances in neural information processing systems, 2021, 34: 16686–16699.
[30] 许子微, 陈秀宏. 自步稀疏最优均值主成分分析[J]. 智能系统学报, 2021, 16(3): 416–424
XU Ziwei, CHEN Xiuhong. Sparse optimal mean principal component analysis based on self-paced learning[J]. CAAI transactions on intelligent systems, 2021, 16(3): 416–424
[31] WANG Weiran, RAMAN Arora, LIVESCU, K, et al. On deep multi-view representation learning[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille: PMLR, 2015: 1083?1092.
[32] ANDREW G, ARORA R, BILMES J, et al. Deep canonical correlation analysis[C]//ICML’13: Proceedings of the 30th International Conference on International Conference on Machine Learning-Volume 28. New York: ACM, 2013: 1247–1255.
[33] ZHANG Zheng, LIU Li, SHEN Fumin, et al. Binary multi-view clustering[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(7): 1774–1782.
[34] LIU Xinwang, LI Miaomiao, TANG Chang, et al. Efficient and effective regularized incomplete multi-view clustering[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(8): 2634–2646.
[35] ZHANG Handong, LIU Hongfu, FU Yun. Incomplete multi-modal visual data grouping[C]// International Joint Conference on Artificial Intelligence. New York: AAAI, 2016: 2392?2398.
[36] WEN Jie, ZHANG Zheng, XU Yong, et al. Unified embedding alignment with missing views inferring for incomplete multi-view clustering[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Honolulu: AAAI, 2019: 5393?5400.
[37] VAN der Maaten L, HINTON G. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(11): 2579?2605.
- 相似文献/References:
-
[1]张继福,张素兰,胡立华.约束概念格及其构造方法[J].智能系统学报,2006,1(2):31.
ZHANG Ji-fu,ZHANG Su-lan,HU Li-hua.Constrained concept lattice and its construction method[J].CAAI Transactions on Intelligent Systems,2006,1():31.
[2]王国胤,张清华,胡? 军.粒计算研究综述[J].智能系统学报,2007,2(6):8.
WANG Guo-yin,ZHANG Qing-hua,HU Jun.An overview of granular computing[J].CAAI Transactions on Intelligent Systems,2007,2():8.
[3]杨小兵,何灵敏,孔繁胜.切换回归模型的抗噪音聚类算法[J].智能系统学报,2009,4(6):497.[doi:10.3969/j.issn.1673-4785.2009.06.005]
YANG Xiao-bing,HE Ling-min,KONG Fan-sheng.A noise-resistant clustering algorithm for switching regression models[J].CAAI Transactions on Intelligent Systems,2009,4():497.[doi:10.3969/j.issn.1673-4785.2009.06.005]
[4]季瑞瑞,刘?? 丁.支持向量数据描述的基因表达数据聚类方法[J].智能系统学报,2009,4(6):544.[doi:10.3969/j.issn.1673-4785.2009.06.013]
JI Rui-rui,LIU Ding.Improved gene expression data clustering using a support vector domain description algorithm[J].CAAI Transactions on Intelligent Systems,2009,4():544.[doi:10.3969/j.issn.1673-4785.2009.06.013]
[5]张秀玲,逄宗鹏,李少清,等.ANFIS的板形控制动态影响矩阵方法[J].智能系统学报,2010,5(4):360.
ZHANG Xiu-ling,PANG Zong-peng,LI Shao-qing,et al.A dynamic influence matrix method for flatness control based on adaptivenetworkbased fuzzy inference systems[J].CAAI Transactions on Intelligent Systems,2010,5():360.
[6]李伟,杨晓峰,张重阳,等.复杂网络社团的投影聚类划分[J].智能系统学报,2011,6(1):57.
LI Wei,YANG Xiaofeng,ZHANG Chongyang,et al.A clustering method for community detection on complex networks[J].CAAI Transactions on Intelligent Systems,2011,6():57.
[7]陈岳峰,苗夺谦,李文,等.基于概念的词汇情感倾向识别方法[J].智能系统学报,2011,6(6):489.
CHEN Yuefeng,MIAO Duoqian,LI Wen,et al.Semantic orientation computing based on concepts[J].CAAI Transactions on Intelligent Systems,2011,6():489.
[8]何清.物联网与数据挖掘云服务[J].智能系统学报,2012,7(3):189.
HE Qing.The Internet of things and the data mining cloud service[J].CAAI Transactions on Intelligent Systems,2012,7():189.
[9]方然,苗夺谦,张志飞.一种基于情感的中文微博话题检测方法[J].智能系统学报,2013,8(3):208.
FANG Ran,MIAO Duoqian,ZHANG Zhifei.An emotion-based method of topic detection from Chinese microblogs[J].CAAI Transactions on Intelligent Systems,2013,8():208.
[10]李海林,郭韧,万校基.基于特征矩阵的多元时间序列最小距离度量方法[J].智能系统学报,2015,10(3):442.[doi:10.3969/j.issn.1673-4785.201405047]
LI Hailin,GUO Ren,WAN Xiaoji.A minimum distance measurement method for amultivariate time series based on the feature matrix[J].CAAI Transactions on Intelligent Systems,2015,10():442.[doi:10.3969/j.issn.1673-4785.201405047]
[11]刘大莲,田英杰.可拓数据挖掘在学生成绩分析中的应用研究[J].智能系统学报,2022,17(4):707.[doi:10.11992/tis.202112020]
LIU Dalian,TIAN Yingjie.Application of extension data mining in student achievement analysis[J].CAAI Transactions on Intelligent Systems,2022,17():707.[doi:10.11992/tis.202112020]
备注/Memo
收稿日期:2022-03-24。
基金项目:国家自然科学基金项目(61976247).
作者简介:张智慧,硕士研究生,主要研究方向为数据挖掘和多视图聚类;杨燕,教授,博士生导师,博士,四川省学术和技术带头人,CCF杰出会员,主要研究方向为人工智能、大数据分析与挖掘、集成学习、多视图学习、聚类分析和时空挖掘。主持承担国家自然科学基金等科技项目10余项。发表学术论文230余篇,入选2021年度中国百篇最具国际影响力学术论文1篇;张熠玲,博士,主要研究方向为多视图学习、多任务学习、聚类分析和时空挖掘
通讯作者:杨燕.E-mail:yyang@swjtu.edu.cn
更新日期/Last Update:
1900-01-01