[1]李旭,蔡彪,胡能兵.基于三元互信息的图对比学习方法研究[J].智能系统学报,2024,19(5):1257-1267.[doi:10.11992/tis.202308004]
LI Xu,CAI Biao,HU Nengbing.Research on graph contrastive learning method based on ternary mutual information[J].CAAI Transactions on Intelligent Systems,2024,19(5):1257-1267.[doi:10.11992/tis.202308004]
点击复制
LI Xu,CAI Biao,HU Nengbing.Research on graph contrastive learning method based on ternary mutual information[J].CAAI Transactions on Intelligent Systems,2024,19(5):1257-1267.[doi:10.11992/tis.202308004]
基于三元互信息的图对比学习方法研究
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第5期
页码:
1257-1267
栏目:
学术论文—人工智能基础
出版日期:
2024-09-05
- Title:
- Research on graph contrastive learning method based on ternary mutual information
- Keywords:
- graph contrastive learning; mutual information; graph neural network; unsupervised machine learning; contrastive learning; representation learning; node classification; deep learning
- 分类号:
- TP183
- 文献标志码:
- 2024-03-21
- 摘要:
- 最近，图对比学习成为一种成功的无监督图表示学习方法，大多数方法都基于最大化互信息原则，通过数据增强来得到两个视图，并最大化两个视图的互信息。然而，两个视图的互信息可能包含不利于下游任务的信息。为了克服这些缺陷，提出基于三元互信息的图对比学习框架。该框架首先对输入图进行随机数据增强来生成两个视图，使用权重共享的编码器获得两个节点表示矩阵，随后使用共享权重解码器解码两个视图的节点表示。通过对比损失函数分别计算视图之间和视图与原图之间的损失，以最大化视图之间和视图与原图之间的互信息。实验结果表明，该方法在节点分类准确性方面的表现优于基线方法，甚至超过部分监督学习方法，验证了框架的有效性。
- Abstract:
- Recently, graph contrastive learning has emerged as a successful method for unsupervised graph representation learning. Most existing methods are based on the principle of maximizing mutual information by obtaining two views through data augmentation and maximizing their mutual information. However, the mutual information of these two views may include information that is not beneficial for downstream tasks. To overcome these shortcomings, we propose a graph contrastive learning framework based on ternary mutual information. The framework first performs stochastic data augmentation on the input graph to generate two views. A weight-shared encoder is then used to obtain two node representation matrices. Subsequently, a shared weight decoder decodes the node representations of the two views. The loss between views, and between views and the original graph, is calculated separately using a contrast loss function. This approach maximizes the mutual information between the views, as well as between views and the original graph. The experimental results show that the method outperforms the baseline method in terms of node classification accuracy. Moreover, it even outperforms some supervised machine learning methods, verifying the effectiveness of the framework.
- 参考文献/References:
-
[1] CAI Biao, ZHU Xinping, QIN Yangxin. Parameters optimization of hybrid strategy recommendation based on particle swarm algorithm[J]. Expert systems with applications, 2021, 168: 114388.
[2] CAI Biao, YANG Xiaowang, HUANG Yusheng, et al. A triangular personalized recommendation algorithm for improving diversity[J]. Discrete dynamics in nature and society, 2018, 2018: 3162068.
[3] HU Fenyu, ZHU Yanqiao, WU Shu, et al. Hierarchical graph convolutional networks for semi-supervised node classification[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao: International Joint Conferences on Artificial Intelligence Organization, 2019: 4532–4539.
[4] KIPF T, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of the 5th International Conference on Learning Representations. Toulon: ICLR, 2017: 1–14.
[5] VELIKOVI P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]// Proceedings of the 6th International Conference on Learning Representations. Vancouver: ICLR, 2018: 1–12.
[6] LINSKER R. Self-organization in a perceptual network[J]. Computer, 1988, 21(3): 105–117.
[7] BACHMAN P, HJELM R D, BUCNWALTERuchwalter W. Learning representations by maximizing mutual information across views[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2019: 15535–15545.
[8] HE Kaiming, FAN Haoqi, WU Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 9726–9735.
[9] TIAN Yonglong, KRISHNAN D, ISOLA P. Contrastive multiview coding[C]//European Conference on Computer Vision. Cham: Springer, 2020: 776–794.
[10] COLLOBERT R, WESTON J. A unified architecture for natural language processing: deep neural networks with multitask learning[C]//Proceedings of the 25th International Conference on Machine Learning-ICML ’08. Helsinki: ACM, 2008: 160–167.
[11] MNIH A, KAVUKCUOGLU K. Learning word embeddings efficiently with noise-contrastive estimation[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2013: 2265–2273.
[12] VELIKOVI P, FEDUS W, HAMILTON W L, et al. Deep graph infomax[C]//Proceedings of the 7th International Conference on Learning Representations. New Orleans: ICLR, 2018: 1–17.
[13] HASSANI K, KHASAHMADI A H. Contrastive multi-view representation learning on graphs[C]//Proceedings of the 37th International Conference on Machine Learning. New York: Association for Computing Machinery, 2020: 4116–4126.
[14] ZHU Yanqiao, XU Yichen, YU Feng, et al. Deep graph contrastive representation learning[EB/OL]. (2020–06–07)[2023–08–03]. http://arxiv.org/abs/2006.04131.
[15] SONG Jingkuan, ZHANG Hanwang, LI Xiangpeng, et al. Self-supervised video hashing with hierarchical binary auto-encoder[J]. IEEE transactions on image processing: a publication of the IEEE signal processing society, 2018, 27(7): 3210–3221.
[16] XU Xing, LU Huimin, SONG Jingkuan, et al. Ternary adversarial networks with self-supervision for zero-shot cross-modal retrieval[J]. IEEE transactions on cybernetics, 2020, 50(6): 2400–2413.
[17] VAN DEN OORD A, LI Yazhe, VINYALS O. Representation learning with contrastive predictive coding[EB/OL]. (2018–07–10) [2023–08–03]. http://arxiv.org/abs/1807.03748.
[18] HJELM R D, FEDOROV A, LAVOIE-MARCHILDON S, et al. Learning deep representations by mutual information estimation and maximization[EB/OL]. (2018–08–20)[2023–08–03]. http://arxiv.org/abs/1808.06670.
[19] CHEN Ting, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ACM, 2020: 1597–1607.
[20] GRILL J B, STRUB F, ALTCHé F, et al. Bootstrap your own latent: A new approach to self-supervised learning[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2020: 21271–21284.
[21] ZBONTAR J, JING L, MISRA I, et al. Barlow Twins: self-supervised learning via redundancy reduction[C]//Proceedings of the 38th International Conference on Machine Learning. New York: Association for Computing Machinery, 2021: 1231012320.
[22] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2014: 701–710.
[23] GROVER A, LESKOVEC J. Node2Vec: scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco: ACM, 2016: 855–864.
[24] QIU Jiezhong, DONG Yuxiao, MA Hao, et al. Network embedding as matrix factorization: unifying deepwalk, line, pte, and node2vec[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. New York: Association for Computing Machinery, 2018: 459–467.
[25] HAMILTON W L, YING R, LESKOVC J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2017: 1025–1035.
[26] PENG Zhen, HUANG Wenbing, LUO Minnan, et al. Graph representation learning via graphical mutual information maximization[C]//Proceedings of The Web Conference 2020. New York: Association for Computing Machinery, 2020: 259–270.
[27] ZHU Yanqiao, XU Yichen, YU Feng, et al. Graph contrastive learning with adaptive augmentation[C]//Proceedings of the Web Conference 2021. New York: Association for Computing Machinery, 2021: 2069–2080.
[28] THAKOOR S, TALLEC C, AZAR M G, et al. Large-scale representation learning on graphs via bootstrapping[EB/OL]. (2021–02–12)[2023–08–03]. http://arxiv.org/abs/2102.06514.
[29] BIELAK P, KAJDANOWICZ T, CHAWLA N V. Graph barlow twins: a self-supervised representation learning framework for graphs[EB/OL]. (2021–06–04)[2023–08–03]. https://arxiv.org/abs/2106.02466.
[30] ZHANG Yifei, ZHU Hao, SONG Zixing, et al. COSTA: covariance-preserving feature augmentation for graph contrastive learning[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York: Association for Computing Machinery, 2022: 2524–2534.
[31] KINGMA D, BA J. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego: ICLR, 2015: 1–15.
[32] CLEVERT D A, UNTERTHINER T, HOCHREITER S. Fast and accurate deep network learning by exponential linear units (ELUs)[C]//Proceedings of the 4th International Conference on Learning Representations. San Juan: ICLR, 2016: 1–14.
[33] KIPF T N, WELLING M. Variational graph auto-encoders[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management. New York: Association for Computing Machinery, 2021: 2827–2831.
[34] MO Yujie, PENG Liang, XU Jie, et al. Simple unsupervised graph representation learning[C]//Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2022, 36(7): 7797–7805.
- 相似文献/References:
-
[1]陈伟卿,李冠华,欧宗瑛,等.基于灰度互信息和梯度相似性的医学图像配准及其加速处理[J].智能系统学报,2008,3(6):498.
CHEN Wei-qing,LI Guan-hua,OU Zong-ying,et al.Medical image registration based on grey mutual information and gradient similarity with an accelerated processing method[J].CAAI Transactions on Intelligent Systems,2008,3():498.
[2]李冰寒,高晓利,刘三阳,等.利用互信息学习贝叶斯网络结构[J].智能系统学报,2011,6(1):68.
LI-Binghan,GAO-Xiaoli,LIU-Sanyang,et al.Learning Bayesian network structures based on mutual information[J].CAAI Transactions on Intelligent Systems,2011,6():68.
[3]张昭昭,乔俊飞,杨刚.自适应前馈神经网络结构优化设计[J].智能系统学报,2011,6(4):312.
ZHANG Zhaozhao,QIAO Junfei,YANG Gang.An adaptive algorithm for designingoptimal feedforward neural network architecture[J].CAAI Transactions on Intelligent Systems,2011,6():312.
[4]宋晓丽,刘冀伟,张晓星.分布式视频编码的关键帧提取算法[J].智能系统学报,2011,6(6):539.
SONG Xiaoli,LIU Jiwei,ZHANG Xiaoxing.A key frame selection algorithm for distributed video coding[J].CAAI Transactions on Intelligent Systems,2011,6():539.
[5]周红标,乔俊飞.基于高维k-近邻互信息的特征选择方法[J].智能系统学报,2017,12(5):595.[doi:10.11992/tis.201609020]
ZHOU Hongbiao,QIAO Junfei.Feature selection method based on high dimensional k-nearest neighbors mutual information[J].CAAI Transactions on Intelligent Systems,2017,12():595.[doi:10.11992/tis.201609020]
[6]王贺彬,葛泉波,刘华平,等.面向观测融合和吸引因子的多机器人主动SLAM[J].智能系统学报,2021,16(2):371.[doi:10.11992/tis.202006019]
WANG Hebin,GE Quanbo,LIU Huaping,et al.Multi-robot active SLAM for observation fusion and attractor[J].CAAI Transactions on Intelligent Systems,2021,16():371.[doi:10.11992/tis.202006019]
[7]郑静,熊伟丽.基于互信息的多块k近邻故障监测及诊断[J].智能系统学报,2021,16(4):717.[doi:10.11992/tis.202007035]
ZHENG Jing,XIONG Weili.Multiblock k-nearest neighbor fault monitoring and diagnosis based on mutual information[J].CAAI Transactions on Intelligent Systems,2021,16():717.[doi:10.11992/tis.202007035]
[8]曲海成,王宇萍,谢梦婷,等.结合亮度感知与密集卷积的红外与可见光图像融合[J].智能系统学报,2022,17(3):643.[doi:10.11992/tis.202104004]
QU Haicheng,WANG Yuping,XIE Mengting,et al.Infrared and visible image fusion combined with brightness perception and dense convolution[J].CAAI Transactions on Intelligent Systems,2022,17():643.[doi:10.11992/tis.202104004]
[9]张智慧,杨燕,张熠玲.面向不完整多视图聚类的深度互信息最大化方法[J].智能系统学报,2023,18(1):12.[doi:10.11992/tis.202203051]
ZHANG Zhihui,YANG Yan,ZHANG Yiling.Deep mutual information maximization method for incomplete multi-view clustering[J].CAAI Transactions on Intelligent Systems,2023,18():12.[doi:10.11992/tis.202203051]
[10]卢毅,陈亚冉,赵冬斌,等.关键点图对比图像分类方法[J].智能系统学报,2023,18(1):36.[doi:10.11992/tis.202112001]
LU Yi,CHEN Yaran,ZHAO Dongbin,et al.Keypoint-based graph contrastive neural network for image classification[J].CAAI Transactions on Intelligent Systems,2023,18():36.[doi:10.11992/tis.202112001]
备注/Memo
收稿日期:2023-8-3。
基金项目:国家自然科学基金项目（61802034）.
作者简介:李旭，硕士研究生，主要研究方向为图神经网络、图对比学习。E-mail：905376942@qq.com;蔡彪，教授，博士，主要研究方向为数据挖掘、图神经网络、图像分割。负责四川省科技厅项目1项、教育部重点实验室基金项目1项；作为第一完成人主持国家自然基金项目和四川省科技攻关项目各1项。以第一作者和通信作者发表学术论文20余篇。 E-mail：caibiao@cdut.edu.cn;胡能兵，硕士研究生，主要研究方向为图神经网络、图对比学习。E-mail：1575843213@qq.com
通讯作者:蔡彪. E-mail：caibiao@cdut.edu.cn
更新日期/Last Update:
2024-09-05