[1]孟祥福,温晶,李子函,等.多重注意力指导下的异构图嵌入方法[J].智能系统学报,2023,18(4):688-698.[doi:10.11992/tis.202204006]
MENG Xiangfu,WEN Jing,LI Zihan,et al.Heterogeneous graph embedding method guided by the multi-attention mechanism[J].CAAI Transactions on Intelligent Systems,2023,18(4):688-698.[doi:10.11992/tis.202204006]
点击复制
MENG Xiangfu,WEN Jing,LI Zihan,et al.Heterogeneous graph embedding method guided by the multi-attention mechanism[J].CAAI Transactions on Intelligent Systems,2023,18(4):688-698.[doi:10.11992/tis.202204006]
多重注意力指导下的异构图嵌入方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第4期
页码:
688-698
栏目:
学术论文—机器学习
出版日期:
2023-07-15
- Title:
- Heterogeneous graph embedding method guided by the multi-attention mechanism
- Keywords:
- heterogeneous information network; graph representation learning; heterogeneous graph embedding; metapath; metapath instance; graph attention; heterogeneous graph; graph neural network
- 分类号:
- TP18
- 摘要:
- 现有的异构图嵌入学习方法存在两个方面的问题，一是没有考虑不同节点属性间的深层联系，二是通过注意力机制聚合邻居节点来生成目标节点的向量表示，忽略了目标节点的特征在向量表示中起的作用。为解决上述问题，本文提出了一种多重注意力指导下的异构图神经网络，从点?线?网3个角度学习异构节点嵌入向量。使用双向长短期记忆模型(bidirectional long short - term memory networks, Bi-LSTM)挖掘不同节点的属性间的深层关联关系并将其映射到同一向量空间，利用级联网络对单条元路径实例上的邻居节点和目标节点的特征信息进行融合，从而增强嵌入向量对目标节点信息的表达能力，提出一种多重注意力机制来聚合多条元路径实例上的节点信息，生成最终的节点嵌入向量表示。在3个大型异构图上的实验结果表明，本文提出的模型在异构图嵌入的效果方面优于现有基线模型，并且对于增强节点属性信息上的表达展现出了良好的性能。
- Abstract:
- There are two problems in the existing heterogeneous graph embedding learning methods. One is that the deep relationship between different node attributes is not considered, the other is the problem of ignorance of the role of the features of the target node in the vector representation when generating the vector representation of the target node by aggregating neighboring nodes through attention mechanism. In order to solve above problems, this paper proposes a heterogeneous graph neural network under the guidance of multiple attentions, which learns the embedding vectors of heterogeneous nodes from three perspectives of Point-Line-Net. Bi-LSTM is used to mine the deep relationship between attributes of different nodes and map them to the same vector space. A cascaded network is used to fuse the feature information of neighbor nodes and target nodes on a single meta-path instance, so as to enhance the expression ability of embedded vectors to target node information. A multi-attention mechanism is proposed to aggregate node information on multiple meta-path instances and generate the final node embedding vector representation. Experimental results on three large heterogeneous graphs show that the proposed model is superior to the existing baseline model in the embedding effect of heterogeneous graphs, and shows good performance in enhancing the expression of node attribute information.
- 参考文献/References:
-
[1] BILò D, FRIEDRICH T, LENZNER P, et al. Selfish creation of social networks[C]//The 35th AAAI Conference on Artificial Intelligence. Online: AAAI, 2021: 5185-5193.
[2] WANG Daixin, CUI Peng, ZHU Wenwu. Structural deep network embedding[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 1225?1234.
[3] GAO Yuan, LI Yi, SUN Yunchuan, et al. IEEE access special section: privacy preservation for large-scale user data in social networks[J]. IEEE access, 2022, 10: 4374–4379.
[4] SHEN Jiayi, WANG Haotao, GUI Shupeng, et al. UMEC: Unified model and embedding compression for efficient recommendation systems[C]//International Conference on Learning Representations. Addis Ababa:Ithaca, 2020:1-13.
[5] HAN Xiaotian, SHI Chuan, WANG Senzhang, et al. Aspect-level deep collaborative filtering via heterogeneous information networks[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. New York: ACM, 2018: 3393?3399.
[6] HU Binbin, SHI Chuan, ZHAO W X, et al. Leveraging meta-path based context for top-N recommendation with a neural co-attention model[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2018: 1531?1540.
[7] SHI Chuan, LI Yitong, ZHANG Jiawei, et al. A survey of heterogeneous information network analysis[J]. IEEE transactions on knowledge and data engineering, 2017, 29(1): 17–37.
[8] ZHANG Yizhou, XIONG Yun, KONG Xiangnan, et al. Deep collective classification in heterogeneous information networks[C]//Proceedings of the 2018 World Wide Web Conference. New York: ACM, 2018: 399?408.
[9] CHEN Mingren, HUANG Ping, LIN Yu, et al. SSNE: effective node representation for link prediction in sparse networks[J]. IEEE access, 2021, 9: 57874–57885.
[10] TANG Fengqin, WANG Chunning, SU Jinxia, et al. Spectral clustering-based community detection using graph distance and node attributes[J]. Computational statistics, 2020, 35(1): 69–94.
[11] REN Xiang, LIU Jialu, YU Xiao, et al. ClusCite: effective citation recommendation by information network-based clustering[C]//Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. New York: ACM, 2014: 821?830.
[12] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. New York: ACM, 2014: 701?710.
[13] MIKOLOV T, CHEN KAI, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. (2013-01-16)[2022-04-04].https://arxiv.org/abs/1301.3781.
[14] GROVER A, LESKOVEC J. Node2vec: scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 855?864.
[15] SHANG Jinbo, QU Meng, LIU Jialu, et al. Meta-path guided embedding for similarity search in large-scale heterogeneous information networks[EB/OL].(2016?10?31)[2022?04?04]. https://arxiv.org/abs/1610.09769.
[16] DONG Yuxiao, CHAWLA N V, SWAMI A. Metapath2vec: scalable representation learning for heterogeneous networks[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2017: 135?144.
[17] WANG Xiao, JI Houye, SHI Chuan, et al. Heterogeneous graph attention network[C]//The World Wide Web Conference. New York: ACM, 2019: 2022–2032.
[18] FU Xinyu, ZHANG Jiani, MENG Ziqiao, et al. MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding[C]//Proceedings of The Web Conference 2020. New York: ACM, 2020: 2331?2341.
[19] HONG Huiting, GUO Hantao, LIN Yucheng, et al. An attention-based graph neural network for heterogeneous structural learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020: 4132-4139.
[20] ZHANG Chuxu, SONG Dongjin, HUANG Chao, et al. Heterogeneous graph neural network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2019: 793?803.
[21] CEN Yukuo, ZOU Xu, ZHANG Jianwei, et al. Representation learning for attributed multiplex heterogeneous network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2019: 1358?1368.
[22] LI Yujia, TARLOW D, BROCKSCHMIDT M, et al. Gated graph sequence neural networks[C]//The 4th International Conference on Learning Representations, Caribe Hilton, USA, 2016.
[23] SCARSELLI F, GORI M, TSOI A C, et al. The graph neural network model[J]. IEEE transactions on neural networks, 2009, 20(1): 61–80.
[24] GORI M, MONFARDINI G, SCARSELLI F. A new model for learning in graph domains[C]//Proceedings of 2005 IEEE International Joint Conference on Neural Networks. Piscataway: IEEE, 2005: 729?734.
[25] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. (2016?09?09)[2022?04?04]. https://arxiv.org/abs/1609.02907.
[26] HAMILTON W L, YING R, LESKOVEC J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 1025?1035.
[27] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. (2017?10?31)[2022?04?04]. https://arxiv.org/abs/1710.10903.
[28] ZHANG Jiani, SHI Xingjian, XIE Junyuan, et al. Gaan: gated attention networks for learning on large and spatiotemporal graphs[C]//The Conference on Uncertainty in Artificial Intelligence. California: PMLR, 2018: 339?349 .
[29] SHI Chuan, HU Binbin, ZHAO W X, et al. Heterogeneous information network embedding for recommendation[J]. IEEE transactions on knowledge and data engineering, 2019, 31(2): 357–370.
[30] ZHOU Xiaohan, YI Yunhui, JIA Geng. Path-RotatE: knowledge graph embedding by relational rotation of path in complex space[C]// IEEE/CIC International Conference on Communications in China. Piscataway: IEEE, 2021: 905?910.
[31] YANG Yaming, GUAN Ziyu, LI Jianxin, et al. Interpretable and efficient heterogeneous graph convolutional network[J]. IEEE transactions on knowledge and data engineering, 2023, 35(2): 1637–1650.
[32] PARK H, LEE S, HWANG D, et al. Learning augmentation for GNNs with consistency regularization[J]. IEEE access, 2021, 9: 127961–127972.
[33] YU Yue, CHEN Jie, GAO Tian, et al. DAG-GNN: DAG structure learning with graph neural networks[EB/OL]. (2019?04?22)[2022?04?04]. https://arxiv.org/abs/1904.10098.
[34] CHEN Zhengdao, VILLAR S, CHEN Lei, et al. On the equivalence between graph isomorphism testing and function approximation with GNNs[EB/OL]. (2019?05?29)[2022?04?04]. https://arxiv.org/abs/1905.12560.
[35] LIMA M, SILVA R, LOPES DE SOUZA MENDES F, et al. Inferring about fraudulent collusion risk on Brazilian public works contracts in official texts using a Bi-LSTM approach[C]//Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg: Association for Computational Linguistics, 2020: 1580?1588.
[36] TANG Jian, QU Meng, WANG Mingzhe, et al. LINE: large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web. New York: ACM, 2015: 1067?1077.
[37] YU Xie, CHENG Chen, GONG Maoguo, et al. Graph embedding via multi-scale graph representations[J]. Information sciences, 2021, 578: 102?115.
[38] LEE Mengyuan, YU Guanding, LI G Y. Graph embedding-based wireless link scheduling with few training samples[J]. IEEE transactions on wireless communications, 2021, 20(4): 2282–2294.
[39] VANDER M L, HINTON G. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(11): 2579–2605.
备注/Memo
收稿日期:2022-04-04。
基金项目:国家自然科学基金项目（61772249）；辽宁省教育厅项目（LJKZ0355）.
作者简介:孟祥福，教授，博士，主要研究方向为空间关键字查询、大数据分析与可视化、机器学习和推荐系统。主持国家自然科学基金项目20项，获授权发明专利1项。出版专著1部，发表学术论文50余篇;温晶，硕士研究生, 主要研究方向为图表示学习、图数据挖掘;李子函，硕士研究生，主要研究方向为知识图谱、图表示学习
通讯作者:孟祥福.E-mail:marxi@126.com
更新日期/Last Update:
1900-01-01