[1]MENG Xiangfu,WEN Jing,LI Zihan,et al.Heterogeneous graph embedding method guided by the multi-attention mechanism[J].CAAI Transactions on Intelligent Systems,2023,18(4):688-698.[doi:10.11992/tis.202204006]
Copy

Heterogeneous graph embedding method guided by the multi-attention mechanism

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 4 Page number: 688-698 Column: 学术论文—机器学习 Public date: 2023-07-15
Title:
Heterogeneous graph embedding method guided by the multi-attention mechanism
Author(s):
MENG Xiangfu;  WEN Jing;  LI Zihan;  JI Hongzhang
School of Electronic and Information Engineering, Liaoning Technical University, Huludao 125105, China
Keywords:
heterogeneous information network; graph representation learning; heterogeneous graph embedding; metapath; metapath instance; graph attention; heterogeneous graph; graph neural network
CLC:
TP18
DOI:
10.11992/tis.202204006
Abstract:
There are two problems in the existing heterogeneous graph embedding learning methods. One is that the deep relationship between different node attributes is not considered, the other is the problem of ignorance of the role of the features of the target node in the vector representation when generating the vector representation of the target node by aggregating neighboring nodes through attention mechanism. In order to solve above problems, this paper proposes a heterogeneous graph neural network under the guidance of multiple attentions, which learns the embedding vectors of heterogeneous nodes from three perspectives of Point-Line-Net. Bi-LSTM is used to mine the deep relationship between attributes of different nodes and map them to the same vector space. A cascaded network is used to fuse the feature information of neighbor nodes and target nodes on a single meta-path instance, so as to enhance the expression ability of embedded vectors to target node information. A multi-attention mechanism is proposed to aggregate node information on multiple meta-path instances and generate the final node embedding vector representation. Experimental results on three large heterogeneous graphs show that the proposed model is superior to the existing baseline model in the embedding effect of heterogeneous graphs, and shows good performance in enhancing the expression of node attribute information.
References:
[1] BILò D, FRIEDRICH T, LENZNER P, et al. Selfish creation of social networks[C]//The 35th AAAI Conference on Artificial Intelligence. Online: AAAI, 2021: 5185-5193.
[2] WANG Daixin, CUI Peng, ZHU Wenwu. Structural deep network embedding[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 1225?1234.
[3] GAO Yuan, LI Yi, SUN Yunchuan, et al. IEEE access special section: privacy preservation for large-scale user data in social networks[J]. IEEE access, 2022, 10: 4374–4379.
[4] SHEN Jiayi, WANG Haotao, GUI Shupeng, et al. UMEC: Unified model and embedding compression for efficient recommendation systems[C]//International Conference on Learning Representations. Addis Ababa:Ithaca, 2020:1-13.
[5] HAN Xiaotian, SHI Chuan, WANG Senzhang, et al. Aspect-level deep collaborative filtering via heterogeneous information networks[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. New York: ACM, 2018: 3393?3399.
[6] HU Binbin, SHI Chuan, ZHAO W X, et al. Leveraging meta-path based context for top-N recommendation with a neural co-attention model[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2018: 1531?1540.
[7] SHI Chuan, LI Yitong, ZHANG Jiawei, et al. A survey of heterogeneous information network analysis[J]. IEEE transactions on knowledge and data engineering, 2017, 29(1): 17–37.
[8] ZHANG Yizhou, XIONG Yun, KONG Xiangnan, et al. Deep collective classification in heterogeneous information networks[C]//Proceedings of the 2018 World Wide Web Conference. New York: ACM, 2018: 399?408.
[9] CHEN Mingren, HUANG Ping, LIN Yu, et al. SSNE: effective node representation for link prediction in sparse networks[J]. IEEE access, 2021, 9: 57874–57885.
[10] TANG Fengqin, WANG Chunning, SU Jinxia, et al. Spectral clustering-based community detection using graph distance and node attributes[J]. Computational statistics, 2020, 35(1): 69–94.
[11] REN Xiang, LIU Jialu, YU Xiao, et al. ClusCite: effective citation recommendation by information network-based clustering[C]//Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. New York: ACM, 2014: 821?830.
[12] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. New York: ACM, 2014: 701?710.
[13] MIKOLOV T, CHEN KAI, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. (2013-01-16)[2022-04-04].https://arxiv.org/abs/1301.3781.
[14] GROVER A, LESKOVEC J. Node2vec: scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 855?864.
[15] SHANG Jinbo, QU Meng, LIU Jialu, et al. Meta-path guided embedding for similarity search in large-scale heterogeneous information networks[EB/OL].(2016?10?31)[2022?04?04]. https://arxiv.org/abs/1610.09769.
[16] DONG Yuxiao, CHAWLA N V, SWAMI A. Metapath2vec: scalable representation learning for heterogeneous networks[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2017: 135?144.
[17] WANG Xiao, JI Houye, SHI Chuan, et al. Heterogeneous graph attention network[C]//The World Wide Web Conference. New York: ACM, 2019: 2022–2032.
[18] FU Xinyu, ZHANG Jiani, MENG Ziqiao, et al. MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding[C]//Proceedings of The Web Conference 2020. New York: ACM, 2020: 2331?2341.
[19] HONG Huiting, GUO Hantao, LIN Yucheng, et al. An attention-based graph neural network for heterogeneous structural learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020: 4132-4139.
[20] ZHANG Chuxu, SONG Dongjin, HUANG Chao, et al. Heterogeneous graph neural network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2019: 793?803.
[21] CEN Yukuo, ZOU Xu, ZHANG Jianwei, et al. Representation learning for attributed multiplex heterogeneous network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2019: 1358?1368.
[22] LI Yujia, TARLOW D, BROCKSCHMIDT M, et al. Gated graph sequence neural networks[C]//The 4th International Conference on Learning Representations, Caribe Hilton, USA, 2016.
[23] SCARSELLI F, GORI M, TSOI A C, et al. The graph neural network model[J]. IEEE transactions on neural networks, 2009, 20(1): 61–80.
[24] GORI M, MONFARDINI G, SCARSELLI F. A new model for learning in graph domains[C]//Proceedings of 2005 IEEE International Joint Conference on Neural Networks. Piscataway: IEEE, 2005: 729?734.
[25] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. (2016?09?09)[2022?04?04]. https://arxiv.org/abs/1609.02907.
[26] HAMILTON W L, YING R, LESKOVEC J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 1025?1035.
[27] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. (2017?10?31)[2022?04?04]. https://arxiv.org/abs/1710.10903.
[28] ZHANG Jiani, SHI Xingjian, XIE Junyuan, et al. Gaan: gated attention networks for learning on large and spatiotemporal graphs[C]//The Conference on Uncertainty in Artificial Intelligence. California: PMLR, 2018: 339?349 .
[29] SHI Chuan, HU Binbin, ZHAO W X, et al. Heterogeneous information network embedding for recommendation[J]. IEEE transactions on knowledge and data engineering, 2019, 31(2): 357–370.
[30] ZHOU Xiaohan, YI Yunhui, JIA Geng. Path-RotatE: knowledge graph embedding by relational rotation of path in complex space[C]// IEEE/CIC International Conference on Communications in China. Piscataway: IEEE, 2021: 905?910.
[31] YANG Yaming, GUAN Ziyu, LI Jianxin, et al. Interpretable and efficient heterogeneous graph convolutional network[J]. IEEE transactions on knowledge and data engineering, 2023, 35(2): 1637–1650.
[32] PARK H, LEE S, HWANG D, et al. Learning augmentation for GNNs with consistency regularization[J]. IEEE access, 2021, 9: 127961–127972.
[33] YU Yue, CHEN Jie, GAO Tian, et al. DAG-GNN: DAG structure learning with graph neural networks[EB/OL]. (2019?04?22)[2022?04?04]. https://arxiv.org/abs/1904.10098.
[34] CHEN Zhengdao, VILLAR S, CHEN Lei, et al. On the equivalence between graph isomorphism testing and function approximation with GNNs[EB/OL]. (2019?05?29)[2022?04?04]. https://arxiv.org/abs/1905.12560.
[35] LIMA M, SILVA R, LOPES DE SOUZA MENDES F, et al. Inferring about fraudulent collusion risk on Brazilian public works contracts in official texts using a Bi-LSTM approach[C]//Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg: Association for Computational Linguistics, 2020: 1580?1588.
[36] TANG Jian, QU Meng, WANG Mingzhe, et al. LINE: large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web. New York: ACM, 2015: 1067?1077.
[37] YU Xie, CHENG Chen, GONG Maoguo, et al. Graph embedding via multi-scale graph representations[J]. Information sciences, 2021, 578: 102?115.
[38] LEE Mengyuan, YU Guanding, LI G Y. Graph embedding-based wireless link scheduling with few training samples[J]. IEEE transactions on wireless communications, 2021, 20(4): 2282–2294.
[39] VANDER M L, HINTON G. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(11): 2579–2605.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems