[1]ZHANG Xiaokun,LIU Yan,CHEN Jing.Representation learning using network embedding based on external word vectors[J].CAAI Transactions on Intelligent Systems,2019,14(5):1056-1063.[doi:10.11992/tis.201809037]
Copy

Representation learning using network embedding based on external word vectors

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 14 Number of periods: 2019 5 Page number: 1056-1063 Column: 学术论文—自然语言处理与理解 Public date: 2019-09-05
Title:
Representation learning using network embedding based on external word vectors
Author(s):
ZHANG Xiaokun;  LIU Yan;  CHEN Jing
Mathematical Engineering and Advanced Computing State Key Laboratory, Zhengzhou 450000
Keywords:
network embedding; content information network; auto-encoder; external word vectors; vertex classification; word vectors; distributed representation; representation learning
CLC:
TP181
DOI:
10.11992/tis.201809037
Abstract:
Network embedding, which preserves a network’s sophisticated features, can effectively learn the low-dimensional embedding of vertices in order to lower the computing and storage costs. Content information networks (such as Twitter), which contain rich text information, are commonly used in daily life. Most studies on content information network are based on the information of the network itself. Distributed word vectors are becoming increasingly popular in natural language processing tasks. As a low-dimensional representation of the semantic feature space, word vectors can preserve syntactic and semantic regularities. By introducing external word vectors into the modeling process, we can use the external syntactic and semantic features. Hence, in this paper, we propose network embedding based on external word vectors (NE-EWV), whereby the feature fusion representation is learned from both semantic feature space as well as structural feature space. Empirical experiments were conducted using real-world content information network datasets to validate the effectiveness of the model. The results show that in link prediction task, the AUC of the model was 7% to 19% higher than that of the model that considers only the structural features, and in most cases was 1% to 12% higher than the model that considers structural and text features. In node classification tasks, the performance is comparable with that of context-aware network embedding (CANE), which was the state-of-the-art baseline model.
References:
[1] CUI Peng, WANG Xiao, PEI Jian, et al. A survey on network embedding[J]. IEEE transactions on knowledge and data engineering, 2019, 31(5):833-852.
[2] LIBEN NOWELL D, KLEINBERG J. The link prediction problem for social networks[J]. Journal of the American society for information science and technology, 2007, 58(7):1019-1031.
[3] LANCICHINETTI A, FORTUNATO S. Community detection algorithms:a comparative analysis[J]. Physical review E, 2009, 80:056117.
[4] BHAGAT S, CORMODE G, MUTHUKRISHNAN S. Node classification in social networks[M]//AGGARWAL C C. Social Network Data Analytics. Boston, MA:Springer, 2011:115-148.
[5] DONG Xin, HALEVY A, MADHAVAN J, et al. Similarity search for web services[C]//Proceedings of the Thirtieth International Conference on Very Large Data Bases. Toronto, Canada, 2004:372-383.
[6] BASTIAN M, HEYMANN S, JACOMY M. Gephi:an open source software for exploring and manipulating networks[C]//Proceedings of International AAAI Conference on Weblogs and Social Media. San Jose, California, USA, 2009:361-362.
[7] PEROZZI B, AL-RFOU R, SKIENA S. Deepwalk:online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA, 2014:701-710.
[8] TANG Jian, QU Meng, WANG Mingzhe, et al. LINE:large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web. Florence, Italy, 2005:1067-1077.
[9] GROVER A, LESKOVEC J. node2vec:scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco, California, USA, 2016:855-864.
[10] YANG Cheng, LIU Zhiyuan, ZHAO Deli, et al. Network representation learning with rich text information[C]//Proceedings of the 24th International Conference on Artificial Intelligence. Buenos Aires, Argentina, 2015:2111-2117.
[11] TU Cuchao, LIU Han, LIU Zhiyuan, et al. Cane:context-aware network embedding for relation modeling[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada, 2007:1722-1731.
[12] TANG Lei, LIU Huan. Scalable learning of collective behavior based on sparse social dimensions[C]//Proceedings of the 18th ACM Conference on Information and Knowledge Management. Hong Kong, China, 2009:1107-1116.
[13] BALASUBRAMANIAN M, SCHWARTZ E L, TENENBAUM J B, et al. The isomap algorithm and topological stability[J]. Science, 2002, 295(5552):7-7.
[14] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, 2013:3111-3119.
[15] RECHT B, Ré C, WRIGHT S J, et al. Hogwild:a lock-free approach to parallelizing stochastic gradient descent[C]//Proceedings of Advances in Neural Information Processing Systems. 2011:693-701.
[16] CAO Shaosheng, LU Wei, XU Qiongkai. Grarep:learning graph representations with global structural information[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. Melbourne, Australia, 2005:891-900.
[17] TU Cunchao, ZHANG Weicheng, LIU Zhiyuan, et al. Max-margin deepwalk:discriminative learning of network representation[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. New York, USA, 2006:3889-3895.
[18] PAGE L, BRIN S, MOTWANI R, et al. The PageRank citation ranking:Bringing order to the web[R]. Palo Alto:Stanford InfoLab, 1999.
[19] KINGMA D, BA J. Adam:a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diago, USA, 2015.
[20] HSU C W, CHANG C C, LIN C J. A practical guide to support vector classification. http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf.
[21] HANLEY J A, MCNEIL B J. The meaning and use of the area under a Receiver Operating Characteristic (ROC) curve[J]. Radiology, 1982, 143:29-36.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems