[1]CHEN Xinyuan,XIE Shengyi,CHEN Qingqiang,et al.Knowledge-based inference on convolutional feature extraction and path semantics[J].CAAI Transactions on Intelligent Systems,2021,16(4):729-738.[doi:10.11992/tis.202008007]
Copy

Knowledge-based inference on convolutional feature extraction and path semantics

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 4 Page number: 729-738 Column: 学术论文—知识工程 Public date: 2021-07-05
Title:
Knowledge-based inference on convolutional feature extraction and path semantics
Author(s):
CHEN Xinyuan1; 2;  XIE Shengyi3;  CHEN Qingqiang4;  LIU Yu5
1. College of Computer and Control Engineering, Minjiang University, Fuzhou 350121, China;
2. Department of Information Engineering, Fuzhou Melbourne Polytechnic, Fuzhou 350121, China;
3. Teaching and Research Division,, Fujian Vocational College of Agriculture, Fuzhou 350181, China;
4. Information Science and Engineering College, Fujian University of Technology, Fuzhou 350118, China;
5. Modern Education Technical Center, Fuzhou Melbourne Polytechnic, Fuzhou 350121, China
Keywords:
knowledge graph; knowledge inference; embedding representation; path information; convolutional neural network (CNN); long-short term memory (LSTM); attention mechanism; link prediction
CLC:
TP391
DOI:
10.11992/tis.202008007
Abstract:
Embedding-based feature extraction methods usually ignore path semantics; there is still scope of improvement of relational path-based algorithms, which generally consider single paths. To further boost the performance of knowledge-based inferences, a self-defined convolutional neural network framework was employed to encode multiple paths generated by random walks into low-dimensional representations that are merged to form a single vector of hidden states with long-short term memory (LSTM); this is accomplished by combining the attention mechanism-based processes. Semantic information of multiple paths is integrated with various weight distributions used for measuring probability scores of triples comprising candidate relations and entity pairs to determine whether the triples hold or not. Link prediction experiments performed on NELL995 and FB15k-237 demonstrated the capability of the proposed model. Scores of F1 and other indicators also confirmed the advantages of our framework compared with mainstream models. The model was further tested on FC17 and NELL-One.
References:
[1] LEHMANN J, ISELE R, JAKOB M, et al. DBpedia-a large-scale, multilingual knowledge base extracted from Wikipedia[J]. Semantic web, 2015, 6(2): 167-195.
[2] MITCHELL T, COHEN W, HRUSCHKA E, et al. Never-ending learning[J]. Communications of the ACM, 2018, 61(5): 103-115.
[3] REBELE T, SUCHANEK F, HOFFART J, et al. YAGO: a multilingual knowledge base from Wikipedia, Wordnet, and Geonames[C]//Proceedings of the 15th International Semantic Web Conference on the Semantic Web. Kobe, Japan, 2016: 177-185.
[4] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of 2008 ACM SIGMOD International Conference on Management of Data. Vancouver, Canada, 2008: 1247-1250.
[5] XIONG Chenyan, POWER R, CALLAN J. Explicit semantic ranking for academic search via knowledge graph embedding[C]//Proceedings of the 26th International Conference on World Wide Web. Perth, Australia, 2017: 1271-1279.
[6] HAO Yanchao, ZHANG Yuanzhe, LIU Kang, et al. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada, 2017: 221-231.
[7] 刘知远, 孙茂松, 林衍凯, 等. 知识表示学习研究进展[J]. 计算机研究与发展, 2016, 53(2): 247-261
LIU Zhiyuan, SUN Maosong, LIN Yankai, et al. Knowledge representation learning: a review[J]. Computer research and development, 2016, 53(2): 247-261
[8] WEST R, GABRILOVICH E, MURPHY K, et al. Knowledge base completion via search-based question answering[C]//Proceedings of the 23rd International Conference on World Wide Web. Seoul, Korea, 2014: 515-526.
[9] 刘峤, 李杨, 段宏, 等. 知识图谱构建技术综述[J]. 计算机研究与发展, 2016, 53(3): 582-600
LIU Qiao, LI Yang, DUAN Hong, et al. Knowledge graph construction techniques[J]. Journal of computer research and development, 2016, 53(3): 582-600
[10] 徐增林, 盛泳潘, 贺丽荣, 等. 知识图谱技术综述[J]. 电子科技大学学报, 2016, 45(4): 589-606
XU Zenglin, SHENG Yongpan, HE Lirong, et al. Review on knowledge graph techniques[J]. Journal of University of Electronic Science and Technology of China, 2016, 45(4): 589-606
[11] CHEN Xiaojun, JIA Shengbin, XIANG Yang. A review: knowledge reasoning over knowledge graph[J]. Expert systems with applications, 2020, 141: 112948.
[12] 官赛萍, 靳小龙, 贾岩涛, 等. 面向知识图谱的知识推理研究进展[J]. 软件学报, 2018, 29(10): 2966-2994
GUAN Saiping, JIN Xiaolong, JIA Yantao, et al. Knowledge reasoning over knowledge graph: a survey[J]. Journal of software, 2018, 29(10): 2966-2994
[13] CAI Hongyun, ZHENG V W, CHANG K C C. A comprehensive survey of graph embedding: problems, techniques, and applications[J]. IEEE transactions on knowledge and data engineering, 2018, 30(9): 1616-1637.
[14] WANG Quan, MAO Zhendong, WANG Bin, et al. Knowledge graph embedding: a survey of approaches and applications[J]. IEEE transactions on knowledge and data engineering, 2017, 29(12): 2724-2743.
[15] NICKEL M, MURPHY K, TRESP V, et al. A review of relational machine learning for knowledge graphs[J]. Proceedings of the IEEE, 2016, 104(1): 11-33.
[16] Bordes A, Usunier N, Garcia-Durán A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe, United States, 2013: 2787-2795.
[17] YANG Bishan, YIH W T, HE Xiaodong, et al. Embedding entities and relations for learning and inference in knowledge bases[J/OL].(2020-01-01)[2020-05-01] https://arxiv.org/abs/1412.6575.
[18] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18). New Orleans, USA, 2017: 1811-1818.
[19] LAO Ni, COHEN W W. Relational retrieval using a combination of path-constrained random walks[J]. Machine learning, 2010, 81(1): 53-67.
[20] LAO Ni, MITCHELL T, COHEN W W. Random walk inference and learning in a large scale knowledge base[C]//Proceedings of 2011 Conference on Empirical Methods in Natural Language Processing. Edinburgh, United Kingdom, 2011: 529-539.
[21] NEELAKANTAN A, ROTH B, MCCALLUM A. Compositional vector space models for knowledge base completion[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Beijing, China, 2015: 156-166.
[22] DAS R, NEELAKANTAN A, BELANGER D, et al. Chains of reasoning over entities, relations, and text using recurrent neural networks[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Valencia, Spain, 2017: 132-141.
[23] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[24] XU K, BA J L, KIROS R, et al. Show, attend and tell: neural image caption generation with visual attention[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille, France, 2015: 2048-2057.
[25] XIONG Wenhan, HOANG T, WANG W Y. Deeppath: a reinforcement learning method for knowledge graph reasoning[C]//Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark, 2017: 564-573.
[26] ZHOU Peng, SHI Wei, TIAN Jun, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 207-212.
[27] JIANG Xiaotian, WANG Quan, QI Baoyuan, et al. Attentive path combination for knowledge graph completion[C]//Proceedings of the 9th Asian Conference on Machine Learning. Seoul, Korea, 2017: 590-605.
[28] SHEN Yelong, CHEN Jianshu, HUANG Posen, et al. M-walk: learning to walk over graphs using monte Carlo tree search[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal, Canada, 2018: 6787-6798.
[29] WANG Zhen, ZHANG Jianwen, FENG Jianlin, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence. Montréal, Canada, 2014: 1112-1119.
[30] LIN Yankai, LIU Zhiyuan, SUN Maosong, et al. Learning entity and relation embeddings for knowledge graph completion[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence. Austin, USA, 2015: 2181-2187.
[31] KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Doha, Qatar, 2014: 1746-1751.
[32] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning. New York, USA, 2016: 2071-2080.
[33] BALA?EVI? I, ALLEN C, HOSPEDALES T M. Hypernetwork knowledge graph embeddings[C]//Proceedings of the 28th International Conference on Artificial Neural Networks and Machine Learning. Munich, Germany, 2019: 553-565.
[34] NATHANI D, CHAUHAN J, SHARMA C, et al. Learning attention-based embeddings for relation prediction in knowledge graphs[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics. Florence, Italy, 2019: 4710-4723.
[35] TAKAHASHI R, TIAN R, INUI K. Interpretable and compositional relation learning by joint training with an autoencoder[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia, 2018: 2148-2159.
[36] ZHANG Yao, ZHANG Xu, WANG Jun, et al. GMH: a general multi-hop reasoning model for KG completion[J]. (2020-01-01)[2020-05-01] https://arxiv.org/abs/2010.07620.
[37] XIONG Wenhan, YU Mo, CHANG Shiyu, et al. One-shot relational learning for knowledge graphs[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium, 2018: 1980-1990.
[38] NGUYEN D Q, SIRTS K, QU Lizhen, et al. Neighborhood mixture model for knowledge base completion[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. Berlin, Germany, 2016: 40-50.
[39] TOUTANOVA K, LIN X V, YIH W T, et al. Compositional learning of embeddings for relation paths in knowledge base and text[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 1434-1444.
[40] DAS R, DHULIAWALA S, ZAHEER M, et al. Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2017.
[41] LIN X V, SOCHER R, XIONG Caiming. Multi-hop knowledge graph reasoning with reward shaping[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium, 2018: 3243-3253.
[42] LIN Yankai, LIU Zhiyuan, LUAN Huabo, et al. Modeling relation paths for representation learning of knowledge bases[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal, 2015: 705-714.
[43] LUO Yuanfei, WANG Quan, WANG Bin, et al. Context-dependent knowledge graph embedding[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal, 2015: 1656-1661.
[44] SHANG Chao, TANG Yun, HUANG Jing, et al. End-to-end structure-aware convolutional networks for knowledge base completion[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, the 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Honolulu, United States, 2019: 3060-3067.
[45] TUAN Yilin, CHEN Y N, LEE H Y. DyKgChat: Benchmarking dialogue generation grounding on dynamic knowledge graphs[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China, 2019: 1855-1865.
[46] GREFF K, SRIVASTAVA R K, KOUTNíK J, et al. LSTM: A search space odyssey[J]. IEEE transactions on neural networks and learning systems, 2017, 28(10): 2222-2232.
[47] XIE Qizhe, MA Xuezhe, DAI Zihang, et al. An interpretable knowledge transfer model for knowledge base completion[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada, 2017: 950-962.
[48] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego, USA, 2014.
[49] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30. Long Beach, USA, 2017: 5998-6008.
[50] WANG Xiang, WANG Dingxian, XU Canran, et al. Explainable reasoning over knowledge graphs for recommendation[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, the 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. New York, United States, 2019: 5329-5336.
[51] GRAVES A, MOHAMED A R, HINTON G. Speech recognition with deep recurrent neural networks[C]//Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, Canada, 2013: 6645-6649.
[52] CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J]. Transactions of the association for computational linguistics, 2016, 4: 357-370.
[53] LU R, DUAN Z. Bidirectional GRU for sound event detection[C]//Detection and Classification of Acoustic Scenes and Events. [S. l.]. 2017: 17-20.
[54] CHUNG J, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. (2020-01-01)[2020-05-01] https://arxiv.org/abs/1412.3555.
[55] KINGMA D P, BA J. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego, USA, 2014: 604?612
[56] JIANG Tianwen, ZHAO Tong, QIN Bing, et al. The role of "Condition": a novel scientific knowledge graph representation and construction model[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage, United States, 2019: 1634-1642.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems