[1]陈新元,谢晟祎,陈庆强,等.结合卷积特征提取和路径语义的知识推理[J].智能系统学报,2021,16(4):729-738.[doi:10.11992/tis.202008007]
CHEN Xinyuan,XIE Shengyi,CHEN Qingqiang,et al.Knowledge-based inference on convolutional feature extraction and path semantics[J].CAAI Transactions on Intelligent Systems,2021,16(4):729-738.[doi:10.11992/tis.202008007]
点击复制
CHEN Xinyuan,XIE Shengyi,CHEN Qingqiang,et al.Knowledge-based inference on convolutional feature extraction and path semantics[J].CAAI Transactions on Intelligent Systems,2021,16(4):729-738.[doi:10.11992/tis.202008007]
结合卷积特征提取和路径语义的知识推理
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第4期
页码:
729-738
栏目:
学术论文—知识工程
出版日期:
2021-07-05
- Title:
- Knowledge-based inference on convolutional feature extraction and path semantics
- Keywords:
- knowledge graph; knowledge inference; embedding representation; path information; convolutional neural network (CNN); long-short term memory (LSTM); attention mechanism; link prediction
- 分类号:
- TP391
- 摘要:
- 传统特征提取方法大多基于嵌入表达，常忽略了路径语义；基于关系路径的推理方法多考虑单一路径，性能仍有提升空间。为进一步提升知识推理能力，使用自定义的卷积神经网络框架编码随机游走生成的多条路径，利用双向长短期记忆网络的隐藏状态合并向量序列，结合注意力机制实现差异化的多路径语义信息集成，计算候选关系与实体对的概率得分，用于判断三元组是否成立。NELL995和FB15k-237数据集上的链路预测结果证明方案可行，F1等指标相比主流模型也有一定优势；进一步在大型数据集和稀疏数据集上验证方案可行。
- Abstract:
- Embedding-based feature extraction methods usually ignore path semantics; there is still scope of improvement of relational path-based algorithms, which generally consider single paths. To further boost the performance of knowledge-based inferences, a self-defined convolutional neural network framework was employed to encode multiple paths generated by random walks into low-dimensional representations that are merged to form a single vector of hidden states with long-short term memory (LSTM); this is accomplished by combining the attention mechanism-based processes. Semantic information of multiple paths is integrated with various weight distributions used for measuring probability scores of triples comprising candidate relations and entity pairs to determine whether the triples hold or not. Link prediction experiments performed on NELL995 and FB15k-237 demonstrated the capability of the proposed model. Scores of F1 and other indicators also confirmed the advantages of our framework compared with mainstream models. The model was further tested on FC17 and NELL-One.
- 参考文献/References:
-
[1] LEHMANN J, ISELE R, JAKOB M, et al. DBpedia-a large-scale, multilingual knowledge base extracted from Wikipedia[J]. Semantic web, 2015, 6(2): 167-195.
[2] MITCHELL T, COHEN W, HRUSCHKA E, et al. Never-ending learning[J]. Communications of the ACM, 2018, 61(5): 103-115.
[3] REBELE T, SUCHANEK F, HOFFART J, et al. YAGO: a multilingual knowledge base from Wikipedia, Wordnet, and Geonames[C]//Proceedings of the 15th International Semantic Web Conference on the Semantic Web. Kobe, Japan, 2016: 177-185.
[4] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of 2008 ACM SIGMOD International Conference on Management of Data. Vancouver, Canada, 2008: 1247-1250.
[5] XIONG Chenyan, POWER R, CALLAN J. Explicit semantic ranking for academic search via knowledge graph embedding[C]//Proceedings of the 26th International Conference on World Wide Web. Perth, Australia, 2017: 1271-1279.
[6] HAO Yanchao, ZHANG Yuanzhe, LIU Kang, et al. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada, 2017: 221-231.
[7] 刘知远, 孙茂松, 林衍凯, 等. 知识表示学习研究进展[J]. 计算机研究与发展, 2016, 53(2): 247-261
LIU Zhiyuan, SUN Maosong, LIN Yankai, et al. Knowledge representation learning: a review[J]. Computer research and development, 2016, 53(2): 247-261
[8] WEST R, GABRILOVICH E, MURPHY K, et al. Knowledge base completion via search-based question answering[C]//Proceedings of the 23rd International Conference on World Wide Web. Seoul, Korea, 2014: 515-526.
[9] 刘峤, 李杨, 段宏, 等. 知识图谱构建技术综述[J]. 计算机研究与发展, 2016, 53(3): 582-600
LIU Qiao, LI Yang, DUAN Hong, et al. Knowledge graph construction techniques[J]. Journal of computer research and development, 2016, 53(3): 582-600
[10] 徐增林, 盛泳潘, 贺丽荣, 等. 知识图谱技术综述[J]. 电子科技大学学报, 2016, 45(4): 589-606
XU Zenglin, SHENG Yongpan, HE Lirong, et al. Review on knowledge graph techniques[J]. Journal of University of Electronic Science and Technology of China, 2016, 45(4): 589-606
[11] CHEN Xiaojun, JIA Shengbin, XIANG Yang. A review: knowledge reasoning over knowledge graph[J]. Expert systems with applications, 2020, 141: 112948.
[12] 官赛萍, 靳小龙, 贾岩涛, 等. 面向知识图谱的知识推理研究进展[J]. 软件学报, 2018, 29(10): 2966-2994
GUAN Saiping, JIN Xiaolong, JIA Yantao, et al. Knowledge reasoning over knowledge graph: a survey[J]. Journal of software, 2018, 29(10): 2966-2994
[13] CAI Hongyun, ZHENG V W, CHANG K C C. A comprehensive survey of graph embedding: problems, techniques, and applications[J]. IEEE transactions on knowledge and data engineering, 2018, 30(9): 1616-1637.
[14] WANG Quan, MAO Zhendong, WANG Bin, et al. Knowledge graph embedding: a survey of approaches and applications[J]. IEEE transactions on knowledge and data engineering, 2017, 29(12): 2724-2743.
[15] NICKEL M, MURPHY K, TRESP V, et al. A review of relational machine learning for knowledge graphs[J]. Proceedings of the IEEE, 2016, 104(1): 11-33.
[16] Bordes A, Usunier N, Garcia-Durán A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe, United States, 2013: 2787-2795.
[17] YANG Bishan, YIH W T, HE Xiaodong, et al. Embedding entities and relations for learning and inference in knowledge bases[J/OL].(2020-01-01)[2020-05-01] https://arxiv.org/abs/1412.6575.
[18] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18). New Orleans, USA, 2017: 1811-1818.
[19] LAO Ni, COHEN W W. Relational retrieval using a combination of path-constrained random walks[J]. Machine learning, 2010, 81(1): 53-67.
[20] LAO Ni, MITCHELL T, COHEN W W. Random walk inference and learning in a large scale knowledge base[C]//Proceedings of 2011 Conference on Empirical Methods in Natural Language Processing. Edinburgh, United Kingdom, 2011: 529-539.
[21] NEELAKANTAN A, ROTH B, MCCALLUM A. Compositional vector space models for knowledge base completion[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Beijing, China, 2015: 156-166.
[22] DAS R, NEELAKANTAN A, BELANGER D, et al. Chains of reasoning over entities, relations, and text using recurrent neural networks[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Valencia, Spain, 2017: 132-141.
[23] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[24] XU K, BA J L, KIROS R, et al. Show, attend and tell: neural image caption generation with visual attention[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille, France, 2015: 2048-2057.
[25] XIONG Wenhan, HOANG T, WANG W Y. Deeppath: a reinforcement learning method for knowledge graph reasoning[C]//Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark, 2017: 564-573.
[26] ZHOU Peng, SHI Wei, TIAN Jun, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 207-212.
[27] JIANG Xiaotian, WANG Quan, QI Baoyuan, et al. Attentive path combination for knowledge graph completion[C]//Proceedings of the 9th Asian Conference on Machine Learning. Seoul, Korea, 2017: 590-605.
[28] SHEN Yelong, CHEN Jianshu, HUANG Posen, et al. M-walk: learning to walk over graphs using monte Carlo tree search[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal, Canada, 2018: 6787-6798.
[29] WANG Zhen, ZHANG Jianwen, FENG Jianlin, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence. Montréal, Canada, 2014: 1112-1119.
[30] LIN Yankai, LIU Zhiyuan, SUN Maosong, et al. Learning entity and relation embeddings for knowledge graph completion[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence. Austin, USA, 2015: 2181-2187.
[31] KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Doha, Qatar, 2014: 1746-1751.
[32] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning. New York, USA, 2016: 2071-2080.
[33] BALA?EVI? I, ALLEN C, HOSPEDALES T M. Hypernetwork knowledge graph embeddings[C]//Proceedings of the 28th International Conference on Artificial Neural Networks and Machine Learning. Munich, Germany, 2019: 553-565.
[34] NATHANI D, CHAUHAN J, SHARMA C, et al. Learning attention-based embeddings for relation prediction in knowledge graphs[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics. Florence, Italy, 2019: 4710-4723.
[35] TAKAHASHI R, TIAN R, INUI K. Interpretable and compositional relation learning by joint training with an autoencoder[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia, 2018: 2148-2159.
[36] ZHANG Yao, ZHANG Xu, WANG Jun, et al. GMH: a general multi-hop reasoning model for KG completion[J]. (2020-01-01)[2020-05-01] https://arxiv.org/abs/2010.07620.
[37] XIONG Wenhan, YU Mo, CHANG Shiyu, et al. One-shot relational learning for knowledge graphs[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium, 2018: 1980-1990.
[38] NGUYEN D Q, SIRTS K, QU Lizhen, et al. Neighborhood mixture model for knowledge base completion[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. Berlin, Germany, 2016: 40-50.
[39] TOUTANOVA K, LIN X V, YIH W T, et al. Compositional learning of embeddings for relation paths in knowledge base and text[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany, 2016: 1434-1444.
[40] DAS R, DHULIAWALA S, ZAHEER M, et al. Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2017.
[41] LIN X V, SOCHER R, XIONG Caiming. Multi-hop knowledge graph reasoning with reward shaping[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium, 2018: 3243-3253.
[42] LIN Yankai, LIU Zhiyuan, LUAN Huabo, et al. Modeling relation paths for representation learning of knowledge bases[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal, 2015: 705-714.
[43] LUO Yuanfei, WANG Quan, WANG Bin, et al. Context-dependent knowledge graph embedding[C]//Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal, 2015: 1656-1661.
[44] SHANG Chao, TANG Yun, HUANG Jing, et al. End-to-end structure-aware convolutional networks for knowledge base completion[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, the 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Honolulu, United States, 2019: 3060-3067.
[45] TUAN Yilin, CHEN Y N, LEE H Y. DyKgChat: Benchmarking dialogue generation grounding on dynamic knowledge graphs[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China, 2019: 1855-1865.
[46] GREFF K, SRIVASTAVA R K, KOUTNíK J, et al. LSTM: A search space odyssey[J]. IEEE transactions on neural networks and learning systems, 2017, 28(10): 2222-2232.
[47] XIE Qizhe, MA Xuezhe, DAI Zihang, et al. An interpretable knowledge transfer model for knowledge base completion[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada, 2017: 950-962.
[48] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego, USA, 2014.
[49] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30. Long Beach, USA, 2017: 5998-6008.
[50] WANG Xiang, WANG Dingxian, XU Canran, et al. Explainable reasoning over knowledge graphs for recommendation[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, the 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. New York, United States, 2019: 5329-5336.
[51] GRAVES A, MOHAMED A R, HINTON G. Speech recognition with deep recurrent neural networks[C]//Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, Canada, 2013: 6645-6649.
[52] CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J]. Transactions of the association for computational linguistics, 2016, 4: 357-370.
[53] LU R, DUAN Z. Bidirectional GRU for sound event detection[C]//Detection and Classification of Acoustic Scenes and Events. [S. l.]. 2017: 17-20.
[54] CHUNG J, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. (2020-01-01)[2020-05-01] https://arxiv.org/abs/1412.3555.
[55] KINGMA D P, BA J. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego, USA, 2014: 604?612
[56] JIANG Tianwen, ZHAO Tong, QIN Bing, et al. The role of "Condition": a novel scientific knowledge graph representation and construction model[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage, United States, 2019: 1634-1642.
- 相似文献/References:
-
[1]崔婉秋,杜军平,周南,等.基于用户意图理解的社交网络跨媒体搜索与挖掘[J].智能系统学报,2017,12(6):761.[doi:10.11992/tis.201706075]
CUI Wanqiu,DU Junping,ZHOU Nan,et al.Social network cross-media searching and mining based on user intention[J].CAAI Transactions on Intelligent Systems,2017,12():761.[doi:10.11992/tis.201706075]
[2]常亮,张伟涛,古天龙,等.知识图谱的推荐系统综述[J].智能系统学报,2019,14(2):207.[doi:10.11992/tis.201805001]
CHANG Liang,ZHANG Weitao,GU Tianlong,et al.Review of recommendation systems based on knowledge graph[J].CAAI Transactions on Intelligent Systems,2019,14():207.[doi:10.11992/tis.201805001]
[3]贾中浩,古天龙,宾辰忠,等.旅游知识图谱特征学习的景点推荐[J].智能系统学报,2019,14(3):430.[doi:10.11992/tis.201810032]
JIA Zhonghao,GU Tianlong,BIN Chenzhong,et al.Tourism knowledge-graph feature learning for attraction recommendations[J].CAAI Transactions on Intelligent Systems,2019,14():430.[doi:10.11992/tis.201810032]
[4]王坤,谢振平,陈梅婕.基于图约简的知识联想关系网络建模[J].智能系统学报,2019,14(4):679.[doi:10.11992/tis.201808009]
WANG Kun,XIE Zhenping,CHEN Meijie.Modeling knowledge network on associative relations based on graph reduction[J].CAAI Transactions on Intelligent Systems,2019,14():679.[doi:10.11992/tis.201808009]
[5]秦娅,申国伟,余红星.基于Hadoop的大规模网络安全实体识别方法[J].智能系统学报,2019,14(5):1017.[doi:10.11992/tis.201809024]
QIN Ya,SHEN Guowei,YU Hongxing.Large-scale network security entity recognition method based on Hadoop[J].CAAI Transactions on Intelligent Systems,2019,14():1017.[doi:10.11992/tis.201809024]
[6]饶官军,古天龙,常亮,等.基于相似性负采样的知识图谱嵌入[J].智能系统学报,2020,15(2):218.[doi:10.11992/tis.201811022]
RAO Guanjun,GU Tianlong,CHANG Liang,et al.Knowledge graph embedding based on similarity negative sampling[J].CAAI Transactions on Intelligent Systems,2020,15():218.[doi:10.11992/tis.201811022]
[7]贾中浩,宾辰忠,古天龙,等.基于知识图谱和用户长短期偏好的个性化景点推荐[J].智能系统学报,2020,15(5):990.[doi:10.11992/tis.201904064]
JIA Zhonghao,BIN Chenzhong,GU Tianlong,et al.Personalized attraction recommendation based on the knowledge graph and users’ long-term and short-term preferences[J].CAAI Transactions on Intelligent Systems,2020,15():990.[doi:10.11992/tis.201904064]
[8]罗玲,李硕凯,何清,等.基于知识图谱、TF-IDF和BERT模型的冬奥知识问答系统[J].智能系统学报,2021,16(4):819.[doi:10.11992/tis.202105047]
LUO Ling,LI Shuokai,HE Qing,et al.Winter Olympic Q & A system based on knowledge map, TF-IDF and BERT model[J].CAAI Transactions on Intelligent Systems,2021,16():819.[doi:10.11992/tis.202105047]
[9]黄庆明,王树徽,许倩倩,等.以图像视频为中心的跨媒体分析与推理[J].智能系统学报,2021,16(5):835.[doi:10.11992/tis.202105042]
HUANG Qingming,WANG Shuhui,XU Qianqian,et al.Image video centered cross-media analysis and reasoning[J].CAAI Transactions on Intelligent Systems,2021,16():835.[doi:10.11992/tis.202105042]
[10]肖京,王磊,杨余久,等.感知认知技术在金融风险预警中的应用研究[J].智能系统学报,2021,16(5):941.[doi:10.11992/tis.202107027]
XIAO Jing,WANG Lei,YANG Yujiu,et al.A systematic review of perceptual cognitive technology and its application in the field of financial risk early warning[J].CAAI Transactions on Intelligent Systems,2021,16():941.[doi:10.11992/tis.202107027]
[11]郭一楠,王斌,巩敦卫,等.实体结构与语义融合的多层注意力知识表示学习[J].智能系统学报,2023,18(3):577.[doi:10.11992/tis.202204026]
GUO Yinan,WANG Bin,GONG Dunwei,et al.Multi-layer attention knowledge representation learning by integrating entity structure with semantics[J].CAAI Transactions on Intelligent Systems,2023,18():577.[doi:10.11992/tis.202204026]
备注/Memo
收稿日期:2020-08-06。
基金项目:中国高等教育学会2020年度中外合作办学研究课题(ZWHZBX202003)
作者简介:陈新元，讲师，主要研究方向为NLP、知识表达与推理。主持并参与省市级科研课题10余项，主持横向课题多项。发表学术论文10余篇;谢晟祎，高级工程师，主要研究方向为人工智能、机器视觉。参与省级科研课题1项，主持市厅级课题2项。发表学术论文7篇;陈庆强，教授，主要研究方向为图像处理、知识推理。发表学术论文10余篇
通讯作者:陈庆强.E-mail:3204193260@qq.com
更新日期/Last Update:
1900-01-01