[1]张勇,高大林,巩敦卫,等.用于关系抽取的注意力图长短时记忆神经网络[J].智能系统学报,2021,16(3):518-527.[doi:10.11992/tis.202008036]
ZHANG Yong,GAO Dalin,GONG Dunwei,et al.Attention graph long short-term memory neural network for relation extraction[J].CAAI Transactions on Intelligent Systems,2021,16(3):518-527.[doi:10.11992/tis.202008036]
点击复制
ZHANG Yong,GAO Dalin,GONG Dunwei,et al.Attention graph long short-term memory neural network for relation extraction[J].CAAI Transactions on Intelligent Systems,2021,16(3):518-527.[doi:10.11992/tis.202008036]
用于关系抽取的注意力图长短时记忆神经网络
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第3期
页码:
518-527
栏目:
学术论文—知识工程
出版日期:
2021-05-05
- Title:
- Attention graph long short-term memory neural network for relation extraction
- Keywords:
- relation extraction; sentence structure tree; syntactic diagram; graph neural network; AGLSTM; soft pruning strategy; attention mechanism; LSTM
- 分类号:
- TP311
- 摘要:
- 关系抽取是信息获取中一项关键技术。句子结构树能够捕获单词之间的长距离依赖关系，已被广泛用于关系抽取任务中。但是，现有方法存在过度依赖句子结构树本身信息而忽略外部信息的不足。本文提出一种新型的图神经网络模型，即注意力图长短时记忆神经网络(attention graph long short term memory neural network, AGLSTM)。该模型采用一种软修剪策略自动学习对关系抽取有用的句子结构信息；通过引入注意力机制，结合句法图信息学习句子的结构特征；并设计一种新型的图长短时记忆神经网络，使得模型能够更好地融合句法图信息和句子的时序信息。与10种典型的关系抽取方法进行对比，实验验证了该模型的优异性能。
- Abstract:
- Relation extraction is a key technology in information acquisition. The sentence structure tree that can capture long-distance dependencies between words has been widely used in relational extraction tasks. However, existing methods still have the disadvantage of relying too much on the information of sentence structure tree and ignoring external information. This paper proposes a new graph neural network structure, namely the attention graph long short term memory neural network (AGLSTM). The model adopts a soft pruning strategy to automatically learn sentence structure information useful for relation extraction tasks; then the attention mechanism is introduced and combined with the syntactic graph information to learn the structural features of the sentence; And designed a new type of graph long short term memory neural network to better fuse syntactic graph information and sentence timing information. Compared with 10 typical relational extraction methods, experiments verify the excellent performance of the proposed method.
- 参考文献/References:
-
[1] 杨志豪,洪莉,林鸿飞,等. 基于支持向量机的生物医学文献蛋白质关系抽取[J]. 智能系统学报, 2008(4):361-369
Yang Zhihao, Hong Li, Lin Hongfei, et al. Extraction of information on protein-protein interaction from biomedical literatures using an SVM[J]. CAAI transactions on intelligent systems, 2008(4):361-369
[2] 李智超. 图文知识图谱中的关系抽取算法研究[D]. 北京:北京邮电大学, 2018.
LI Zhichao. A relation extraction algorithm in multi-modal knowledge graph[D]. Beijing:Beijing University of Posts and Telecommunications, 2018.
[3] 张涛,贾真,李天瑞,等. 基于知识库的开放领域问答系统[J]. 智能系统学报, 2018, 13(4):557-563
ZHANG Tao, JIA Zhen, LI Tianrui, et al. Open-domain question-answering system based on large-scale knowledge base[J]. CAAI transactions on intelligent systems, 2018, 13(4):557-563
[4] ZHOU Peng, SHI Wei, TIAN Jun, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany:Association for Computational Linguistics, 2016:207-212.
[5] ZHANG Lei, XIANG Fusheng. Relation classification via BiLSTM-CNN[C]//Proceedings of the 3rd International Conference on Data Mining and Big Data. Shanghai, China:Springer, 2018:373-382.
[6] XU Yan, MOU Lili, LI Ge, et al. Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal:Association for Computational Linguistics, 2015:1785-1794.
[7] TAI K S, SOCHER R, MANNING C D. Improved semantic representations from tree-structured long short-Term memory networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Beijing, China:Association for Computational Linguistics, 2015:1556-1566.
[8] ZHANG Yuhao, QI Peng, MANNING C D. Graph convolution over pruned dependency trees improves relation extraction[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium:Association for Computational Linguistics, 2018:2205-2215.
[9] 甘丽新, 万常选, 刘德喜, 等. 基于句法语义特征的中文实体关系抽取[J]. 计算机研究与发展, 2016, 53(2):284-302
GAN Lixin, WANG Changxuan, LIU Dexi, et al. Chinese named entity relation extraction based on syntactic and semantic features[J]. Journal of computer research and development, 2016, 53(2):284-302
[10] GUO Zhijiang, ZHANG Yan, LU Wei. Attention guided graph convolutional networks for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy:ACL, 241-251.
[11] FU T J, LI P H, MA Weiyun. GraphRel:modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy:Association for Computational Linguistics, 2019:1409-1418.
[12] PENG Nanyun, POON H, QUIRK C, et al. Cross-sentence N-ary relation extraction with graph LSTMs[J]. Transactions of the association for computational linguistics, 2017, 5:101-115.
[13] SONG Linfeng, ZHANG Yue, WANG Zhiguo, et al. N-ary relation extraction using graph state LSTM[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium:Association for Computational Linguistics, 2018:2226-2235.
[14] ZHOU Peng, XU Jiaming, QI Zhenyu, et al. Distant supervision for relation extraction with hierarchical selective attention[J]. Neural networks, 2018, 108:240-247.
[15] JI Guoliang, LIU Kang, HE Shizhu, et al. Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, USA:AAAI Press, 2017.
[16] ZHANG Shu, ZHENG Dequan, HU Xinchen, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation. Shanghai, China:PACLIC, 2015:73-78.
[17] ZELENKO D, AONE C, RICHARDELLA A. Kernel methods for relation extraction[J]. The journal of machine learning research, 2003, 3:1083-1106.
[18] ZENG Daojian, LIU Kang, LAI Siwei, et al. Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics:Technical Papers. Dublin, Ireland:Dublin City University and Association for Computational Linguistics, 2014:2335-2344.
[19] ZHANG Yuhao, ZHONG V, CHEN Danqi, et al. Position-aware attention and supervised data improve slot filling[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark:Association for Computational Linguistics, 2017:35-45.
[20] 马语丹, 赵义, 金婧, 等. 结合实体共现信息与句子语义特征的关系抽取方法[J]. 中国科学:信息科学, 2018, 48(11):1533-1545
MA Yudan, ZHAO Yi, JIN Jing, et al. Combining entity co-occurrence information and sentence semantic features for relation extraction[J]. Scientia sinica informationis, 2018, 48(11):1533-1545
[21] MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Suntec, Singapore:Association for Computational Linguistics, 2009:1003-1011.
[22] ZENG D, KANG L, CHEN Y, et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processin. Lisbon, Portugal, 2015:1753-1762.
[23] HENDRICKX I, KIM S N, KOZAREVA Z, et al. Semeval-2010 task 8:Multi-way classification of semantic relations between pairs of nominals[C]//Proceedings of the 5th International Workshop on Semantic Evaluation. Uppsala, Sweden:ACM, 2010:33-38.
[24] PENNINGTON J, SOCHER R, MANNING C. GloVe:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Doha, Qatar:Association for Computational Linguistics, 2014:1532-1543.
[25] YU Bowen, ZHANG Zhenyu, LIU Tingwen, et al. Beyond word attention:using segment attention in neural relation extraction[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. Macao, China:IJCAI, 2019:33-38.
[26] LEE J, SEO S, CHOI Y S. Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing[J]. Symmetry, 2019, 11(6):785.
- 相似文献/References:
-
[1]杨志豪,洪 莉,林鸿飞,等.基于支持向量机的生物医学文献蛋白质关系抽取[J].智能系统学报,2008,3(4):361.
YANG Zhi-hao,HONG L i,L IN Hong-fei,et al.Extraction of information on prote in2prote in interaction from biomedical literatures using an SVM[J].CAAI Transactions on Intelligent Systems,2008,3():361.
[2]贾真,何大可,杨燕,等.基于弱监督学习的中文网络百科关系抽取[J].智能系统学报,2015,10(1):113.[doi:10.10.3969/j.issn.1673-4785.201311017]
JIA Zhen,HE Dake,YANG Yan,et al.Relation extraction from Chinese online encyclopedia based on weakly supervised learnin[J].CAAI Transactions on Intelligent Systems,2015,10():113.[doi:10.10.3969/j.issn.1673-4785.201311017]
[3]姚贤明,甘健侯,徐坚.面向中文开放领域的多元实体关系抽取研究[J].智能系统学报,2019,14(3):597.[doi:10.11992/tis.201805006]
YAO Xianming,GAN Jianhou,XU Jian.Chinese open domain oriented n-ary entity relation extraction[J].CAAI Transactions on Intelligent Systems,2019,14():597.[doi:10.11992/tis.201805006]
[4]胡峰,杨新瑞,汤成富,等.基于自适应损失函数的句子级远程监督关系抽取[J].智能系统学报,2024,19(3):697.[doi:10.11992/tis.202205034]
HU Feng,YANG Xinrui,TANG Chengfu,et al.Sentence-level distant supervision relation extraction based on self-adaptive loss function[J].CAAI Transactions on Intelligent Systems,2024,19():697.[doi:10.11992/tis.202205034]
备注/Memo
收稿日期:2020-08-30。
基金项目:国家重点研发计划项目(2020YFB1708200)；科技部科技创新2030重大项目(2020AAA0107300)
作者简介:张勇，教授，博士生导师，博士，中国人工智能学会自然计算与数字智能城市专委会委员，主要研究方向为智能优化和数据挖掘。主持国家自然科学基金3项，中国博士后科学基金特别资助等省部级科研项目5项。获教育部高等学校科学研究优秀成果二等奖。获授权发明专利4项，发表学术论文50余篇;高大林，硕士研究生，主要研究方向为自然语言处理、关系抽取;巩敦卫，教授，博士生导师，博士，江苏省自动化学会常务理事、副秘书长，主要研究方向为智能优化和软件测试。主持国家“973”计划子课题1项，国家重点研发计划子课题1项，国家自然科学基金6项，省部级科研项目8项。获高等学校科学研究优秀成果二等奖、江苏省科学技术二等奖。获授权发明专利15项。出版专著8部，发表学术论文100余篇
通讯作者:高大林.E-mail:1367963012@qq.com
更新日期/Last Update:
2021-06-25