[1]SHEN Jian,XIA Hongbin,LIU Yuan.Entity relation extraction model with dual relation prediction and feature fusion[J].CAAI Transactions on Intelligent Systems,2024,19(2):462-471.[doi:10.11992/tis.202204047]
Copy

Entity relation extraction model with dual relation prediction and feature fusion

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 2 Page number: 462-471 Column: 人工智能院长论坛 Public date: 2024-03-05
Title:
Entity relation extraction model with dual relation prediction and feature fusion
Author(s):
SHEN Jian1;  XIA Hongbin1; 2;  LIU Yuan1; 2
1. School of Artificial Intelligence and Computer, Jiangnan University, Wuxi 214122, China;
2. Jiangsu Key Laboratory of Media Design and Software Technology, Wuxi 214122, China
Keywords:
entity relation extraction; relational triple; BERT pretrained model; dual relation prediction; pointer network; feature fusion; gated linear unit; conditional layer normalization
CLC:
TP391
DOI:
10.11992/tis.202204047
Abstract:
The staged decoding entity relation extraction model still has an insufficient feature fusion problem between stages, which increases the impact of exposure bias on the extraction performance. Herein, we propose a new entity relation extraction model with dual relation prediction and feature fusion (DRPFF). DRPFF uses a pretrained model of bidirectional encoder representation from transformers to encode texts, and a two-stage dual relation prediction structure is developed to reduce the false triples’ generation. Between stages, a structure combining gated linear units and conditional layer normalization is utilized to fuse features better between entities. Experimental findings on two public datasets, NYT and WebNLG, demonstrate that the presented method has better results than the baseline methods.
References:
[1] 张勇, 高大林, 巩敦卫, 等. 用于关系抽取的注意力图长短时记忆神经网络[J]. 智能系统学报, 2021, 16(3): 518–527
ZHANG Yong, GAO Dalin, GONG Dunwei, et al. Attention graph long short-term memory neural network for relation extraction[J]. CAAI transactions on intelligent systems, 2021, 16(3): 518–527
[2] 杨志豪, 洪莉, 林鸿飞, 等. 基于支持向量机的生物医学文献蛋白质关系抽取[J]. 智能系统学报, 2008, 3(4): 361–369
YANG Zhihao, HONG Li, LIN Hongfei, et al. Extraction of information on protein-protein interaction from biomedical literatures using an SVM[J]. CAAI transactions on intelligent systems, 2008, 3(4): 361–369
[3] 范智渊, 何璇, 梁品, 等. 中文医学文献的实体关系提取研究及在糖尿病医学文献中的应用[J]. 生物医学工程学杂志, 2021, 38(3): 563–573
FAN Zhiyuan, HE Xuan, LIANG Pin, et al. Research on entity relationship extraction of Chinese medical literature and application in diabetes medical literature[J]. Journal of biomedical engineering, 2021, 38(3): 563–573
[4] KAMBHATLA N. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations[C]//Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions. Morristown: Association for Computational Linguistics, 2004: 178-181.
[5] ZHAO Shubin, GRISHMAN R. Extracting relations with integrated information using kernel methods[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics-ACL ’05. Morristown: Association for Computational Linguistics, 2005: 419-426.
[6] SOARES L B, FITZGERALD N, LING J, et al. Matching the blanks: distributional similarity for relation learning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 2895-2905.
[7] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000-6010.
[8] CHUNG Y A, ZHU Chenguang, ZENG M. SPLAT: speech-language joint pre-training for spoken language understanding[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2021: 4171-4186.
[9] 罗欣, 陈艳阳, 耿昊天, 等. 基于深度强化学习的文本实体关系抽取方法[J]. 电子科技大学学报, 2022, 51(1): 91–99
LUO Xin, CHEN Yanyang, GENG Haotian, et al. Entity relationship extraction from text data based on deep reinforcement learning[J]. Journal of University of Electronic Science and Technology of China, 2022, 51(1): 91–99
[10] ZHENG Suncong, WANG Feng, BAO Hongyun, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 1227–1236.
[11] 苗琳, 张英俊, 谢斌红, 等. 基于图神经网络的联合实体关系抽取[J]. 计算机应用研究, 2022, 39(2): 424–431
MIAO Lin, ZHANG Yingjun, XIE Binhong, et al. Joint entity relation extraction based on graph neural network[J]. Application research of computers, 2022, 39(2): 424–431
[12] WEI Zhepei, SU Jianlin, WANG Yue, et al. A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 1476-1488.
[13] WANG Yucheng, YU Bowen, ZHANG Yueyang, et al. TPLinker: single-stage joint extraction of entities and relations through token pair linking[C]//Proceedings of the 28th International Conference on Computational Linguistics. Stroudsburg: International Committee on Computational Linguistics, 2020: 1572–1582.
[14] DAUPHIN Y N, FAN A, AULI M, et al. Language modeling with gated convolutional networks[C]// Proceedings of the 34th International Conference on Machine Learning. Sydney: ICML, 2017: 933-941.
[15] 张龙辉, 尹淑娟, 任飞亮, 等. BSLRel: 基于二元序列标注的级联关系三元组抽取模型[J]. 中文信息学报, 2021, 35(6): 74–84
ZHANG Longhui, YIN Shujuan, REN Feiliang, et al. BSLRel: a binary sequence labeling based cascading relation triple extraction model[J]. Journal of Chinese information processing, 2021, 35(6): 74–84
[16] ZHANG Yue, YANG Jie. Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 1554-1564.
[17] LI Xiaonan, YAN Hang, QIU Xipeng, et al. FLAT: Chinese NER using flat-lattice transformer[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 6836-6842.
[18] LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields: probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the Eighteenth International Conference on Machine Learning. New York: ACM, 2001: 282-289.
[19] ZENG Xiangrong, HE Shizhu, ZENG Daojian, et al. Learning the extraction order of multiple relational facts in a sentence with reinforcement learning[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2019: 367–377.
[20] ZENG Xiangrong, ZENG Daojian, HE Shizhu, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 506–514.
[21] EBERTS M, ULGES A. Span-based joint entity and relation extraction with transformer pre-training[C]//Proceedings of the 24th European Conference on Artificial Intelligence. Santiago de Compostela: ECAI, 2020: 2006-2013.
[22] LI Xiaoya, FENG Jingrong, MENG Yuxian, et al. A unified MRC framework for named entity recognition[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 5849-5859.
[23] YU Bowen, ZHANG Zhenyu, SU Jianlin, et al. Joint extraction of entities and relations based on a novel decomposition strategy[C] // Proceedings of the 24th European Conference on Artificial Intelligence. Santiago de Compostela: ECAI, 2020: 2282-2289.
[24] 王泽儒, 柳先辉. 基于指针级联标注的中文实体关系联合抽取模型[J]. 武汉大学学报(理学版), 2022, 68(3): 304–310
WANG Zeru, LIU Xianhui. Joint model of Chinese entity-relation extraction based on a pointer cascade tagging strategy[J]. Journal of Wuhan University (natural science edition), 2022, 68(3): 304–310
[25] DE VRIES H, STRUB F, MARY J, et al. Modulating early visual processing by language[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6597-6607.
[26] RIEDEL S, YAO Limin, MCCALLUM A. Modeling relations and their mentions without labeled text[C]//Proceedings of the 2010th European Conference on Machine Learning and Knowledge Discovery in Databases-Volume Part III. New York: ACM, 2010: 148-163.
[27] GARDENT C, SHIMORINA A, NARAYAN S, et al. Creating training corpora for NLG micro-planners[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 179–188.
[28] KINGMA D P, BA J L. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego: ICLR, 2015.
[29] FU T J, LI P H, MA Weiyun. GraphRel: modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 1409–1418.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems