[1]沈健,夏鸿斌,刘渊.双关系预测与特征融合的实体关系抽取模型[J].智能系统学报,2024,19(2):462-471.[doi:10.11992/tis.202204047]
SHEN Jian,XIA Hongbin,LIU Yuan.Entity relation extraction model with dual relation prediction and feature fusion[J].CAAI Transactions on Intelligent Systems,2024,19(2):462-471.[doi:10.11992/tis.202204047]
点击复制
SHEN Jian,XIA Hongbin,LIU Yuan.Entity relation extraction model with dual relation prediction and feature fusion[J].CAAI Transactions on Intelligent Systems,2024,19(2):462-471.[doi:10.11992/tis.202204047]
双关系预测与特征融合的实体关系抽取模型
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第2期
页码:
462-471
栏目:
人工智能院长论坛
出版日期:
2024-03-05
- Title:
- Entity relation extraction model with dual relation prediction and feature fusion
- Keywords:
- entity relation extraction; relational triple; BERT pretrained model; dual relation prediction; pointer network; feature fusion; gated linear unit; conditional layer normalization
- 分类号:
- TP391
- 文献标志码:
- 2023-11-13
- 摘要:
- 现有分阶段解码的实体关系抽取模型仍存在着阶段间特征融合不充分的问题，会增大曝光偏差对抽取性能的影响。为此，提出一种双关系预测和特征融合的实体关系抽取模型（entity relation extraction model with dual relation prediction and feature fusion，DRPFF），该模型使用预训练的基于Transformer的双向编码表示模型（bidirectional encoder representation from transformers, BERT）对文本进行编码，并设计两阶段的双关系预测结构来减少抽取过程中错误三元组的生成。在阶段间通过门控线性单元（gated linear unit，GLU）和条件层规范化（conditional layer normalization，CLN）组合的结构来更好地融合实体之间的特征。在NYT和WebNLG这2个公开数据集上的试验结果表明，该模型相较于基线方法取得了更好的效果。
- Abstract:
- The staged decoding entity relation extraction model still has an insufficient feature fusion problem between stages, which increases the impact of exposure bias on the extraction performance. Herein, we propose a new entity relation extraction model with dual relation prediction and feature fusion (DRPFF). DRPFF uses a pretrained model of bidirectional encoder representation from transformers to encode texts, and a two-stage dual relation prediction structure is developed to reduce the false triples’ generation. Between stages, a structure combining gated linear units and conditional layer normalization is utilized to fuse features better between entities. Experimental findings on two public datasets, NYT and WebNLG, demonstrate that the presented method has better results than the baseline methods.
- 参考文献/References:
-
[1] 张勇, 高大林, 巩敦卫, 等. 用于关系抽取的注意力图长短时记忆神经网络[J]. 智能系统学报, 2021, 16(3): 518–527
ZHANG Yong, GAO Dalin, GONG Dunwei, et al. Attention graph long short-term memory neural network for relation extraction[J]. CAAI transactions on intelligent systems, 2021, 16(3): 518–527
[2] 杨志豪, 洪莉, 林鸿飞, 等. 基于支持向量机的生物医学文献蛋白质关系抽取[J]. 智能系统学报, 2008, 3(4): 361–369
YANG Zhihao, HONG Li, LIN Hongfei, et al. Extraction of information on protein-protein interaction from biomedical literatures using an SVM[J]. CAAI transactions on intelligent systems, 2008, 3(4): 361–369
[3] 范智渊, 何璇, 梁品, 等. 中文医学文献的实体关系提取研究及在糖尿病医学文献中的应用[J]. 生物医学工程学杂志, 2021, 38(3): 563–573
FAN Zhiyuan, HE Xuan, LIANG Pin, et al. Research on entity relationship extraction of Chinese medical literature and application in diabetes medical literature[J]. Journal of biomedical engineering, 2021, 38(3): 563–573
[4] KAMBHATLA N. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations[C]//Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions. Morristown: Association for Computational Linguistics, 2004: 178-181.
[5] ZHAO Shubin, GRISHMAN R. Extracting relations with integrated information using kernel methods[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics-ACL ’05. Morristown: Association for Computational Linguistics, 2005: 419-426.
[6] SOARES L B, FITZGERALD N, LING J, et al. Matching the blanks: distributional similarity for relation learning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 2895-2905.
[7] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000-6010.
[8] CHUNG Y A, ZHU Chenguang, ZENG M. SPLAT: speech-language joint pre-training for spoken language understanding[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2021: 4171-4186.
[9] 罗欣, 陈艳阳, 耿昊天, 等. 基于深度强化学习的文本实体关系抽取方法[J]. 电子科技大学学报, 2022, 51(1): 91–99
LUO Xin, CHEN Yanyang, GENG Haotian, et al. Entity relationship extraction from text data based on deep reinforcement learning[J]. Journal of University of Electronic Science and Technology of China, 2022, 51(1): 91–99
[10] ZHENG Suncong, WANG Feng, BAO Hongyun, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 1227–1236.
[11] 苗琳, 张英俊, 谢斌红, 等. 基于图神经网络的联合实体关系抽取[J]. 计算机应用研究, 2022, 39(2): 424–431
MIAO Lin, ZHANG Yingjun, XIE Binhong, et al. Joint entity relation extraction based on graph neural network[J]. Application research of computers, 2022, 39(2): 424–431
[12] WEI Zhepei, SU Jianlin, WANG Yue, et al. A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 1476-1488.
[13] WANG Yucheng, YU Bowen, ZHANG Yueyang, et al. TPLinker: single-stage joint extraction of entities and relations through token pair linking[C]//Proceedings of the 28th International Conference on Computational Linguistics. Stroudsburg: International Committee on Computational Linguistics, 2020: 1572–1582.
[14] DAUPHIN Y N, FAN A, AULI M, et al. Language modeling with gated convolutional networks[C]// Proceedings of the 34th International Conference on Machine Learning. Sydney: ICML, 2017: 933-941.
[15] 张龙辉, 尹淑娟, 任飞亮, 等. BSLRel: 基于二元序列标注的级联关系三元组抽取模型[J]. 中文信息学报, 2021, 35(6): 74–84
ZHANG Longhui, YIN Shujuan, REN Feiliang, et al. BSLRel: a binary sequence labeling based cascading relation triple extraction model[J]. Journal of Chinese information processing, 2021, 35(6): 74–84
[16] ZHANG Yue, YANG Jie. Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 1554-1564.
[17] LI Xiaonan, YAN Hang, QIU Xipeng, et al. FLAT: Chinese NER using flat-lattice transformer[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 6836-6842.
[18] LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields: probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the Eighteenth International Conference on Machine Learning. New York: ACM, 2001: 282-289.
[19] ZENG Xiangrong, HE Shizhu, ZENG Daojian, et al. Learning the extraction order of multiple relational facts in a sentence with reinforcement learning[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2019: 367–377.
[20] ZENG Xiangrong, ZENG Daojian, HE Shizhu, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 506–514.
[21] EBERTS M, ULGES A. Span-based joint entity and relation extraction with transformer pre-training[C]//Proceedings of the 24th European Conference on Artificial Intelligence. Santiago de Compostela: ECAI, 2020: 2006-2013.
[22] LI Xiaoya, FENG Jingrong, MENG Yuxian, et al. A unified MRC framework for named entity recognition[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 5849-5859.
[23] YU Bowen, ZHANG Zhenyu, SU Jianlin, et al. Joint extraction of entities and relations based on a novel decomposition strategy[C] // Proceedings of the 24th European Conference on Artificial Intelligence. Santiago de Compostela: ECAI, 2020: 2282-2289.
[24] 王泽儒, 柳先辉. 基于指针级联标注的中文实体关系联合抽取模型[J]. 武汉大学学报(理学版), 2022, 68(3): 304–310
WANG Zeru, LIU Xianhui. Joint model of Chinese entity-relation extraction based on a pointer cascade tagging strategy[J]. Journal of Wuhan University (natural science edition), 2022, 68(3): 304–310
[25] DE VRIES H, STRUB F, MARY J, et al. Modulating early visual processing by language[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6597-6607.
[26] RIEDEL S, YAO Limin, MCCALLUM A. Modeling relations and their mentions without labeled text[C]//Proceedings of the 2010th European Conference on Machine Learning and Knowledge Discovery in Databases-Volume Part III. New York: ACM, 2010: 148-163.
[27] GARDENT C, SHIMORINA A, NARAYAN S, et al. Creating training corpora for NLG micro-planners[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 179–188.
[28] KINGMA D P, BA J L. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego: ICLR, 2015.
[29] FU T J, LI P H, MA Weiyun. GraphRel: modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 1409–1418.
备注/Memo
收稿日期:2022-04-29。
基金项目:国家自然科学基金项目（61972182）.
作者简介:沈健，硕士研究生，主要研究方向为自然语言处理、实体关系抽取。E-mail：1452112297@qq.com;夏鸿斌，教授， 博士，江苏省特色化软件人才培养专委会委员，江南大学人工智能与计算机学院副院长。主要研究方向为大数据分析与处理、个性化推荐系统、自然语言处理。主持工信部、江苏省科研项目2项，参加国家科技支撑项目1项、江苏省自然科学基金重点项目1项。获江苏省教育成果二等奖、教育部科技成果奖、中国商业联合会科技进步特等奖。发表学术论文20余篇。E-mail：hbxia@jiangnan.edu.cn;刘渊，教授，博士生导师，中国网络空间安全协会会员，江南大学人工智能与计算机学院院长，主要研究方向为网络安全、社交网络。作为项目负责人完成省部级科研项目多项。获中国商业联合会科技奖特等奖、无锡市有突出贡献的中青年专家称号。发表学术论文40余篇。E-mail：lyuan1800@sina.com
通讯作者:夏鸿斌. E-mail：hbxia@jiangnan.edu.cn
更新日期/Last Update:
1900-01-01