[1]胡峰,杨新瑞,汤成富,等.基于自适应损失函数的句子级远程监督关系抽取[J].智能系统学报,2024,19(3):697-706.[doi:10.11992/tis.202205034]
HU Feng,YANG Xinrui,TANG Chengfu,et al.Sentence-level distant supervision relation extraction based on self-adaptive loss function[J].CAAI Transactions on Intelligent Systems,2024,19(3):697-706.[doi:10.11992/tis.202205034]
点击复制
HU Feng,YANG Xinrui,TANG Chengfu,et al.Sentence-level distant supervision relation extraction based on self-adaptive loss function[J].CAAI Transactions on Intelligent Systems,2024,19(3):697-706.[doi:10.11992/tis.202205034]
基于自适应损失函数的句子级远程监督关系抽取
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第3期
页码:
697-706
栏目:
学术论文—自然语言处理与理解
出版日期:
2024-05-05
- Title:
- Sentence-level distant supervision relation extraction based on self-adaptive loss function
- Keywords:
- natural language processing; information extraction; relation extraction; distant supervision; noise separation; noise label; negative training; self-adaptive loss function
- 分类号:
- TP391
- 文献标志码:
- 2023-10-09
- 摘要:
- 远程监督关系抽取是一种关系抽取方法，现有方法主要采用多实例学习，在具有相同实体对的样例包上进行关系抽取。但是，包级方法只能缓解却并不能完全解决错误标签问题。基于此，文中首先分析了干净数据和噪声数据的分布，提出了一种新的自适应损失函数；在此基础上，提出了一种基于自适应损失函数的句子级远程监督关系抽取方法。在公开数据集NYT-10以及基于TACRED的合成数据集上的实验结果表明：文中提出的方法优于对比文献中的方法，能够更有效地区分错误标签噪声样例和干净样例，提高了句子级远程监督关系抽取的准确率。
- Abstract:
- Distant supervision relation extraction is a kind of relation extraction method. The existing methods, which mainly employ multi-instance learning and relation extraction, are conducted in the sample bag that contains the same entity pair. However, the bag-level method can only alleviate but cannot completely solve the problem of wrong labeling. Therefore, herein, the distribution of clean data and noise data is analyzed, proposing a new self-adaptive loss function. On this basis, a method for sentence-level distant supervision relation extraction based on self-adaptive loss function is given. The experimental results obtained on the public dataset NYT-10 and the TACRED-based synthetic dataset show that the proposed method is better than that given in the compared studies. It can distinguish the wrongly labeled noise samples from the clean samples more effectively, improving the accuracy of sentence-level distant supervision relation extraction.
- 参考文献/References:
-
[1] 张涛, 贾真, 李天瑞, 等. 基于知识库的开放领域问答系统[J]. 智能系统学报, 2018, 13(4): 557–563
ZHANG Tao, JIA Zhen, LI Tianrui, et al. Open-domain question-answering system based on large-scale knowledge base[J]. CAAI transactions on intelligent systems, 2018, 13(4): 557–563
[2] MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]//Conference of Association for Computational Linguistics. Stroudsburg: ACL, 2009: 1003–1011.
[3] RIEDEL S, YAO Limin, MCCALLUM A. Modeling relations and their mentions without labeled text[C]//Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases-Volume Part III. New York: ACM, 2010: 148–163.
[4] YAO Xuchen, VAN DURME B. Information extraction over structured data: question answering with freebase[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2014: 956–966.
[5] XU Kun, REDDY S, FENG Yansong, et al. Question answering on freebase via relation extraction and textual evidence[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2016: 2326–2336.
[6] JIA Wei, DAI Dai, XIAO Xinyan, et al. ARNOR: attention regularization based noise reduction for distant supervision relation classification[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 1399–1408.
[7] FENG Jun, HUANG Minlie, ZHAO Li, et al. Reinforcement learning for relation classification from noisy data[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New Orleans: AAAI, 2018, 32(1): 5779–5786.
[8] MA Ruotian, GUI Tao, LI Linyang, et al. SENT: sentence-level distant relation extraction via negative training[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2021: 6201–6213.
[9] KIM Y, YIM J, YUN J, et al. NLNL: negative learning for noisy labels[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 101–110.
[10] HOFFMANN R, ZHANG Congle, LING Xiao, et al. Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Portland: ACM, 2011: 541–550.
[11] SURDEANU M, TIBSHIRANI J, NALLAPATI R, et al. Multi-instance multi-label learning for relation extraction[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Jeju Island: ACM, 2012: 455–465.
[12] LIN Yankai, SHEN Shiqi, LIU Zhiyuan, et al. Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2016: 2124–2133.
[13] YE Zhixiu, LING Zhenhua. Distant supervision relation extraction with intra-bag and inter-bag attentions[C]//Proceedings of the 2019 Conference of the North. Stroudsburg: Association for Computational Linguistics, 2019: 2810–2819.
[14] LI Yang, LONG Guodong, SHEN Tao, et al. Self-attention enhanced selective gate with entity-aware embedding for distantly supervised relation extraction[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020, 34(5): 8269–8276.
[15] LIU Tianyu, WANG Kexiang, CHANG Baobao, et al. A soft-label method for noise-tolerant distantly supervised relation extraction[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 1790–1795.
[16] QIN Pengda, XU Weiran, WANG W Y. Robust distant supervision relation extraction via deep reinforcement learning[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2018: 2137–2147.
[17] SHANG Yuming, HUANG Heyan, MAO Xianling, et al. Are noisy sentences useless for distant supervised relation extraction? [C]//Proceedings of the AAAI Conference on Artificial Intelligen. New York: AAAI, 2020, 34(5): 8799–8806.
[18] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2999–3007.
[19] TAN Qingyu, HE Ruidan, BING Lidong, et al. Document-level relation extraction with adaptive focal loss and knowledge distillation[C]//Findings of the Association for Computational Linguistics: ACL 2022. Stroudsburg: Association for Computational Linguistics, 2022: 1–10.
[20] 彭正阳, 吕立, 于碧辉. 基于动态损失函数的远程监督关系抽取[J]. 小型微型计算机系统, 2021, 42(2): 251–255
PENG Zhengyang, LYV Li, YU Bihui. Dynamic loss function for distant supervision relation extraction[J]. Journal of Chinese computer systems, 2021, 42(2): 251–255
[21] ZHANG Yuhao, ZHONG V, CHEN Danqi, et al. Position-aware attention and supervised data improve slot filling[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 35–45.
[22] ZENG Daojian, LIU Kang, LAI Siwei, et al. Relation classification via convolutional deep neural network[C]// COLING 2014—25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers. Stroudsburg: Association for Computational Linguistics, 2014: 2335–2344.
[23] ZENG Daojian, LIU Kang, CHEN Yubo, et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2015: 1753–1762.
[24] ZHANG S, ZHENG D, HU X, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation. Stroudsburg: Association for Computational Linguistics, 2015: 73–78.
[25] DEVLIN J, CHANG Mingwei, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018–10–11)[2022–05–23]. http://arxiv.org/abs/1810.04805.
[26] 张勇, 高大林, 巩敦卫, 等. 用于关系抽取的注意力图长短时记忆神经网络[J]. 智能系统学报, 2021, 16(3): 518–527
ZHANG Yong, GAO Dalin, GONG Dunwei, et al. Attention graph long short-term memory neural network for relation extraction[J]. CAAI transactions on intelligent systems, 2021, 16(3): 518–527
- 相似文献/References:
-
[1]李 蕾,周延泉,钟义信.基于语用的自然语言处理研究与应用初探[J].智能系统学报,2006,1(2):1.
LI Lei,ZHOU Yan-quan,ZHONG Yi-xin.Pragmatic Information Based NLP Research and Application[J].CAAI Transactions on Intelligent Systems,2006,1():1.
[2]贾真,何大可,杨燕,等.基于弱监督学习的中文网络百科关系抽取[J].智能系统学报,2015,10(1):113.[doi:10.10.3969/j.issn.1673-4785.201311017]
JIA Zhen,HE Dake,YANG Yan,et al.Relation extraction from Chinese online encyclopedia based on weakly supervised learnin[J].CAAI Transactions on Intelligent Systems,2015,10():113.[doi:10.10.3969/j.issn.1673-4785.201311017]
[3]王定桥,李卫华,杨春燕.从用户需求语句建立问题可拓模型的研究[J].智能系统学报,2015,10(6):865.[doi:10.11992/tis.201507038]
WANG Dingqiao,LI Weihua,YANG Chunyan.Research on building an extension model from user requirements[J].CAAI Transactions on Intelligent Systems,2015,10():865.[doi:10.11992/tis.201507038]
[4]李德毅.AI——人类社会发展的加速器[J].智能系统学报,2017,12(5):583.[doi:10.11992/tis.201710016]
LI Deyi.Artificial intelligence:an accelerator for the development of human society[J].CAAI Transactions on Intelligent Systems,2017,12():583.[doi:10.11992/tis.201710016]
[5]陈培,景丽萍.融合语义信息的矩阵分解词向量学习模型[J].智能系统学报,2017,12(5):661.[doi:10.11992/tis.201706012]
CHEN Pei,JING Liping.Word representation learning model using matrix factorization to incorporate semantic information[J].CAAI Transactions on Intelligent Systems,2017,12():661.[doi:10.11992/tis.201706012]
[6]张森,张晨,林培光,等.基于用户查询日志的网络搜索主题分析[J].智能系统学报,2017,12(5):668.[doi:10.11992/tis.201706096]
ZHANG Sen,ZHANG Chen,LIN Peiguang,et al.Web search topic analysis based on user search query logs[J].CAAI Transactions on Intelligent Systems,2017,12():668.[doi:10.11992/tis.201706096]
[7]王一成,万福成,马宁.融合多层次特征的中文语义角色标注[J].智能系统学报,2020,15(1):107.[doi:10.11992/tis.201910012]
WANG Yicheng,WAN Fucheng,MA Ning.Chinese semantic role labeling with multi-level linguistic features[J].CAAI Transactions on Intelligent Systems,2020,15():107.[doi:10.11992/tis.201910012]
[8]毛明毅,吴晨,钟义信,等.加入自注意力机制的BERT命名实体识别模型[J].智能系统学报,2020,15(4):772.[doi:10.11992/tis.202003003]
MAO Mingyi,WU Chen,ZHONG Yixin,et al.BERT named entity recognition model with self-attention mechanism[J].CAAI Transactions on Intelligent Systems,2020,15():772.[doi:10.11992/tis.202003003]
[9]胡康,何思宇,左敏,等.基于CNN-BLSTM的化妆品违法违规行为分类模型[J].智能系统学报,2021,16(6):1151.[doi:10.11992/tis.202104001]
HU Kang,HE Siyu,ZUO Min,et al.Classification model for judging illegal and irregular behavior for cosmetics based on CNN-BLSTM[J].CAAI Transactions on Intelligent Systems,2021,16():1151.[doi:10.11992/tis.202104001]
[10]喻波,王志海,孙亚东,等.非结构化文档敏感数据识别与异常行为分析[J].智能系统学报,2021,16(5):932.[doi:10.11992/tis.202104028]
YU Bo,WANG Zhihai,SUN Yadong,et al.Unstructured document sensitive data identification and abnormal behavior analysis[J].CAAI Transactions on Intelligent Systems,2021,16():932.[doi:10.11992/tis.202104028]
[11]于润羽,杜军平,薛哲,等.面向科技学术会议的命名实体识别研究[J].智能系统学报,2022,17(1):50.[doi:10.11992/tis.202107010]
YU Runyu,DU Junping,XUE Zhe,et al.Research on named entity recognition for scientific and technological conferences[J].CAAI Transactions on Intelligent Systems,2022,17():50.[doi:10.11992/tis.202107010]
[12]黄河燕,刘啸.面向新领域的事件抽取研究综述[J].智能系统学报,2022,17(1):201.[doi:10.11992/tis.202109045]
HUANG Heyan,LIU Xiao.A survey on event extraction in new domains[J].CAAI Transactions on Intelligent Systems,2022,17():201.[doi:10.11992/tis.202109045]
备注/Memo
收稿日期:2022-05-23。
基金项目:国家重点研发计划项目 (2018YFC0832102)；重庆市教委重点合作项目(HZ2021008); 重庆市自然科学基金项目 (cstc2021jcyj-msxmX0849).
作者简介:胡峰，博士，教授，主要研究方向为粗糙集、粒计算、数据挖掘。主持和参与国家自然科学基金项目4项，参与科技部重点研发计划项目3项，作为参与者获吴文俊人工智能科学技术奖、重庆市自然科学奖各1项，发表学术论文40余篇。 E-mail: hufeng@cqupt. edu.cn;杨新瑞，硕士研究生，主要研究方向为自然语言处理、信息抽取、数据挖掘。E-mail: 1158737962@qq.com;汤成富，硕士研究生，主要研究方向为计算机视觉、图像识别、数据挖掘。E-mail: tangcfmail@163.com
通讯作者:胡峰. E-mail: hufeng@cqupt.edu.cn
更新日期/Last Update:
1900-01-01