[1]HU Feng,YANG Xinrui,TANG Chengfu,et al.Sentence-level distant supervision relation extraction based on self-adaptive loss function[J].CAAI Transactions on Intelligent Systems,2024,19(3):697-706.[doi:10.11992/tis.202205034]
Copy
Sentence-level distant supervision relation extraction based on self-adaptive loss function
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
19
Number of periods:
2024 3
Page number:
697-706
Column:
学术论文—自然语言处理与理解
Public date:
2024-05-05
- Title:
- Sentence-level distant supervision relation extraction based on self-adaptive loss function
- Keywords:
- natural language processing; information extraction; relation extraction; distant supervision; noise separation; noise label; negative training; self-adaptive loss function
- CLC:
- TP391
- DOI:
- 10.11992/tis.202205034
- Abstract:
- Distant supervision relation extraction is a kind of relation extraction method. The existing methods, which mainly employ multi-instance learning and relation extraction, are conducted in the sample bag that contains the same entity pair. However, the bag-level method can only alleviate but cannot completely solve the problem of wrong labeling. Therefore, herein, the distribution of clean data and noise data is analyzed, proposing a new self-adaptive loss function. On this basis, a method for sentence-level distant supervision relation extraction based on self-adaptive loss function is given. The experimental results obtained on the public dataset NYT-10 and the TACRED-based synthetic dataset show that the proposed method is better than that given in the compared studies. It can distinguish the wrongly labeled noise samples from the clean samples more effectively, improving the accuracy of sentence-level distant supervision relation extraction.
- References:
-
[1] 张涛, 贾真, 李天瑞, 等. 基于知识库的开放领域问答系统[J]. 智能系统学报, 2018, 13(4): 557–563
ZHANG Tao, JIA Zhen, LI Tianrui, et al. Open-domain question-answering system based on large-scale knowledge base[J]. CAAI transactions on intelligent systems, 2018, 13(4): 557–563
[2] MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]//Conference of Association for Computational Linguistics. Stroudsburg: ACL, 2009: 1003–1011.
[3] RIEDEL S, YAO Limin, MCCALLUM A. Modeling relations and their mentions without labeled text[C]//Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases-Volume Part III. New York: ACM, 2010: 148–163.
[4] YAO Xuchen, VAN DURME B. Information extraction over structured data: question answering with freebase[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2014: 956–966.
[5] XU Kun, REDDY S, FENG Yansong, et al. Question answering on freebase via relation extraction and textual evidence[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2016: 2326–2336.
[6] JIA Wei, DAI Dai, XIAO Xinyan, et al. ARNOR: attention regularization based noise reduction for distant supervision relation classification[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 1399–1408.
[7] FENG Jun, HUANG Minlie, ZHAO Li, et al. Reinforcement learning for relation classification from noisy data[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New Orleans: AAAI, 2018, 32(1): 5779–5786.
[8] MA Ruotian, GUI Tao, LI Linyang, et al. SENT: sentence-level distant relation extraction via negative training[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2021: 6201–6213.
[9] KIM Y, YIM J, YUN J, et al. NLNL: negative learning for noisy labels[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 101–110.
[10] HOFFMANN R, ZHANG Congle, LING Xiao, et al. Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Portland: ACM, 2011: 541–550.
[11] SURDEANU M, TIBSHIRANI J, NALLAPATI R, et al. Multi-instance multi-label learning for relation extraction[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Jeju Island: ACM, 2012: 455–465.
[12] LIN Yankai, SHEN Shiqi, LIU Zhiyuan, et al. Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2016: 2124–2133.
[13] YE Zhixiu, LING Zhenhua. Distant supervision relation extraction with intra-bag and inter-bag attentions[C]//Proceedings of the 2019 Conference of the North. Stroudsburg: Association for Computational Linguistics, 2019: 2810–2819.
[14] LI Yang, LONG Guodong, SHEN Tao, et al. Self-attention enhanced selective gate with entity-aware embedding for distantly supervised relation extraction[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020, 34(5): 8269–8276.
[15] LIU Tianyu, WANG Kexiang, CHANG Baobao, et al. A soft-label method for noise-tolerant distantly supervised relation extraction[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 1790–1795.
[16] QIN Pengda, XU Weiran, WANG W Y. Robust distant supervision relation extraction via deep reinforcement learning[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2018: 2137–2147.
[17] SHANG Yuming, HUANG Heyan, MAO Xianling, et al. Are noisy sentences useless for distant supervised relation extraction? [C]//Proceedings of the AAAI Conference on Artificial Intelligen. New York: AAAI, 2020, 34(5): 8799–8806.
[18] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2999–3007.
[19] TAN Qingyu, HE Ruidan, BING Lidong, et al. Document-level relation extraction with adaptive focal loss and knowledge distillation[C]//Findings of the Association for Computational Linguistics: ACL 2022. Stroudsburg: Association for Computational Linguistics, 2022: 1–10.
[20] 彭正阳, 吕立, 于碧辉. 基于动态损失函数的远程监督关系抽取[J]. 小型微型计算机系统, 2021, 42(2): 251–255
PENG Zhengyang, LYV Li, YU Bihui. Dynamic loss function for distant supervision relation extraction[J]. Journal of Chinese computer systems, 2021, 42(2): 251–255
[21] ZHANG Yuhao, ZHONG V, CHEN Danqi, et al. Position-aware attention and supervised data improve slot filling[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 35–45.
[22] ZENG Daojian, LIU Kang, LAI Siwei, et al. Relation classification via convolutional deep neural network[C]// COLING 2014—25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers. Stroudsburg: Association for Computational Linguistics, 2014: 2335–2344.
[23] ZENG Daojian, LIU Kang, CHEN Yubo, et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2015: 1753–1762.
[24] ZHANG S, ZHENG D, HU X, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation. Stroudsburg: Association for Computational Linguistics, 2015: 73–78.
[25] DEVLIN J, CHANG Mingwei, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018–10–11)[2022–05–23]. http://arxiv.org/abs/1810.04805.
[26] 张勇, 高大林, 巩敦卫, 等. 用于关系抽取的注意力图长短时记忆神经网络[J]. 智能系统学报, 2021, 16(3): 518–527
ZHANG Yong, GAO Dalin, GONG Dunwei, et al. Attention graph long short-term memory neural network for relation extraction[J]. CAAI transactions on intelligent systems, 2021, 16(3): 518–527
- Similar References:
Memo
-
Last Update:
1900-01-01