[1]WANG Yuhui,DU Junping,SHAO Yingxia.An intellectual property entity recognition method based on Transformer and technological word information[J].CAAI Transactions on Intelligent Systems,2023,18(1):186-193.[doi:10.11992/tis.202203036]
Copy

An intellectual property entity recognition method based on Transformer and technological word information

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 1 Page number: 186-193 Column: 吴文俊人工智能科学技术奖论坛 Public date: 2023-01-05
Title:
An intellectual property entity recognition method based on Transformer and technological word information
Author(s):
WANG Yuhui1; 2;  DU Junping1; 2;  SHAO Yingxia1; 2
1. School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing 100876, China
Keywords:
entity recognition named in Chinese; intellectual property; Transformer encoder; information fusion; vector representation; science and technology big data; patent; deep learning
CLC:
TP391
DOI:
10.11992/tis.202203036
Abstract:
Patent text contains abundant entity information, from which the intellectual property (IP) entity information containing key information can be extracted through named entity recognition, which helps researchers understand patent content faster. For the existing named entity extraction method, the semantic information at the word level brought by a change in technical words is difficult to fully use. In this paper, the IP entity information extraction method based on Transformer and technical word information is proposed, which provides exact word vector representation based on the BERT language model. In the process of word vector generation, this method improves the representation ability of IP entities by adding the technical word information extracted by iterated dilated convolution neural network. Finally, the Transformer encoder with relative position coding is used to learn the deep semantic information of the text from the word vector sequence, realizing the prediction of the entity label. Experimental results on public and annotated patent datasets show that this method improves entity recognition accuracy.
References:
[1] 杨佳鑫, 杜军平, 邵蓥侠, 等. 面向知识产权的科技资源画像构建方法[J]. 软件学报, 2022, 33(4): 1439–1450
YANG Jiaxin, DU Junping, SHAO Yingxia, et al. Construction method of intellectual-property-oriented scientific and technological resources portrait[J]. Journal of software, 2022, 33(4): 1439–1450
[2] WANG Yuhui, DU Junping, SHAO Yingxia, et al. A patent text classification method based on phrase-context fusion feature[C]//Proceedings of 2021 Chinese Intelligent Automation Conference. Singapore: Springer, 2022: 157-164.
[3] XU Mingying, DU Junping, XUE Zhe, et al. A scientific research topic trend prediction model based on multi-LSTM and graph convolutional network[J]. International journal of intelligent systems, 2022, 37(9): 6331–6353.
[4] KOWSARI K, JAFARI M K, MOJTABA H, et al. Text classification algorithms: A survey[J]. Information, 2019, 10(4): 150.
[5] DEVLIN J, CHANG MING-WEI, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018?10?11)[2022?05?23].https://arxiv.org/abs/1810.04805.
[6] KOU Feifei, DU Junping, HE Yijiang, et al. Social network search based on semantic analysis and learning[J]. CAAI transactions on intelligence technology, 2016, 1(4): 293–302.
[7] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017: 30.
[8] CHEN Hui, LIN Zijia, DING Guiguang, et al. GRN: gated relation network to enhance convolutional neural network for named entity recognition[J]. Proceedings of the AAAI conference on artificial intelligence, 2019, 33(1): 6236–6243.
[9] SUN Bo, DU Junping, GAO Tian. Study on the improvement of K-nearest-neighbor algorithm[C]//2009 International Conference on Artificial Intelligence and Computational Intelligence. Shanghai: IEEE, 2009: 390-393.
[10] CHEN Tianci, LUO Mengfei, FU Hao, et al. Application of NER and association rules to traditional Chinese medicine patent mining[C]//2020 International Conferences on Internet of Things and IEEE Green Computing and Communications and IEEE Cyber, Physical and Social Computing and IEEE Smart Data and IEEE Congress on Cybermatics. Rhodes: IEEE, 2020: 767?772.
[11] XUE Zhe, DU Junping, DU Dawei, et al. Deep low-rank subspace ensemble for multi-view clustering[J]. Information sciences, 2019, 482: 210–227.
[12] FANG Yuke, DENG Weihong, DU Junping, et al. Identity-aware CycleGAN for face photo-sketch synthesis and recognition[J]. Pattern recognition, 2020, 102: 107249.
[13] KRESTEL R, CHIKKAMATH R, HEWEL C, et al. A survey on deep learning for patent analysis[J]. World patent information, 2021, 65: 102035.
[14] WANG Yu, LI Yun, ZHU Ziye, et al. SC-NER: a sequence-to-sequence model with sentence classification for named entity recognition[M]//Advances in Knowledge Discovery and Data Mining. Cham: Springer International Publishing, 2019: 198?209.
[15] SAAD F, ARAS H, HACKL-SOMMER R. Improving named entity recognition for biomedical and patent data using Bi-LSTM deep neural network models[M]//Natural Language Processing and Information Systems. Cham: Springer International Publishing, 2020: 25?36.
[16] ZHAI Zenan, NGUYEN D Q, AKHONDI S A, et al. Improving chemical named entity recognition in patents with contextualized word embeddings[EB/OL]. (2019?07?05) [2022?05?23].https://arxiv.org/abs/1907.02679.
[17] ZHANG Yue, YANG Jie. Chinese NER using lattice LSTM[EB/OL]. (2018?05?05)[2022?05?23].https://arxiv.org/abs/1805.02023.
[18] YAN Xingyu, XIONG Xiaofan, CHENG Xiufeng, et al. HMM-BiMM: hidden Markov model-based word segmentation via improved bi-directional maximal matching algorithm[J]. Computers & electrical engineering, 2021, 94: 107354.
[19] ZHAO Hongke, LIU Qi, ZHU Hengshu, et al. A sequential approach to market state modeling and analysis in online P2P lending[J]. IEEE transactions on systems, man, and cybernetics:systems, 2018, 48(1): 21–33.
[20] ALZAIDY R, CARAGEA C, GILES C L. Bi-LSTM-CRF sequence labeling for keyphrase extraction from scholarly documents[C]//WWW’19: The World Wide Web Conference. New York: ACM, 2019: 2551?2557.
[21] JIN Yanliang, XIE Jinfei, GUO Weisi, et al. LSTM-CRF neural network with gated self attention for Chinese NER[J]. IEEE access, 2019, 7: 136694–136703.
[22] LI Xiaonan, YAN Hang, QIU Xipeng, et al. FLAT: Chinese NER using flat-lattice transformer[EB/OL]. (2020-04-24)[2022-05-23].https://arxiv.org/abs/2004.11795.
[23] DAI Zihang, YANG Zhilin, YANG Yiming, et al. Transforme-xl: Attentive language models beyond a fixed-length context[EB/OL]. (2019?01?09)[2022?05?23].https://arxiv.org/abs/1901.02860.
[24] YAN Hang, DENG Bocao, LI Xiaonan, et al. TENER: adapting transformer encoder for named entity recognition[EB/OL]. (2019-11-10)[2022-05-23].https://arxiv.org/abs/1911.04474.
[25] YIN Xunwei, ZHENG Shuang, WANG Quanmin. Fine-grained Chinese named entity recognition based on RoBERTa-WWM-BiLSTM-CRF model[C]//2021 6th International Conference on Image, Vision and Computing. Qingdao: IEEE, 2021: 408?413.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems