[1]邸剑,刘骏华,曹锦纲.利用BERT和覆盖率机制改进的HiNT文本检索模型[J].智能系统学报,2024,19(3):719-727.[doi:10.11992/tis.202201020]
DI Jian,LIU Junhua,CAO Jingang.An improved HiNT text retrieval model using BERT and coverage mechanism[J].CAAI Transactions on Intelligent Systems,2024,19(3):719-727.[doi:10.11992/tis.202201020]
点击复制
DI Jian,LIU Junhua,CAO Jingang.An improved HiNT text retrieval model using BERT and coverage mechanism[J].CAAI Transactions on Intelligent Systems,2024,19(3):719-727.[doi:10.11992/tis.202201020]
利用BERT和覆盖率机制改进的HiNT文本检索模型
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第3期
页码:
719-727
栏目:
学术论文—自然语言处理与理解
出版日期:
2024-05-05
- Title:
- An improved HiNT text retrieval model using BERT and coverage mechanism
- Keywords:
- bidirectional encoder representations from transformers; hierarchical neural matching model; coverage mechanism; text retrieval; semantic representation; feature extraction; natural language processing; similarity; multigranularity
- 分类号:
- TP311
- 文献标志码:
- 2023-09-27
- 摘要:
- 为有效提升文本语义检索的准确度，本文针对当前文本检索模型衡量查询和文档的相关性时不能很好地解决文本歧义和一词多义等问题，提出一种基于改进的分层神经匹配模型(hierarchical neural matching model，HiNT)。该模型先对文档的各个段提取关键主题词，然后用基于变换器的双向编码器（bidirectional encoder representations from transformers, BERT）模型将其编码为多个稠密的语义向量，再利用引入覆盖率机制的局部匹配层进行处理，使模型可以根据文档的局部段级别粒度和全局文档级别粒度进行相关性计算，提高检索的准确率。本文提出的模型在MS MARCO和webtext2019zh数据集上与多个检索模型进行对比，取得了最优结果，验证了本文提出模型的有效性。
- Abstract:
- To effectively improve the accuracy of text semantic retrieval, an improved hierarchical neural matching model is proposed, which can solve the problems of text ambiguity and polysemy when using text retrieval models to measure the relevance of queries and documents. The model first extracts key subject words from each segment of the document and then encodes them into multiple dense semantic vectors using the BERT model. Afterward, the local matching layer introduced with the coverage mechanism is used for processing so that the model can calculate the correlation according to the local segment-level granularity and the global document-level granularity of the document and improve the retrieval accuracy. The proposed model is compared with multiple retrieval models on the MS MARCO and webtext2019zh datasets, and the optimal results obtained verify the effectiveness of the proposed model.
- 参考文献/References:
-
[1] STADIG I, SVANBERG T. Overview of information retrieval in a hospital-based health technology assessment center in a Swedish Region[J]. International journal of technology assessment in health care, 2021, 37(1): e52.
[2] FAN Yixing, GUO Jiafeng, LAN Yanyan, et al. Modeling diverse relevance patterns in ad-hoc retrieval[C]//SIGIR ’18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Ann Arbor: ACM, 2018: 375–384.
[3] ROBERTSON S E, JONES K S. Relevance weighting of search terms[J]. Journal of the American society for information science, 1976, 27(3): 129–146.
[4] ROBERTSON S E, WALKER S, JONES S, et al. Okapi at TREC-3[EB/OL]. (2022–01–13)[2024–02–27]. https://api.semanticscholar.org/CorpusID:3946054.
[5] DATTA S, GANGULY D, ROY D, et al. Overview of the causality-driven adhoc information retrieval (CAIR) task at FIRE-2021[C]//Proceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation. Virtual Event: ACM, 2021: 25–27.
[6] 庞亮, 兰艳艳, 徐君, 等. 深度文本匹配综述[J]. 计算机学报, 2017, 40(4): 985–1003
PANG Liang, LAN Yanyan, XU Jun, et al. A survey on deep text matching[J]. Chinese journal of computers, 2017, 40(4): 985–1003
[7] HUANG Posen, HE Xiaodong, GAO Jianfeng, et al. Learning deep structured semantic models for web search using clickthrough data[C]//Proceedings of the 22nd ACM International Conference on Information & Knowledge Management. San Francisco: ACM, 2013: 2333–2338.
[8] SHEN Yelong, HE Xiaodong, GAO Jianfeng, et al. Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of the 23rd International Conference on World Wide Web. Seoul: ACM, 2014: 373–374.
[9] PALANGI H, DENG L, SHEN Y, et al. Semantic modelling with long short-tremmemory for infomation retrieval[EB/OL]. (2015–05–27)[2022–01–13]. https://arxiv.org/pdf/1412.6629.
[10] GUO Jiafeng, FAN Yixing, AI Qingyao, et al. A deep relevance matching model for ad-hoc retrieval[C]//Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. Indianapolis: ACM, 2016: 55–64.
[11] HUI Kai, YATES A, BERBERICH K, et al. PACRR: a position-aware neural IR model for relevance matching[EB/OL]. (2017–04–12)[2024–02–27]. http://arxiv.org/abs/1704.03940.pdf.
[12] HUI Kai, YATES A, BERBERICH K, et al. Co-PACRR: a context-aware neural IR model for ad-hoc retrieval[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. Marina Del Rey: ACM, 2018: 279–287.
[13] ALTNEL B, GANIZ M C. Semantic text classification: a survey of past and recent advances[J]. Information processing & management, 2018, 54(6): 1129–1153.
[14] MIKOLOV T, CHEN Kai, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. (2013–01–16)[2024–02–27]. http://arxiv.org/abs/1301.3781.pdf.
[15] LU Yiwei, YANG Ruopeng, JIANG Xuping, et al. Research on military event detection method based on BERT-BiGRU-attention[C]//2021 IEEE International Conference on Consumer Electronics and Computer Engineering. Guangzhou: IEEE, 2021: 1–5.
[16] DEVLIN J, CHANG Mingwei, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018–10–11)[2024–02–27]. http://arxiv.org/abs/1810.04805.pdf.
[17] 于润羽, 杜军平, 薛哲, 等. 面向科技学术会议的命名实体识别研究[J]. 智能系统学报, 2022, 17(1): 50–58
YU Runyu, DU Junping, XUE Zhe, et al. Research on named entity recognition for scientific and technological conferences[J]. CAAI transactions on intelligent systems, 2022, 17(1): 50–58
[18] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: ACM, 2017: 6000–6010.
[19] JIANG Teng, ZHANG Zehan, YANG Yupu. Modeling coverage with semantic embedding for image caption generation[J]. The visual computer, 2019, 35(11): 1655–1665.
[20] 蔡银琼, 范意兴, 郭嘉丰, 等. 基于多表达的第一阶段语义检索模型[J]. 计算机工程与应用, 2023, 59(4): 139–146
CAI Yinqiong, FAN Yixing, GUO Jiafeng, et al. Multi-representation model for the first-stage semantic retrieval[J]. Computer engineering and applications, 2023, 59(4): 139–146
[21] 巩轶凡, 刘红岩, 何军, 等. 带有覆盖率机制的文本摘要模型研究[J]. 计算机科学与探索, 2019, 13(2): 205–213
GONG Yifan, LIU Hongyan, HE Jun, et al. Research on text summarization model with coverage mechanism[J]. Journal of frontiers of computer science and technology, 2019, 13(2): 205–213
[22] SEE A, LIU P J, MANNING C D. Get to the point: summarization with pointer-generator networks[EB/OL]. (2017–04–14)[2024–02–07]. http://arxiv.org/abs/1704.04368.
[23] CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[EB/OL]. (2014–06–03)[2024–02–07]. http://arxiv.org/abs/1406.1078.pdf.
[24] NGUYEN T, ROSENBERG M, SONG X, et al. A human generated machine reading comprehension dataset[EB/OL]. (2018–10–31)[2024–02–07]. https://arxiv.org/pdf/1611.09268.pdf.
[25] DAI Zhuyun, CALLAN J. Context-aware term weighting for first stage passage retrieval[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 1533–1536.
[26] NOGUEIRA R, YANG Wei, LIN J, et al. Document expansion by query prediction[EB/OL]. (2019–04–07)[2024–02–07]. http://arxiv.org/abs/1904.08375.
[27] ZHAN Jingtao, MAO Jiaxin, LIU Yiqun, et al. RepBERT: contextualized text embeddings for first-stage retrieval[EB/OL]. (2020–06–28)[2024–02–07]. http://arxiv.org/abs/2006.15498.pdf.
备注/Memo
收稿日期:2022-01-13。
基金项目:中央高校基本科研业务费专项(2021MS085).
作者简介:邸剑，高级工程师，主要研究方向为人工智能及应用、物联网技术及应用、大数据与云计算。先后主研、参研科技项目20余项，获省部级科技进步奖2项，获授权发明专利1项。发表学术论文30余篇，参编教材1部。E-mail：dijian6880@163.com;刘骏华，硕士研究生，主要研究方向为深度学习、自然语言处理。E-mail：220192221061@ncepu.edu.cn;曹锦纲，讲师，博士，主要研究方向为图像处理和模式识别。发表学术论文 10 余篇。E-mail：caojg168@126.com
通讯作者:曹锦纲. E-mail：caojg168@126.com
更新日期/Last Update:
1900-01-01