[1]DI Jian,LIU Junhua,CAO Jingang.An improved HiNT text retrieval model using BERT and coverage mechanism[J].CAAI Transactions on Intelligent Systems,2024,19(3):719-727.[doi:10.11992/tis.202201020]
Copy

An improved HiNT text retrieval model using BERT and coverage mechanism

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 3 Page number: 719-727 Column: 学术论文—自然语言处理与理解 Public date: 2024-05-05
Title:
An improved HiNT text retrieval model using BERT and coverage mechanism
Author(s):
DI Jian1; 2;  LIU Junhua1; 2;  CAO Jingang1; 2
1. School of Control and Computer Engineering, North China Electric Power University, Baoding 071003, China;
2. Engineering Research Center of Intelligent Computing for Complex Energy Systems, Ministry of Education, Baoding 071003, China
Keywords:
bidirectional encoder representations from transformers; hierarchical neural matching model; coverage mechanism; text retrieval; semantic representation; feature extraction; natural language processing; similarity; multigranularity
CLC:
TP311
DOI:
10.11992/tis.202201020
Abstract:
To effectively improve the accuracy of text semantic retrieval, an improved hierarchical neural matching model is proposed, which can solve the problems of text ambiguity and polysemy when using text retrieval models to measure the relevance of queries and documents. The model first extracts key subject words from each segment of the document and then encodes them into multiple dense semantic vectors using the BERT model. Afterward, the local matching layer introduced with the coverage mechanism is used for processing so that the model can calculate the correlation according to the local segment-level granularity and the global document-level granularity of the document and improve the retrieval accuracy. The proposed model is compared with multiple retrieval models on the MS MARCO and webtext2019zh datasets, and the optimal results obtained verify the effectiveness of the proposed model.
References:
[1] STADIG I, SVANBERG T. Overview of information retrieval in a hospital-based health technology assessment center in a Swedish Region[J]. International journal of technology assessment in health care, 2021, 37(1): e52.
[2] FAN Yixing, GUO Jiafeng, LAN Yanyan, et al. Modeling diverse relevance patterns in ad-hoc retrieval[C]//SIGIR ’18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Ann Arbor: ACM, 2018: 375–384.
[3] ROBERTSON S E, JONES K S. Relevance weighting of search terms[J]. Journal of the American society for information science, 1976, 27(3): 129–146.
[4] ROBERTSON S E, WALKER S, JONES S, et al. Okapi at TREC-3[EB/OL]. (2022–01–13)[2024–02–27]. https://api.semanticscholar.org/CorpusID:3946054.
[5] DATTA S, GANGULY D, ROY D, et al. Overview of the causality-driven adhoc information retrieval (CAIR) task at FIRE-2021[C]//Proceedings of the 13th Annual Meeting of the Forum for Information Retrieval Evaluation. Virtual Event: ACM, 2021: 25–27.
[6] 庞亮, 兰艳艳, 徐君, 等. 深度文本匹配综述[J]. 计算机学报, 2017, 40(4): 985–1003
PANG Liang, LAN Yanyan, XU Jun, et al. A survey on deep text matching[J]. Chinese journal of computers, 2017, 40(4): 985–1003
[7] HUANG Posen, HE Xiaodong, GAO Jianfeng, et al. Learning deep structured semantic models for web search using clickthrough data[C]//Proceedings of the 22nd ACM International Conference on Information & Knowledge Management. San Francisco: ACM, 2013: 2333–2338.
[8] SHEN Yelong, HE Xiaodong, GAO Jianfeng, et al. Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of the 23rd International Conference on World Wide Web. Seoul: ACM, 2014: 373–374.
[9] PALANGI H, DENG L, SHEN Y, et al. Semantic modelling with long short-tremmemory for infomation retrieval[EB/OL]. (2015–05–27)[2022–01–13]. https://arxiv.org/pdf/1412.6629.
[10] GUO Jiafeng, FAN Yixing, AI Qingyao, et al. A deep relevance matching model for ad-hoc retrieval[C]//Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. Indianapolis: ACM, 2016: 55–64.
[11] HUI Kai, YATES A, BERBERICH K, et al. PACRR: a position-aware neural IR model for relevance matching[EB/OL]. (2017–04–12)[2024–02–27]. http://arxiv.org/abs/1704.03940.pdf.
[12] HUI Kai, YATES A, BERBERICH K, et al. Co-PACRR: a context-aware neural IR model for ad-hoc retrieval[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. Marina Del Rey: ACM, 2018: 279–287.
[13] ALTNEL B, GANIZ M C. Semantic text classification: a survey of past and recent advances[J]. Information processing & management, 2018, 54(6): 1129–1153.
[14] MIKOLOV T, CHEN Kai, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. (2013–01–16)[2024–02–27]. http://arxiv.org/abs/1301.3781.pdf.
[15] LU Yiwei, YANG Ruopeng, JIANG Xuping, et al. Research on military event detection method based on BERT-BiGRU-attention[C]//2021 IEEE International Conference on Consumer Electronics and Computer Engineering. Guangzhou: IEEE, 2021: 1–5.
[16] DEVLIN J, CHANG Mingwei, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018–10–11)[2024–02–27]. http://arxiv.org/abs/1810.04805.pdf.
[17] 于润羽, 杜军平, 薛哲, 等. 面向科技学术会议的命名实体识别研究[J]. 智能系统学报, 2022, 17(1): 50–58
YU Runyu, DU Junping, XUE Zhe, et al. Research on named entity recognition for scientific and technological conferences[J]. CAAI transactions on intelligent systems, 2022, 17(1): 50–58
[18] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: ACM, 2017: 6000–6010.
[19] JIANG Teng, ZHANG Zehan, YANG Yupu. Modeling coverage with semantic embedding for image caption generation[J]. The visual computer, 2019, 35(11): 1655–1665.
[20] 蔡银琼, 范意兴, 郭嘉丰, 等. 基于多表达的第一阶段语义检索模型[J]. 计算机工程与应用, 2023, 59(4): 139–146
CAI Yinqiong, FAN Yixing, GUO Jiafeng, et al. Multi-representation model for the first-stage semantic retrieval[J]. Computer engineering and applications, 2023, 59(4): 139–146
[21] 巩轶凡, 刘红岩, 何军, 等. 带有覆盖率机制的文本摘要模型研究[J]. 计算机科学与探索, 2019, 13(2): 205–213
GONG Yifan, LIU Hongyan, HE Jun, et al. Research on text summarization model with coverage mechanism[J]. Journal of frontiers of computer science and technology, 2019, 13(2): 205–213
[22] SEE A, LIU P J, MANNING C D. Get to the point: summarization with pointer-generator networks[EB/OL]. (2017–04–14)[2024–02–07]. http://arxiv.org/abs/1704.04368.
[23] CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[EB/OL]. (2014–06–03)[2024–02–07]. http://arxiv.org/abs/1406.1078.pdf.
[24] NGUYEN T, ROSENBERG M, SONG X, et al. A human generated machine reading comprehension dataset[EB/OL]. (2018–10–31)[2024–02–07]. https://arxiv.org/pdf/1611.09268.pdf.
[25] DAI Zhuyun, CALLAN J. Context-aware term weighting for first stage passage retrieval[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 1533–1536.
[26] NOGUEIRA R, YANG Wei, LIN J, et al. Document expansion by query prediction[EB/OL]. (2019–04–07)[2024–02–07]. http://arxiv.org/abs/1904.08375.
[27] ZHAN Jingtao, MAO Jiaxin, LIU Yiqun, et al. RepBERT: contextualized text embeddings for first-stage retrieval[EB/OL]. (2020–06–28)[2024–02–07]. http://arxiv.org/abs/2006.15498.pdf.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems