[1]于润羽,李雅文,李昂.融合领域特征的科技学术会议语义相似性计算方法[J].智能系统学报,2022,17(4):737-743.[doi:10.11992/tis.202203050]
YU Runyu,LI Yawen,LI Ang.Semantic similarity computing for scientific and technological conferences[J].CAAI Transactions on Intelligent Systems,2022,17(4):737-743.[doi:10.11992/tis.202203050]
点击复制
YU Runyu,LI Yawen,LI Ang.Semantic similarity computing for scientific and technological conferences[J].CAAI Transactions on Intelligent Systems,2022,17(4):737-743.[doi:10.11992/tis.202203050]
融合领域特征的科技学术会议语义相似性计算方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第4期
页码:
737-743
栏目:
学术论文—自然语言处理与理解
出版日期:
2022-07-05
- Title:
- Semantic similarity computing for scientific and technological conferences
- Keywords:
- science and technological conference; deep learning; natural language processing; semantic learning; knowledge extraction; semantic similarity; pre-training model; siamese network
- 分类号:
- TP391
- 摘要:
- 针对目前的语义文本相似度计算方法难以准确估计科技学术会议语义相似性的问题，提出了一种融合领域特征的科技学术会议语义相似度计算方法（siamese-BERT semantic similarity calculation algorithm fused with domain feature, SBFD）。通过实体识别和关键词抽取等方式获取会议的领域特征信息，将其作为特征与会议信息共同输入到基于变换器的双向编码器表示网络（bidirectional encoder representations from transformers，BERT）中，采用孪生网络（Siamese Network）的结构解决BERT的各向异性的问题，并对网络的输出进行池化和标准化，利用余弦相似度计算两个会议之间的相似程度。实验结果表明SBFD方法在不同数据集上都取得了较好的效果，斯皮尔曼相关系数有一定程度的提高。
- Abstract:
- Aiming at the problem that the current semantic text similarity calculation methods have difficulty in calculating semantic similarity for scientific and technological conference data accurately, a siamese-BERT semantic similarity calculation algorithm for scientific and technological conferences fused with domain features (SBFD) is proposed in this paper. At first, the domain feature information of conference is obtained through entity recognition and keyword extraction, and it is input into the bidirectional encoder representations from transformers (BERT) network as a feature, together with conference information. The structure of the Siamese network is then used to solve the anisotropy problem of BERT. The output of the network is pooled and normalized, and finally the cosine similarity is used to calculate the similarity between the two conferences. Experimental results show that the SBFD algorithm achieves good results on different data sets, with the Spearman’s rank correlation coefficient improved in a certain extent.
- 参考文献/References:
-
[1] 周园春, 王卫军, 乔子越, 等. 科技大数据知识图谱构建方法及应用研究综述[J]. 中国科学: 信息科学, 2020, 50(7): 957?987.
ZHOU Yuanchun, WANG Weijun, QIAO Ziyue, et al. A survey on the construction methods and applications of sci-tech big data knowledge graph[J]. Scientia sinica (informationis), 2020, 50(7): 957?987.
[2] 苏晓娟, 张英杰, 白晨, 等. 科技大数据背景下的中英双语语料库的构建及其特点研究[J]. 中国科技资源导刊, 2019, 51(6): 87?92.
SU Xiaojuan, ZHANG Yingjie, BAI Chen, et al. Research of bilingual corpus construction and its characteristics in big data[J]. China science & technology resources review, 2019, 51(6): 87?92.
[3] 胡吉颖, 谢靖, 钱力, 等. 基于知识图谱的科技大数据知识发现平台建设[J]. 数据分析与知识发现, 2019, 3(1): 55?62.
HU Jiying, XIE Jing, QIAN Li, et al. Constructing big data platform for sci-tech knowledge discovery with knowledge graph[J]. Data analysis and knowledge discovery, 2019, 3(1): 55?62.
[4] TONG Yuqiang, GU Lize. A news text clustering method based on similarity of text labels[M]//Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Cham: Springer International Publishing, 2019: 496-503.
[5] LI Wenling, JIA Yingmin, DU Junping, et al. Distributed multiple-model estimation for simultaneous localization and tracking with NLOS mitigation[J]. IEEE transactions on vehicular technology, 2013, 62(6): 2824–2830.
[6] DAS A, MANDAL J, DANIAL Z, et al. A novel approach for automatic Bengali question answering system using semantic similarity analysis[J]. International journal of speech technology, 2020, 23(4): 873–884.
[7] FANG Yuke, DENG Weihong, DU Junping, et al. Identity-aware CycleGAN for face photo-sketch synthesis and recognition[J]. Pattern recognition, 2020, 102: 107249.
[8] QIAN Ming, LIU J, LI Chaofeng, et al. A comparative study of English-Chinese translations of court texts by machine and human translators and the Word2Vec based similarity measure’s ability to gauge human evaluation biases[C]//Proceedings of Machine Translation Summit XVII Volume 2: Translator, Project and User Tracks. Dublin: ACL, 2019: 95?100.
[9] XUE Zhe, DU Junping, DU Dawei, et al. Deep low-rank subspace ensemble for multi-view clustering[J]. Information sciences, 2019, 482: 210–227.
[10] RISTAD E S, YIANILOS P N. Learning string-edit distance[J]. IEEE transactions on pattern analysis and machine intelligence, 1998, 20(5): 522–532.
[11] HU Weiming, GAO Jun, LI Bing, et al. Anomaly detection using local kernel density estimation and context-based regression[J]. IEEE transactions on knowledge and data engineering, 2020, 32(2): 218–233.
[12] SUPHAKIT Niwattanakul, JATSADA Singthongchai, EKKACHAI Naenudorn, et al. Using of Jaccard coefficient for keywords similarity[C]//Proceedings of the international multiconference of engineers and computer scientists. Hong Kong: Newswood Limited, 2013, 1(6): 380?384.
[13] KOU Feifei, DU Junping, HE Yijiang, et al. Social network search based on semantic analysis and learning[J]. CAAI transactions on intelligence technology, 2016, 1(4): 293–302.
[14] LI Wenling, JIA Yingmin, DU Junping. Variance-constrained state estimation for nonlinearly coupled complex networks[J]. IEEE transactions on cybernetics, 2018, 48(2): 818–824.
[15] MIKOLOV T, CHEN KAI, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. New York: arXiv, 2013. (2013?01?16) [2022?03?24]. https: //arxiv. org/abs/1301.3781.
[16] PENNINGTON J, SOCHER R, MANNING C. Glove: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2014: 1532?1543.
[17] PETERS M E, NEUMANN M, IYYER M, et al. Deep contextualized word representations[EB/OL]. New York: arXiv, 2018. (2018?03?22)[2020?07?01]. https://arxiv.org/abs/1802.05365.
[18] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[C]//NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000?6010.
[19] RADFORD A, NARASIMHAN K. Improving language understanding by generative pre-training[EB/OL]. (2018?11?05)[2020?07?01].https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035.
[20] DEVLIN J, CHANG MING-WEI, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. New York: arXiv, 2018. (2018?10?11)[2022?03?20]. https: //arxiv. org/abs/1810.04805.
[21] HUANG Posen, HE Xiaodong, GAO Jianfeng, et al. Learning deep structured semantic models for web search using clickthrough data[C]//CIKM’13: Proceedings of the 22nd ACM international conference on Information & Knowledge Management. New York: ACM, 2013: 2333?2338.
[22] PALANGI H, DENG L, SHEN Y, et al. Semantic modelling with long-short-term memory for information retrieval[EB/OL]. New York: arXiv, 2014. (2014?12?20) [2022?03?20].https://arxiv.org/abs/1412.6629.
[23] PONTES E L, HUET S, LINHARES A C, et al. Predicting the semantic textual similarity with Siamese CNN and LSTM[EB/OL]. New York: arXiv, 2018. (2018?10?24) [2022?03?20]. https: //arxiv. org/abs/1810.10641.
[24] REIMERS N, GUREVYCH I. Sentence-BERT: sentence embeddings using Siamese BERT-networks[EB/OL]. New York: arXiv, 2019. (2019?08?27) [2022?03?20]. https: //arxiv. org/abs/1908.10084.
[25] LI BOHAN, ZHOU HAO, HE JUNXIAN, et al. On the sentence embeddings from pre-trained language models[EB/OL]. New York: arXiv, 2020. (2020?11?02) [2022?03?20]. https: //arxiv. org/abs/2011.05864.
- 相似文献/References:
-
[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[4]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[5]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[6]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[7]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
[8]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[9]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(6):894.[doi:10.11992/tis.201706037]
YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12():894.[doi:10.11992/tis.201706037]
[10]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
备注/Memo
收稿日期:2022-03-24。
基金项目:国家重点研发计划项目（2018YFB1402600）；国家自然科学基金项目（61772083,61802028）；广西科技重大专项（桂科AA18118054）
作者简介:于润羽，硕士研究生，主要研究方向为深度学习、数据挖掘;李雅文，副教授，主要研究方向为企业创新、人工智能、大数据;李昂，博士研究生，主要研究方向为信息检索、数据挖掘、机器学习
通讯作者:李雅文. E-mail：warmly0716@126.com
更新日期/Last Update:
1900-01-01