[1]QIN Ya,SHEN Guowei,YU Hongxing.Large-scale network security entity recognition method based on Hadoop[J].CAAI Transactions on Intelligent Systems,2019,14(5):1017-1025.[doi:10.11992/tis.201809024]
Copy

Large-scale network security entity recognition method based on Hadoop

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 14 Number of periods: 2019 5 Page number: 1017-1025 Column: 学术论文—智能系统 Public date: 2019-09-05
Title:
Large-scale network security entity recognition method based on Hadoop
Author(s):
QIN Ya1; 2;  SHEN Guowei1; 2;  YU Hongxing1; 2
1. Department of Computer Science and Technology, Guizhou University, Guiyang 550025, China;
2. Guizhou Provincial Key Laboratory of Public Big Data, Guizhou University, Guiyang 550025, China
Keywords:
big data; heterogeneous data; network security; knowledge graph; security entity; entity recognition; network data; Hadoop; CRF algorithm
CLC:
TP391.0
DOI:
10.11992/tis.201809024
Abstract:
In this era of big data, a fundamental problem for constructing network security knowledge graphs is how to efficiently and accurately identify the network security entities present in multi-source heterogeneous data. This study focuses on text data related to network safety and investigate the use of a security entity recognition algorithm that supports massive-network text data, thereby laying a foundation for building the network security knowledge graph. To efficiently and accurately extract the security entities in massive-network text data, we propose an improved conditional random fields (CRF) algorithm based on the Hadoop distributed computing framework to segment data sets effectively, which realize efficient and accurate recognition of security entities. The experimental results reveal that the proposed security entity recognition algorithm achieved a high precision rate on a large-scale real network data set and improved the efficiency of network security entity recognition..
References:
[1] 廖建新. 大数据技术的应用现状与展望[J]. 电信科学, 2015, 31(7):1-12 LIAO Jianxin. Big data technology:current applications and prospects[J]. Telecommunications science, 2015, 31(7):1-12
[2] 单琳. 网络威胁情报发展现状综述[J]. 保密科学技术, 2016(8):28-33 SHAN Lin. Overview of the development status of cyber threat intelligence[J]. Security science and technology, 2016(8):28-33
[3] 南湘浩, 陈钟. 网络安全技术概论[J]. 计算机安全, 2003(30):76 NAN Xianghao, CHEN Zhong. Introduction to network security technology[J]. Computer security, 2003(30):76
[4] 陈兴蜀, 曾雪梅, 王文贤, 等. 基于大数据的网络安全与情报分析[J]. 工程科学与技术, 2017, 49(3):1-12 CHEN Xingshu, ZENG Xuemei, WANG Wenxian, et al. Big data analytics for network security and intelligence[J]. Advanced engineering sciences, 2017, 49(3):1-12
[5] 张晓艳, 王挺, 陈火旺. 命名实体识别研究[J]. 计算机科学, 2005, 32(4):44-48 ZHANG Xiaoyan, WANG Ting, CHRN Huowang. Research on named entity recognition[J]. Computer science, 2005, 32(4):44-48
[6] JONES C L, BRIDGES R A, HUFFE K M T, et al. Towards a relation extraction framework for cyber-security concepts[C]//Proceedings of the 10th Annual Cyber and Information Security Research Conference. Oak Ridge, USA, 2015:11.
[7] JOSHI A, LAL R, FININ T, et al. Extracting cybersecurity related linked data from text[C]//Proceedings of 2013 IEEE Seventh International Conference on Semantic Computing. Irvine, USA, 2013:252-259.
[8] LAL R. Information extraction of security related entities and concepts from unstructured text[D]. Baltimore County:University of Maryland, 2013.
[9] MULWAD V, LI Wenjia, JOSHI A, et al. Extracting information about security vulnerabilities from web text[C]//Proceedings of 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology. Lyon, France, 2011:257-260.
[10] 翟菊叶, 陈春燕, 张钰, 等. 基于CRF与规则相结合的中文电子病历命名实体识别研究[J]. 包头医学院学报, 2017, 33(11):124-125, 130 ZHAI Juye, CHEN Chunyan, ZHANG Yu, et al. A study on the named entity recognition of Chinese electronic medical record based on combination of CRF and rules[J]. Journal of Baotou Medical College, 2017, 33(11):124-125, 130
[11] 张晓艳, 王挺, 陈火旺. 基于混合统计模型的汉语命名实体识别方法[J]. 计算机工程与科学, 2006, 28(6):135-139 ZHANG Xiaoyan, WANG Ting, CHEN Huowang. A mixed statistical model-based method for chinese named entity recognition[J]. Computer engineering and science, 2006, 28(6):135-139
[12] 徐梓豪. 基于统计模型的中文命名实体识别方法研究及应用[D]. 北京:北京化工大学, 2017. XU Zihao. Statistical model based Chinese named entity recognition methods and its application to medical records[D]. Beijing:Beijing University of Chemical Technology, 2017.
[13] RABINER L R. A tutorial on hidden Markov models and selected applications in speech recognition[M]//WAIBEL A, LEE K F. Readings in Speech Recognition. San Francisco:Morgan Kaufmann, 1990:267-296.
[14] KOELING R. Chunking with maximum entropy models[C]//Proceedings of the 2nd Workshop on Learning Language in Logic and the 4th Conference on Computational Natural Language Learning. Lisbon, Portugal, 2000:139-141.
[15] 郑秋生, 刘守喜. 基于CRF的互联网文本命名实体识别研究[J]. 中原工学院学报, 2016, 27(1):70-73, 95 ZHRNG Qiusheng, LIU Shouxi. Research of web text named entity recognition based on CRF[J]. Journal of Zhongyuan University of Technology, 2016, 27(1):70-73, 95
[16] 朱颢东, 杨立志, 丁温雪, 等. 基于主题标签和CRF的中文微博命名实体识别[J]. 华中师范大学学报(自然科学版), 2018, 52(3):316-321 ZHU Haodong, YANG Lizhi, DING Wenxue, et al. Named entity recognition of Chinese microblog based on theme tag and CRF[J]. Journal of Central China Normal University (Natural Sciences), 2018, 52(3):316-321
[17] TELNOV Y, SAVICHEV I. Ontology-based competency management:infrastructures for the knowledge intensive learning organization[C]//Proceedings of the 1st International Early Research Career Enhancement School. Cham, Switzerland, 2016:249-256.
[18] IANNACONE M, BOHN S, NAKAMURA G, et al. Developing an ontology for cyber security knowledge graphs[C]//Proceedings of the 10th Annual Cyber and Information Security Research Conference. Oak Ridge, USA, 2015:12.
[19] DEAN J, GHEMAWAT S. MapReduce:simplified data processing on large clusters[C]//Proceedings of the 6th Conference on Symposium on Opearting Systems Design and Implementation. San Francisco, USA, 2004:10.
[20] LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. Williamstown, USA, 2001:282-289.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems