[1]孙俊,谢振平,王洪波.耦合演化采样和深度解码的可解释网络流量异常检测模型[J].智能系统学报,2023,18(5):1070-1078.[doi:10.11992/tis.202211035]
SUN Jun,XIE Zhenping,WANG Hongbo.An explainable network traffic anomaly detection model with coupled evolutionary sampling and deep decoding[J].CAAI Transactions on Intelligent Systems,2023,18(5):1070-1078.[doi:10.11992/tis.202211035]
点击复制
SUN Jun,XIE Zhenping,WANG Hongbo.An explainable network traffic anomaly detection model with coupled evolutionary sampling and deep decoding[J].CAAI Transactions on Intelligent Systems,2023,18(5):1070-1078.[doi:10.11992/tis.202211035]
耦合演化采样和深度解码的可解释网络流量异常检测模型
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第5期
页码:
1070-1078
栏目:
学术论文—机器学习
出版日期:
2023-09-05
- Title:
- An explainable network traffic anomaly detection model with coupled evolutionary sampling and deep decoding
- Keywords:
- machine learning; unsupervised learning; traffic anomaly detection; deep neural network; explainability; evolutionary sampling; deep enconding; autoencoder
- 分类号:
- TP391
- 摘要:
- 针对现有网络流量异常检测模型缺乏可解释性的问题，本研究提出了耦合演化采样和深度解码的可解释网络流量异常检测模型。首先，引入演化采样学习抽取代表特征样本，依此实现了强可解释性的样本编码过程；其次，构建了可解释的演化采样样本编码过程和不可解释的深度神经网络解码过程的耦合学习模型；最后，使用样本编码结果和重构误差进行异常检测。在NSL-KDD和CICIDS2017数据集上与现有方法的实验比较结果表明，该方法可显著提升模型可解释性和模型规模效率，并能取得与现有最优方法同等水平的检测性能。此外，上述新的学习策略，也可为可解释机器学习方法研究提供一种极具特色的技术方案参考。
- Abstract:
- Regarding the lack of explainability in existing network traffic anomaly detection models, this study proposed an explainable network traffic anomaly detection model with coupled evolutionary sampling and deep decoding. First, evolutionary sampling learning is introduced to extract representative feature samples, whereby a strongly explainable sample encoding process is implemented. Second, a coupled learning model of the explainable evolutionary sample encoding process and the unexplainable deep neural network decoding process is constructed. Finally, anomaly detection is performed using the sample encoding results and reconstruction errors. The experimental analysis on NSL-KDD and CICIDS2017 datasets are executed for our model and some existing methods, and corresponding results show that our model can significantly improve model explainability and scale efficiency and achieve the same level of detection performance as existing optimal methods. In addition, our proposed joint learning strategy may provide a highly distinctive scheme reference for the development of explainable machine learning methods.
- 参考文献/References:
-
[1] LIU Hongyu, LANG Bo. Machine learning and deep learning methods for intrusion detection systems: a survey[J]. Applied sciences, 2019, 9(20): 4396.
[2] BHATTACHARYYA D K, KALITA J K. Network anomaly detection: a machine learning perspective[M]. Boca Raton: Crc Press, 2013.
[3] 杨月麟, 毕宗泽. 基于深度学习的网络流量异常检测[J]. 计算机科学, 2021, 48(S2): 540-546
YANG Yuelin, BI Zongze. Network anomaly detection based on deep learning[J]. Computer science, 2021, 48(S2): 540-546
[4] ZONG Bo, SONG Qi, MIN M R, et al. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection[C]//International Conference on Learning Representations. Vancouver: OpenReview, 2018: 1?19.
[5] 席亮, 王瑞东, 樊好义, 等. 基于样本关联感知的无监督深度异常检测模型[J]. 计算机学报, 2021, 44(11): 2317-2331
XI Liang, WANG Ruidong, FAN Haoyi, et al. Sample-correlation-aware unsupervised deep anomaly detection model[J]. Chinese journal of computers, 2021, 44(11): 2317-2331
[6] TAN Zhiyuan, JAMDAGNI A, HE Xiangjian, et al. A system for denial-of-service attack detection based on multivariate correlation analysis[J]. IEEE transactions on parallel and distributed systems, 2014, 25(2): 447-456.
[7] AHMED M, NASER M A, HU Jiankun. A survey of network anomaly detection techniques[J]. Journal of network and computer applications, 2016, 60: 19-31.
[8] TRAN C P, TRAN D K. Anomaly detection in POSTFIX mail log using principal component analysis[C]//2018 10th International Conference on Knowledge and Systems Engineering. Ho Chi Minh City: IEEE, 2018: 107?112.
[9] 李贝贝, 彭力, 戴菲菲. 结合马氏距离与自编码器的网络流量异常检测方法[J]. 计算机工程, 2022, 48(4): 133-142
LI Beibei, PENG Li, DAI Feifei. Abnormal network traffic detection method combining mahalanobis distance and autoencoder[J]. Computer engineering, 2022, 48(4): 133-142
[10] SCHLEGL T, SEEB?CK P, WALDSTEIN S M, et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery[C]//International Conference on Information Processing in Medical Imaging. Cham: Springer, 2017: 146?157.
[11] ZENATI H, ROMAIN M, FOO C S, et al. Adversarially learned anomaly detection[C]//2018 IEEE International Conference on Data Mining. Singapore. IEEE, 2018: 727?736.
[12] ZHANG Kunzhong, KANG Xudong, LI Shutao. Isolation forest for anomaly detection in hyperspectral images[C]// 2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama: IEEE, 2019: 437?440.
[13] SINGH K, MATHAI K J. Performance comparison of intrusion detection system between deep belief network (DBN)algorithm and state preserving extreme learning machine (SPELM) algorithm[C]//2019 IEEE International Conference on Electrical, Computer and Communication Technologies. Coimbatore: IEEE, 2019: 1?7.
[14] 王倩倩, 苗夺谦, 张远健. 深度自编码与自更新稀疏组合的异常事件检测算法[J]. 智能系统学报, 2020, 15(6): 1197-1203
WANG Qianqian, MIAO Duoqian, ZHANG Yuanjian. Abnormal event detection method based on deep auto-encoder and self-updating sparse combination[J]. CAAI transactions on intelligent systems, 2020, 15(6): 1197-1203
[15] ZHAI Shuangfei, CHENG Yu, LU Weining, et al. Deep structured energy based models for anomaly detection[C]//International conference on machine learning. New York: PMLR, 2016: 1100?1109.
[16] GONG Dong, LIU Lingqiao, LE V, et al. Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection[C]//IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2020: 1705?1714.
[17] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2. New York: ACM, 2014: 2672?2680.
[18] 黄训华, 张凤斌, 樊好义, 等. 基于多模态对抗学习的无监督时间序列异常检测[J]. 计算机研究与发展, 2021, 58(8): 1655-1667
HUANG Xunhua, ZHANG Fengbin, FAN Haoyi, et al. Multimodal adversarial learning based unsupervised time series anomaly detection[J]. Journal of computer research and development, 2021, 58(8): 1655-1667
[19] AUDIBERT J, MICHIARDI P, GUYARD F, et al. USAD: UnSupervised anomaly detection on multivariate time series[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2020: 3395?3404.
[20] TING Kaiming, XU Bicun, WASHIO T, et al. Isolation distributional kernel: a new tool for kernel based anomaly detection[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2020: 198?206.
[21] CHEN Yuanhong, TIAN Yu, PANG Guansong, et al. Deep one-class classification via interpolated Gaussian descriptor[J]. Proceedings of the AAAI conference on artificial intelligence, 2022, 36(1): 383-392.
[22] XIE Zhenping, SUN Jun, PALADE V, et al. Evolutionary sampling: a novel way of machine learning within a probabilistic framework[J]. Information sciences, 2015, 299: 262-282.
[23] TAVALLAEE M, BAGHERI E, LU Wei, et al. A detailed analysis of the KDD CUP 99 data set[C]//2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications. Ottawa: IEEE, 2009: 1?6.
[24] SHARAFALDIN I, HABIBI L A, GHORBANI A A. Toward generating a new intrusion detection dataset and intrusion traffic characterization[C]//Proceedings of the 4th International Conference on Information Systems Security and Privacy. Portugal: SCITEPRESS-Science and Technology Publications, 2018: 108?116.
[25] LI Kunlun, HUANG Houkuan, TIAN Shengfeng, et al. Improving one-class SVM for anomaly detection[C]//Proceedings of the 2003 International Conference on Machine Learning and Cybernetics. Xi’an: IEEE, 2004: 3077?3081.
[26] AN J, CHO S. Variational autoencoder based anomaly detection using reconstruction probability[J]. Special lecture on IE, 2015, 2(1): 1-18.
- 相似文献/References:
-
[1]叶志飞,文益民,吕宝粮.不平衡分类问题研究综述[J].智能系统学报,2009,4(2):148.
YE Zhi-fei,WEN Yi-min,LU Bao-liang.A survey of imbalanced pattern classification problems[J].CAAI Transactions on Intelligent Systems,2009,4():148.
[2]刘奕群,张 敏,马少平.基于非内容信息的网络关键资源有效定位[J].智能系统学报,2007,2(1):45.
LIU Yi-qun,ZHANG Min,MA Shao-ping.Web key resource page selection based on non-content inf o rmation[J].CAAI Transactions on Intelligent Systems,2007,2():45.
[3]马世龙,眭跃飞,许 可.优先归纳逻辑程序的极限行为[J].智能系统学报,2007,2(4):9.
MA Shi-long,SUI Yue-fei,XU Ke.Limit behavior of prioritized inductive logic programs[J].CAAI Transactions on Intelligent Systems,2007,2():9.
[4]姚伏天,钱沄涛.高斯过程及其在高光谱图像分类中的应用[J].智能系统学报,2011,6(5):396.
YAO Futian,QIAN Yuntao.Gaussian process and its applications in hyperspectral image classification[J].CAAI Transactions on Intelligent Systems,2011,6():396.
[5]文益民,强保华,范志刚.概念漂移数据流分类研究综述[J].智能系统学报,2013,8(2):95.[doi:10.3969/j.issn.1673-4785.201208012]
WEN Yimin,QIANG Baohua,FAN Zhigang.A survey of the classification of data streams with concept drift[J].CAAI Transactions on Intelligent Systems,2013,8():95.[doi:10.3969/j.issn.1673-4785.201208012]
[6]杨成东,邓廷权.综合属性选择和删除的属性约简方法[J].智能系统学报,2013,8(2):183.[doi:10.3969/j.issn.1673-4785.201209056]
YANG Chengdong,DENG Tingquan.An approach to attribute reduction combining attribute selection and deletion[J].CAAI Transactions on Intelligent Systems,2013,8():183.[doi:10.3969/j.issn.1673-4785.201209056]
[7]胡小生,钟勇.基于加权聚类质心的SVM不平衡分类方法[J].智能系统学报,2013,8(3):261.
HU Xiaosheng,ZHONG Yong.Support vector machine imbalanced data classification based on weighted clustering centroid[J].CAAI Transactions on Intelligent Systems,2013,8():261.
[8]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[9]孔庆超,毛文吉,张育浩.社交网站中用户评论行为预测[J].智能系统学报,2015,10(3):349.[doi:10.3969/j.issn.1673-4785.201403019]
KONG Qingchao,MAO Wenji,ZHANG Yuhao.User comment behavior prediction in social networking sites[J].CAAI Transactions on Intelligent Systems,2015,10():349.[doi:10.3969/j.issn.1673-4785.201403019]
[10]申彦,朱玉全.CMP上基于数据集划分的K-means多核优化算法[J].智能系统学报,2015,10(4):607.[doi:10.3969/j.issn.1673-4785.201411036]
SHEN Yan,ZHU Yuquan.An optimized algorithm of K-means based on data set partition on CMP systems[J].CAAI Transactions on Intelligent Systems,2015,10():607.[doi:10.3969/j.issn.1673-4785.201411036]
[11]冯冰,李绍滋.中医脉诊信号的无监督聚类分析研究[J].智能系统学报,2018,13(4):564.[doi:10.11992/tis.201703030]
FENG Bing,LI Shaozi.Unsupervised clustering analysis of human-pulse signal in traditional Chinese medicine[J].CAAI Transactions on Intelligent Systems,2018,13():564.[doi:10.11992/tis.201703030]
[12]杨文元.多标记学习自编码网络无监督维数约简[J].智能系统学报,2018,13(5):808.[doi:10.11992/tis.201804051]
YANG Wenyuan.Unsupervised dimensionality reduction of multi-label learning via autoencoder networks[J].CAAI Transactions on Intelligent Systems,2018,13():808.[doi:10.11992/tis.201804051]
备注/Memo
收稿日期:2022-11-21。
基金项目:国家自然科学基金项目（62272201，61872166）.
作者简介:孙俊，硕士研究生，主要研究方向为网络流量异常检测、机器学习;谢振平，教授，博士生导师，主要研究方向为知识计算与认知学习。获教育部科技进步一等奖、全国商业科技进步特等奖/一等奖等科研奖励。主持或主要参与完成国家、省部级科研项目9项，获授权发明专利6项。发表学术论文50余篇;王洪波，高级工程师，主要研究方向为网络安全软件及系统
通讯作者:谢振平.E-mail：xiezp@jiangnan.edu.cn
更新日期/Last Update:
1900-01-01