[1]冀若含,董红斌.基于重复度分析的森林优化特征选择算法[J].智能系统学报,2022,17(6):1113-1122.[doi:10.11992/tis.202111060]
JI Ruohan,DONG Hongbin.Feature selection using forest optimization algorithm based on duplication analysis[J].CAAI Transactions on Intelligent Systems,2022,17(6):1113-1122.[doi:10.11992/tis.202111060]
点击复制
JI Ruohan,DONG Hongbin.Feature selection using forest optimization algorithm based on duplication analysis[J].CAAI Transactions on Intelligent Systems,2022,17(6):1113-1122.[doi:10.11992/tis.202111060]
基于重复度分析的森林优化特征选择算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第6期
页码:
1113-1122
栏目:
学术论文—机器学习
出版日期:
2022-11-05
- Title:
- Feature selection using forest optimization algorithm based on duplication analysis
- Keywords:
- feature selection; evolutionary algorithm; duplication analysis; information entropy; information gain; restart mechanism; forest optimization algorithm; dimensionality reduction
- 分类号:
- TP301
- 文献标志码:
- 2022-08-24
- 摘要:
- 森林优化算法是一种基于森林中树木播种思想的演化算法，其具有良好的特征空间搜索能力，且实现难度低。但该算法在森林整体的收敛速度和寻优能力上仍存在提升空间，而且对高维数据集的适应度较差。本文针对上述问题提出了基于重复度分析的森林优化特征选择算法(feature selection using forest optimization algorithm based on duplication analysis, DAFSFOA)。该算法提出了基于信息增益的自适应初始化策略、森林重复度分析机制、森林重启机制、候选最优树生成策略、综合考虑特征选择数量和分类正确率的适应度函数。实验结果表明，DAFSFOA在大部分数据集上达到了最高的分类准确率。同时，对于高维数据集SRBCT，在维度缩减率和分类准确率方面，DAFSFOA对比森林优化特征选择算法(feature selection using forest optimization algorithm, FSFOA)都有较大提升。DAFSFOA 比FSFOA具有更强的特征空间探索能力，而且能够适应不同维度的数据集。
- Abstract:
- The forest optimization algorithm is an evolutionary algorithm based on the concept of forest tree planting. It has a strong capability for searching for feature space and low implementation difficulty. However, the algorithm still has room for improvement in the convergence speed and merit-seeking ability of the forest as a whole, and it is not well-suited to high dimensional data sets. In this paper, we propose to use a forest optimization algorithm based on duplication analysis (DAFSFOA) to address the above problems. The algorithm proposes an adaptive initialization strategy based on information gain, a forest repetition analysis mechanism, a forest restart mechanism, a candidate optimal tree generation strategy, and an adaptation function that integrates the number of feature selections and the correct classification rate. The experimental results show that DAFSFOA achieves the highest classification accuracy on most datasets. Meanwhile, for the high dimensional dataset SRBCT, DAFSFOA has a large improvement over feature selection using a forest optimization algorithm (FSFOA) in terms of dimensionality reduction rate and classification accuracy. DAFSFOA has a stronger feature space exploration capability than FSFOA and can adapt to datasets with different dimensions.
- 参考文献/References:
-
[1] LI Jundong, CHENG Kewei, WANG Suhang, et al. Feature selection: a data perspective[J]. ACM computing surveys, 2018, 50(6): 94.
[2] XUE Bing, ZHANG Mengjie, BROWNE W N, et al. A survey on evolutionary computation approaches to feature selection[J]. IEEE transactions on evolutionary computation, 2016, 20(4): 606–626.
[3] DASH M, LIU H. Feature selection for classification[J]. Intelligent data analysis, 1997, 1(1/2/3/4): 131–156.
[4] GHAEMI M, FEIZI-DERAKHSHI M R. Forest optimization algorithm[J]. Expert systems with applications, 2014, 41(15): 6676–6687.
[5] GHAEMI M, FEIZI-DERAKHSHI M R. Feature selection using Forest Optimization Algorithm[J]. Pattern recognition, 2016, 60: 121–129.
[6] 初蓓, 李占山, 张梦林, 等. 基于森林优化特征选择算法的改进研究[J]. 软件学报, 2018, 29(9): 2547–2558
CHU Bei, LI Zhanshan, ZHANG Menglin, et al. Research on improvements of feature selection using forest optimization algorithm[J]. Journal of software, 2018, 29(9): 2547–2558
[7] UNLER A, MURAT A, CHINNAM R B. mr2PSO: a maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification[J]. Information sciences, 2011, 181(20): 4625–4641.
[8] CERVANTE L, BING Xue, ZHANG Mengjie, et al. Binary particle swarm optimisation for feature selection: a filter based approach[C]//2012 IEEE Congress on Evolutionary Computation. Brisbane, 2012: IEEE, 2012: 1-8.
[9] KANNAN S S, RAMARAJ N. A novel hybrid feature selection via Symmetrical Uncertainty ranking based local memetic search algorithm[J]. Knowledge-based systems, 2010, 23(6): 580–585.
[10] SONG Xianfang, ZHANG Yong, GONG Dunwei, et al. Feature selection using bare-bones particle swarm optimization with mutual information[J]. Pattern recognition, 2021, 112: 107804.
[11] XU Hang, XUE Bing, ZHANG Mengjie. A duplication analysis-based evolutionary algorithm for biobjective feature selection[J]. IEEE transactions on evolutionary computation, 2021, 25(2): 205–218.
[12] JOHN LU Z Q. The elements of statistical learning: data mining, inference, and prediction[J]. Journal of the royal statistical society:series A (statistics in society), 2010, 173(3): 693–694.
[13] SUN Zehang, BEBIS G, MILLER R. Object detection using feature subset selection[J]. Pattern recognition, 2004, 37(11): 2165–2176.
[14] DAVIES S, RUSSELL S. NP-completeness of searches for smallest possible feature sets[C]//AAAI Symposium on Intelligent Relevance. Menlo Park: AAAI Press, 1994: 37-39.
[15] MARCANO-CEDEO A, QUINTANILLA-DOMíNGUEZ J, CORTINA-JANUCHS M G, et al. Feature sele-ction using Sequential Forward Selection and classify-cation applying Artificial Metaplasticity Neural Netwo-rk[C]// IECON 2010—36th Annual Conference on IEEE Industrial Electronics Society. Glendale: IEEE, 2010: 2845-2850.
[16] PENG Hongyi, JIANG Chunfu, FANG Xiang, et al. Variable selection for Fisher linear discriminant analysis using the modified sequential backward selection algorithm for the microarray data[J]. Applied mathematics and computation, 2014, 238: 132–140.
[17] I??K ?. Dominant point detection based on suboptimal feature selection methods[J]. Expert systems with applications, 2020, 161: 113741.
[18] SETIAWAN D, KUSUMA W A, WIGENA A H. Sequential forward floating selection with two selection criteria[C]//2017 International Conference on Advanced Computer Science and Information Systems (ICACSIS). Bali: IEEE, 2017: 395-400.
[19] SINGH P, CHAUDHURY S, PANIGRAHI B K. Hybrid MPSO-CNN: multi-level particle swarm optimized hyperparameters of convolutional neural network[J]. Swarm and evolutionary computation, 2021, 63: 100863.
[20] ZHOU Yu, ZHANG Wenjun, KANG Junhao, et al. A problem-specific non-dominated sorting genetic algorithm for supervised feature selection[J]. Information sciences, 2021, 547: 841–859.
[21] LI Yongbo, SOLEIMANI H, ZOHAL M. An improved ant colony optimization algorithm for the multi-depot green vehicle routing problem with multiple objectives[J]. Journal of cleaner production, 2019, 227: 1161–1172.
[22] MIRJALILI S, LEWIS A. The whale optimization algorithm[J]. Advances in engineering software, 2016, 95: 51–67.
[23] DONG Hongbin, PAN Yuyao, SUN Jing. High dimensional feature selection method of dual gbest based on PSO[C]//2020 IEEE Congress on Evolutionary Computation. Glasgow: IEEE, 2020: 1-8.
[24] AGRAWAL R K, KAUR B, SHARMA S. Quantum based Whale Optimization Algorithm for wrapper feature selection[J]. Applied soft computing, 2020, 89: 106092.
[25] LI Anda, XUE Bing, ZHANG Mengjie. Improved binary particle swarm optimization for feature selection with new initialization and search space reduction strategies[J]. Applied soft computing, 2021, 106: 107302.
[26] XIE Qi, CHENG Gengguo, ZHANG Xiao, et al. Feature selection using improved forest optimization algorithm[J]. Information technology and control, 2020, 49(2): 289–301.
[27] MOUSTAKIDIS S P, THEOCHARIS J B. SVM-FuzCoC: a novel SVM-based feature selection method using a fuzzy complementary criterion[J]. Pattern recognition, 2010, 43(11): 3712–3729.
[28] HU Qinghua, CHE Xunjian, ZHANG Lei, et al. Feature evaluation and selection based on neighborhood soft margin[J]. Neurocomputing, 2010, 73(10/11/12): 2114–2124.
[29] HUANG Jinjie, CAI Yunze, XU Xiaoming. A hybrid genetic algorithm for feature selection wrapper based on mutual information[J]. Pattern recognition letters, 2007, 28(13): 1825–1844.
[30] ZHU Wenzhi, SI Gangquan, ZHANG Yanbin, et al. Neighborhood effective information ratio for hybrid feature subset evaluation and selection[J]. Neurocomputing, 2013, 99: 25–37.
[31] TABAKHI S, MORADI P, AKHLAGHIAN F. An unsupervised feature selection algorithm based on ant colony optimization[J]. Engineering applications of artificial intelligence, 2014, 32: 112–123.
[32] XUE Bing, ZHANG Mengjie, BROWNE W N. Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms[J]. Applied soft computing, 2014, 18: 261–276.
- 相似文献/References:
-
[1]孙正兴,张尧烨,李? 彬.基于线性规划分类器的相关反馈技术[J].智能系统学报,2007,2(3):34.
SUN Zheng-xing,ZHANG Yao-ye,LI Bin.Applying relevance feedback with a linear programming classifier[J].CAAI Transactions on Intelligent Systems,2007,2():34.
[2]张志飞,苗夺谦.基于粗糙集的文本分类特征选择算法[J].智能系统学报,2009,4(5):453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
ZHANG Zhi-fei,MIAO Duo-qian.Feature selection for text categorization based on rough set[J].CAAI Transactions on Intelligent Systems,2009,4():453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
[3]顾成杰,张顺颐,杜安源.结合粗糙集和禁忌搜索的网络流量特征选择[J].智能系统学报,2011,6(3):254.
GU Chengjie,ZHANG Shunyi,DU Anyuan.Feature selection of network traffic using a rough set and tabu search[J].CAAI Transactions on Intelligent Systems,2011,6():254.
[4]孙倩茹,王文敏,刘宏.视频序列的人体运动描述方法综述[J].智能系统学报,2013,8(3):189.
SUN Qianru,WANG Wenmin,LIU Hong.Study of human action representation in video sequences[J].CAAI Transactions on Intelligent Systems,2013,8():189.
[5]曹晋,张莉,李凡长.一种基于支持向量数据描述的特征选择算法[J].智能系统学报,2015,10(2):215.[doi:10.3969/j.issn.1673-4785.201405063]
CAO Jin,ZHANG Li,LI Fanzhang.A noval support vector data description-based feature selection method[J].CAAI Transactions on Intelligent Systems,2015,10():215.[doi:10.3969/j.issn.1673-4785.201405063]
[6]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[7]张佳骕,蒋亦樟,王士同.基于特征选择聚类方法的稀疏TSK模糊系统[J].智能系统学报,2015,10(4):583.[doi:10.3969/j.issn.1673-4785.201412001]
ZHANG Jiasu,JIANG Yizhang,WANG Shitong.Sparse TSK fuzzy system based on feature selection clustering method[J].CAAI Transactions on Intelligent Systems,2015,10():583.[doi:10.3969/j.issn.1673-4785.201412001]
[8]陈玉明,吴克寿,李向军.基因表达数据在邻域关系中的特征选择[J].智能系统学报,2014,9(2):210.[doi:10.3969/j.issn.1673-4785.201307014]
CHEN Yuming,WU Keshou,LI Xiangjun.Gene expression data feature selection with neighborhood relation[J].CAAI Transactions on Intelligent Systems,2014,9():210.[doi:10.3969/j.issn.1673-4785.201307014]
[9]郭雨萌,李国正.一种多标记数据的过滤式特征选择框架[J].智能系统学报,2014,9(3):292.[doi:10.3969/j.issn.1673-4785.201403064]
GUO Yumeng,LI Guozheng.A filtering framework for the multi-label feature selection[J].CAAI Transactions on Intelligent Systems,2014,9():292.[doi:10.3969/j.issn.1673-4785.201403064]
[10]滕旭阳,董红斌,孙静.面向特征选择问题的协同演化方法[J].智能系统学报,2017,12(1):24.[doi:10.11992/tis.201611029]
TENG Xuyang,DONG Hongbin,SUN Jing.Co-evolutionary algorithm for feature selection[J].CAAI Transactions on Intelligent Systems,2017,12():24.[doi:10.11992/tis.201611029]
备注/Memo
收稿日期:2021-11-30。
基金项目:黑龙江自然科学基金资助项目(LH2020F023).
作者简介:冀若含，硕士研究生，主要研究方向为演化算法、特征选择;董红斌，教授，博士生导师，中国计算机学会高级研究员，主要研究方向为多智能体系统、机器学习。主持和完成国家自然科学基金项目、工信部基础研究项目、黑龙江省自然科学基金项目等。荣获黑龙江省高校科学技术奖和黑龙江省优秀高等教育科学成果奖。发表学术论文90余篇，主编教材2部
通讯作者:董红斌.E-mail:donghongbin@hrbeu.edu.cn
更新日期/Last Update:
1900-01-01