[1]李顺勇,王改变.一种新的最大相关最小冗余特征选择算法[J].智能系统学报,2021,16(4):649-661.[doi:10.11992/tis.202009016]
LI Shunyong,WANG Gaibian.New MRMR feature selection algorithm[J].CAAI Transactions on Intelligent Systems,2021,16(4):649-661.[doi:10.11992/tis.202009016]
点击复制
LI Shunyong,WANG Gaibian.New MRMR feature selection algorithm[J].CAAI Transactions on Intelligent Systems,2021,16(4):649-661.[doi:10.11992/tis.202009016]
一种新的最大相关最小冗余特征选择算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第4期
页码:
649-661
栏目:
学术论文—机器学习
出版日期:
2021-07-05
- Title:
- New MRMR feature selection algorithm
- Keywords:
- feature selection; redundancy; relevance; dimension reduction; classification; classification accuracy; support vector machines; T-test
- 分类号:
- TP181
- 摘要:
- 传统的基于特征选择的分类算法中，由于其采用的冗余度和相关度评价标准单一，从而使得此类算法应用范围受限。针对这个问题，本文提出一种新的最大相关最小冗余特征选择算法，该算法在度量特征之间冗余度的评价准则中引入了两种不同的评价准则；在度量特征与类别之间的相关度中引入了4种不同的评价准则，衍生出8种不同的特征选择算法，从而使得该算法应用范围增大。此外，由于传统的最大相关最小冗余特征选择算法不能根据用户实际需求的数据维度进行特征选择。所以，引入了指示向量 $\lambda $ 来刻画用户实际的数据维度需求，提出了一种新的目标函数来求解最优特征子集,利用支持向量机对4个UCI数据集的特征子集进行了实验，最后，利用分类正确率、成对单边T检验充分验证了该算法的有效性。
- Abstract:
- The application scopes of traditional classification algorithms based on feature selection are limited due to the single evaluation criteria of redundancy and relevance adopted. To solve this problem, this paper proposes a new maximum relevance, minimum redundancy (MRMR) feature selection algorithm, which enlarges its application scope by introducing two different evaluation criteria to measure the redundancy between features of measurement, measuring the correlation between features and categories, and deriving eight different feature selection algorithms. In addition, because the traditional MRMR feature selection algorithms cannot realize feature selection according to the data dimension of users’ actual demand, the study also applies an indicator vector $\lambda$ to achieve that, proposes a new objective function to obtain the optimal feature subset, and conducts experiments on four feature subsets of UCI using a support vector machine. Finally, the study verifies the effectiveness of the algorithm using classification accuracy and pairs of unilateral T-tests.
- 参考文献/References:
-
[1] 王娟, 慈林林, 姚康泽. 特征选择方法综述[J]. 计算机工程与科学, 2005, 27(12): 68-71
WANG Juan, CI Linlin, YAO Kangze. A survey of feature selection[J]. Computer engineering and science, 2005, 27(12): 68-71
[2] 周红标, 乔俊飞. 基于高维k-近邻互信息的特征选择方法[J]. 智能系统学报, 2017, 12(5): 595-600
ZHOU Hongbiao, QIAO Junfei. Feature selection method based on high dimensional k-nearest neighbors mutual information[J]. CAAI transactions on intelligent systems, 2017, 12(5): 595-600
[3] 徐洪峰, 孙振强. 多标签学习中基于互信息的快速特征选择方法[J]. 计算机应用, 2019, 39(10): 2815-2821
XU Hongfeng, SUN Zhenqiang. Fast feature selection method based on mutual information in multi-label learning[J]. Journal of computer applications, 2019, 39(10): 2815-2821
[4] 计智伟, 胡珉. 一种双重过滤式特征选择算法[J]. 计算机工程与应用, 2011, 47(19): 190-193, 206
JI Zhiwei, HU Min. A double-filtered feature selection algorithm[J]. Computer engineering and applications, 2011, 47(19): 190-193, 206
[5] 吴红霞, 吴悦, 刘宗田, 等. 基于Relief和SVM-RFE的组合式SNP特征选择[J]. 计算机应用研究, 2012, 29(6): 2074-2077
WU Hongxia, WU Yue, LIU Zongtian, et al. Combined SNP feature selection based on Relief and SVM-RFE[J]. Application research of computers, 2012, 29(6): 2074-2077
[6] 肖忆南, 谢榕, 杜娟. 基于t检验和弹性网的数据分类特征选择方法[J]. 小型微型计算机系统, 2015, 36(10): 2213-2217
XIAO Yinan, XIE Rong, DU Juan. Feature selection method for data classification based on t-test and elastic net[J]. Journal of Chinese computer systems, 2015, 36(10): 2213-2217
[7] SRIVIDHYA S, MALLIKA R. Feature selection for high dimensional imbalanced datasets using game theory and fisher score[J]. Journal of advanced research in dynamical and control systems, 2017, 9(8): 195-202.
[8] SAINZ L V, SAINZ J S, LáZARO M. Oscillatory brain activity in morphological parsing of complex words[J]. Information gain from stems and suffixes, 2018, 9(5): 271.
[9] MALDONADO S, WEBER R. A wrapper method for feature selection using support vector machines[J]. Information sciences, 2009, 179(13): 2208-2217.
[10] 林棋, 张宏, 李千目. 一种基于MA-LSSVM的封装式特征选择算法[J]. 南京理工大学学报, 2016, 40(1): 10-16
LIN Qi, ZHANG Hong, LI Qianmu. Wrapper feature selection algorithm based on MA-LSSVM[J]. Journal of Nanjing University of Science and Technology, 2016, 40(1): 10-16
[11] 胡峰, 杨梦. 基于特征聚类的封装特征选择算法[J]. 计算机工程与设计, 2018, 39(1): 230-237
HU Feng, YANG Meng. Algorithm for wrapper feature selection based on feature clustering[J]. Computer engineering and design, 2018, 39(1): 230-237
[12] 王晓初, 王士同, 包芳, 等. 最小化类内距离和分类算法[J]. 电子与信息学报, 2016, 38(3): 532-540
WANG Xiaochu, WANG Shitong, BAO Fang, et al. Minimize intra-class distance and classification algorithm[J]. Journal of electronics & information technology, 2016, 38(3): 532-540
[13] PENG Hanchuan, LONG Fuhui, DING C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE transactions on pattern analysis and machine intelligence, 2005, 27(8): 1226-1238.
[14] 郭凯文, 潘宏亮, 侯阿临. 基于特征选择和聚类的分类算法[J]. 吉林大学学报(理学版), 2018, 56(2): 395-398
GUO Kaiwen, PAN Hongliang, HOU Alin. Classification algorithm based on feature selection and clustering[J]. Journal of Jilin University (science edition), 2018, 56(2): 395-398
[15] 李纯果, 张春琴, 李海峰. 基于排序互信息的无监督特征选择[J]. 河北大学学报(自然科学版), 2020, 40(2): 200-204
LI Chunguo, ZHANG Chunqin, LI Haifeng. Unsupervised feature selection based on ranking mutual information[J]. Journal of Hebei University (natural science edition), 2020, 40(2): 200-204
[16] 刘云, 肖雪, 黄荣乘. 混合蒙特卡罗搜索的特征选择算法的优化[J]. 信息技术, 2020, 44(5): 28-31, 36
LIU Yun, XIAO Xue, HUANG Rongcheng. Optimization of feature selection based on hybrid Monte Carlo Tree[J]. Information technology, 2020, 44(5): 28-31, 36
[17] 周传华, 李鸣, 吴幸运. 最大相关与独立分类信息最大化特征选择算法[J]. 计算机技术与发展, 2020, 30(8): 46-52
ZHOU Chuanhua, LI Ming, WU Xingyun. Feature selection algorithm for maximizing maximum correlation and independent classification information[J]. Computer technology and development, 2020, 30(8): 46-52
[18] 张俐, 王枞. 基于最大相关最小冗余联合互信息的多标签特征选择算法[J]. 通信学报, 2018, 39(5): 111-122
ZHANG Li, WANG Cong. Multi-label feature selection algorithm based on joint mutual information of max-relevance and min-redundancy[J]. Journal on communications, 2018, 39(5): 111-122
[19] 李扬, 顾雪平. 基于改进最大相关最小冗余判据的暂态稳定评估特征选择[J]. 中国电机工程学报, 2013, 33(34): 179-186
LI Yang, GU Xueping. Feature selection for transient stability assessment based on improved maximal relevance and minimal redundancy criterion[J]. Proceedings of the CSEE, 2013, 33(34): 179-186
[20] 胡敏杰, 林耀进, 王晨曦, 等. 基于拉普拉斯评分的多标记特征选择算法[J]. 计算机应用, 2018, 38(11): 3167-3174
HU Minjie, LIN Yaojin, WANG Chenxi, et al. Multi-label feature selection algorithm based on Laplace score[J]. Journal of computer applications, 2018, 38(11): 3167-3174
[21] 陈谌, 梁雪春. 基于基尼指标和卡方检验的特征选择方法[J]. 计算机工程与设计, 2019, 40(8): 2342-2345, 2360
CHEN Chen, LIANG Xuechun. Feature selection method based on Gini index and chi-square test[J]. Computer engineering and design, 2019, 40(8): 2342-2345, 2360
[22] 徐明, 高翔, 许志刚, 等. 基于改进卡方统计的微博特征提取方法[J]. 计算机工程与应用, 2014, 50(19): 113-117, 142
XU Ming, GAO Xiang, XU Zhigang, et al. Microblog feature extraction method based on improved chi-square statistics[J]. Computer engineering and applications, 2014, 50(19): 113-117, 142
[23] BOMZE I M, DE KLERK E. Solving standard quadratic optimization problems via linear, semidefinite and copositive programming[J]. Journal of global optimization, 2002, 24(2): 163-185.
[24] LIU Hairong, YANG Xingwei, LATECKI L J, et al. Dense neighborhoods on affinity graph[J]. International journal of computer vision, 2012, 98(1): 65-82.
[25] DORA S, SUBRAMANIAN K, SURESH S, et al. Development of a self-regulating evolving spiking neural network for classification problem[J]. Neurocomputing, 2016, 171: 1216-1229.
- 相似文献/References:
-
[1]孙正兴,张尧烨,李? 彬.基于线性规划分类器的相关反馈技术[J].智能系统学报,2007,2(3):34.
SUN Zheng-xing,ZHANG Yao-ye,LI Bin.Applying relevance feedback with a linear programming classifier[J].CAAI Transactions on Intelligent Systems,2007,2():34.
[2]张志飞,苗夺谦.基于粗糙集的文本分类特征选择算法[J].智能系统学报,2009,4(5):453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
ZHANG Zhi-fei,MIAO Duo-qian.Feature selection for text categorization based on rough set[J].CAAI Transactions on Intelligent Systems,2009,4():453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
[3]顾成杰,张顺颐,杜安源.结合粗糙集和禁忌搜索的网络流量特征选择[J].智能系统学报,2011,6(3):254.
GU Chengjie,ZHANG Shunyi,DU Anyuan.Feature selection of network traffic using a rough set and tabu search[J].CAAI Transactions on Intelligent Systems,2011,6():254.
[4]孙倩茹,王文敏,刘宏.视频序列的人体运动描述方法综述[J].智能系统学报,2013,8(3):189.
SUN Qianru,WANG Wenmin,LIU Hong.Study of human action representation in video sequences[J].CAAI Transactions on Intelligent Systems,2013,8():189.
[5]曹晋,张莉,李凡长.一种基于支持向量数据描述的特征选择算法[J].智能系统学报,2015,10(2):215.[doi:10.3969/j.issn.1673-4785.201405063]
CAO Jin,ZHANG Li,LI Fanzhang.A noval support vector data description-based feature selection method[J].CAAI Transactions on Intelligent Systems,2015,10():215.[doi:10.3969/j.issn.1673-4785.201405063]
[6]张佳骕,蒋亦樟,王士同.基于特征选择聚类方法的稀疏TSK模糊系统[J].智能系统学报,2015,10(4):583.[doi:10.3969/j.issn.1673-4785.201412001]
ZHANG Jiasu,JIANG Yizhang,WANG Shitong.Sparse TSK fuzzy system based on feature selection clustering method[J].CAAI Transactions on Intelligent Systems,2015,10():583.[doi:10.3969/j.issn.1673-4785.201412001]
[7]陈玉明,吴克寿,李向军.基因表达数据在邻域关系中的特征选择[J].智能系统学报,2014,9(2):210.[doi:10.3969/j.issn.1673-4785.201307014]
CHEN Yuming,WU Keshou,LI Xiangjun.Gene expression data feature selection with neighborhood relation[J].CAAI Transactions on Intelligent Systems,2014,9():210.[doi:10.3969/j.issn.1673-4785.201307014]
[8]郭雨萌,李国正.一种多标记数据的过滤式特征选择框架[J].智能系统学报,2014,9(3):292.[doi:10.3969/j.issn.1673-4785.201403064]
GUO Yumeng,LI Guozheng.A filtering framework for the multi-label feature selection[J].CAAI Transactions on Intelligent Systems,2014,9():292.[doi:10.3969/j.issn.1673-4785.201403064]
[9]滕旭阳,董红斌,孙静.面向特征选择问题的协同演化方法[J].智能系统学报,2017,12(1):24.[doi:10.11992/tis.201611029]
TENG Xuyang,DONG Hongbin,SUN Jing.Co-evolutionary algorithm for feature selection[J].CAAI Transactions on Intelligent Systems,2017,12():24.[doi:10.11992/tis.201611029]
[10]路子祥,屠黎阳,祖辰,等.基于脑连接网络的阿尔茨海默病临床变量值预测[J].智能系统学报,2017,12(3):355.[doi:10.11992/tis.201607020]
LU Zixiang,TU Liyang,ZU Chen,et al.Prediction of clinical variables in Alzheimer’s disease using brain connective networks[J].CAAI Transactions on Intelligent Systems,2017,12():355.[doi:10.11992/tis.201607020]
备注/Memo
收稿日期:2020-09-11。
基金项目:山西省留学人员科技活动择优资助项目(2019-13)；山西省基础研究计划项目(201901D111320)；太原市科技计划研发项目(2018140105000084)；山西省高等学校精品共享课程项目(K2020022)
作者简介:李顺勇，教授，博士，主要研究方向为统计机器学习。发表学术论文30余篇;王改变，硕士研究生，主要研究方向为统计机器学习
通讯作者:李顺勇.E-mail:lisy75@sxu.edu.cn
更新日期/Last Update:
1900-01-01