[1]孙林,梁娜,徐久成.基于邻域互信息与K-means特征聚类的特征选择[J].智能系统学报,2024,19(4):983-996.[doi:10.11992/tis.202208012]
SUN Lin,LIANG Na,XU Jiucheng.Feature selection using neighborhood mutual information and feature clustering with K-means[J].CAAI Transactions on Intelligent Systems,2024,19(4):983-996.[doi:10.11992/tis.202208012]
点击复制
SUN Lin,LIANG Na,XU Jiucheng.Feature selection using neighborhood mutual information and feature clustering with K-means[J].CAAI Transactions on Intelligent Systems,2024,19(4):983-996.[doi:10.11992/tis.202208012]
基于邻域互信息与K-means特征聚类的特征选择
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第4期
页码:
983-996
栏目:
学术论文—知识工程
出版日期:
2024-07-05
- Title:
- Feature selection using neighborhood mutual information and feature clustering with K-means
- Keywords:
- feature selection; neighborhood mutual information; K-means; feature clustering; adaptive K-nearest neighbor; feature weight; weighted k-nearest neighbor density
- 分类号:
- TP181
- 摘要:
- 针对多数邻域系统通过人工调试很难搜索到最佳邻域半径，以及传统的K-means聚类需要随机选取簇中心和指定簇的数目等问题，提出了一种基于邻域互信息与K-means特征聚类的特征选择方法。首先，将样本在各特征下与其他样本距离的平均值作为自适应邻域半径，确定样本的邻域集，并由此构建自适应邻域熵、邻域互信息、归一化邻域互信息等度量，反映特征之间的相关性；然后，基于归一化邻域互信息构建自适应K近邻集合，利用Pearson相关系数表示特征的权重定义加权K近邻密度，实现自动选取K-means算法的簇中心，进而完成K-means特征聚类；最后，给出加权平均冗余度，选出每个特征簇中加权平均冗余度最大的特征构成最优特征子集。实验结果表明所提算法不仅可以有效提升特征选择的分类结果而且可以获得更好的聚类效果。
- 参考文献/References:
-
[1] SONG Xianfang, ZHANG Yong, GONG Dunwei, et al. A fast hybrid feature selection based on correlation-guided clustering and particle swarm optimization for high- dimensional data[J]. IEEE transactions on cybernetics, 2022, 52(9): 9573–9586.
[2] 梁云辉, 甘舰文, 陈艳, 等. 基于对偶流形重排序的无监督特征选择算法[J]. 计算机科学, 2023, 50(7): 72–81
LIANG Yunhui, GANJianwen, CHEN Yan, et al. Unsupervised feature selection algorithm based on dual manifold re-ranking[J]. Computer science, 2023, 50(7): 72–81
[3] 侯天宝, 王爱银. 基于Stacking特征增强多粒度联级Logistic的个人信用评估[J]. 河南师范大学学报(自然科学版), 2023, 51(3): 111–122
HOU Tianbao, WANG Aiyin. Personal credit evaluation based on stacking feature enhancing multi-grained cascade logistic[J]. Journal of Henan Normal University (natural science edition), 2023, 51(3): 111–122
[4] 杨洁, 匡俊成, 王国胤, 等. 代价敏感的多粒度邻域粗糙模糊集的近似表示[J]. 计算机科学, 2023, 50(5): 137–145
YANG Jie, KUANG Juncheng, WANG Guoyin, et al. Cost-sentitive multigranulation approximation of neighborhood rough fuzzy sets[J]. Computer science, 2023, 50(5): 137–145
[5] HU Qinghua, ZHANG Lei, ZHANG David, et al. Measuring relevance between discrete and continuous features based on neighborhood mutual information[J]. Expert systems with applications, 2011, 38: 10737–10750.
[6] 孙林, 梁娜, 徐久成. 基于自适应邻域互信息与谱聚类的特征选择[J]. 山东大学学报(理学版), 2022, 57(12): 13–24
SUN Lin, LIANG Na, XU Jiucheng. Feature selection using adaptive neighborhood mutual information and spectral clustering[J]. Journal of Shandong University (nature science edition), 2022, 57(12): 13–24
[7] RAHMANIAN M, MANSOORI E. Unsupervised fuzzy multivariate symmetric uncertainty feature selection based on constructing virtual cluster representative[J]. Fuzzy sets and systems, 2022, 438: 148–163.
[8] 孙林, 秦小营, 徐久成, 等. 基于K近邻和优化分配策略的密度峰值聚类算法[J]. 软件学报, 2022, 33(4): 1390–1411
SUN Lin, QIN Xiaoying, XU Jiucheng, et al. Density peak clustering algorithm based on k-nearest neighbors and optimized allocation[J]. Journal of software, 2022, 33(4): 1390–1411
[9] 张新元, 贠卫国. 共享K近邻和多分配策略的密度峰值聚类算法[J]. 小型微型计算机系统, 2023, 44(1): 75–82
ZHANG Xinyuan, YUN Weiguo. Sharing K-nearest neighbors and multiple assignment policies density peaks[J]. Journal of chinese computer systems, 2023, 44(1): 75–82
[10] 孙林, 李梦梦, 徐久成. 二进制哈里斯鹰优化及其特征选择算法[J]. 计算机科学, 2023, 50(5): 277–291
SUN Lin, LI Mengmeng, XU Jiucheng. Binary Harris hawk optimization and its feature selection algorithm[J]. Computer science, 2023, 50(5): 277–291
[11] 曹栋涛, 舒文豪, 钱进. 基于粗糙集与密度峰值聚类的特征选择算法[J]. 计算机科学, 2023, 50(10): 37–47
CAO Dongtao, SHU Wenhao, QIAN Jin. Feature selection algorithm based on rough setand density peak clustering[J]. Computer science, 2023, 50(10): 37–47
[12] 徐天杰, 王平心, 杨习贝. 基于人工蜂群的三支k-means聚类算法[J]. 计算机科学, 2023, 50(6): 116–121
XU Tianjie, WANG Pingxin, YANG Xibei. Three-way k-means clustering based on artificial bee colony[J]. Computer science, 2023, 50(6): 116–121
[13] 李冰晓, 万睿之, 朱永杰, 等. 基于种群分区的多策略综合粒子群优化算法[J]. 河南师范大学学报(自然科学版), 2022, 50(3): 85–94
LI Bingxiao, WAN Ruizhi, ZHU Yongjie, et al. Multi-strategy comprehensive particle swarm optimization algorithm based on population partition[J]. Journal of Henan Normal University (natural science edition), 2022, 50(3): 85–94
[14] CHEN Zhijun, CHEN Qiushi, ZHANG Yishi, et al. Clustering-based feature subset selection with analysis on the redundancy-complementarity dimension[J]. Computer communications, 2021, 168: 65–74.
[15] SUN Lin, QIN Xiaoying, DING Weiping, et al. Nearest neighbors-based adaptive density peaks clustering with optimized allocation strategy[J]. Neurocomputing, 2022, 473: 159–181.
[16] 辛永杰, 蔡江辉, 贺艳婷, 等. 基于跨结构特征选择和图循环自适应学习的多视图聚类[J/OL]. 计算机科学, 2024. [2024-05-14]. https://link.cnki.net/urlid/50.1075.tp.20240513.1427.017.
XIN Yongjie, CAI Jianghui, HE Yanting et al. Multi-view clustering based on cross-structural feature selection and graph cycle adaptive learning[J/OL]. Computer science, 2024. [2024-05-14]. https://link.cnki.net/urlid/50.1075.tp.20240513.1427.017.
[17] 孙林, 马天娇. 基于中心偏移的Fisher score与直觉邻域模糊熵的多标记特征选择[J/OL]. 计算机科学, (2023-11-14)[2023-12-06]. https://link.cnki.net/urlid/50.1075.TP.20231113.1009.012.
SUN Lin, MA Tianjiao. Multilabel feature selection using fisher score with center shift and neighborhood intuitionistic fuzzy entropy[J/OL]. Computer science, (2023-11-14)[2023-12-06].https://link.cnki. net/urlid/50.1075.TP.20231113.1009.012.
[18] 赵燕伟, 朱芬, 桂方志, 等. 基于可拓距的改进K-means聚类算法[J]. 智能系统学报, 2020, 15(2): 344–351
ZHAO Yanwei, ZHU Fen, GUI Fangzhi, et al. Improved K-means algorithm based on extension distance[J]. CAAI transactions on intelligent systems, 2020, 15(2): 344–351
[19] 王雷, 杜亮, 周芃. 基于稀疏连接的层次化多核K-Means算法[J]. 计算机科学, 2023, 50(2): 138–145
WAND Lei, DU Liang, ZHOU Peng. Hierarchical multiple kernel K-means algorithm based on sparse connectivity[J]. Computer science, 2023, 50(2): 138–145
[20] GUSTAVO S C, MIGUEL G T, SANTIAGO G G, et al. A multivariate approach to the symmetrical uncertainty measure: Application to feature selection problem[J]. Information sciences, 2019, 494: 1–20.
[21] ZHU Xiaoyan, WANG Yu, LI Yingbin, et al. A new unsupervised feature selection algorithm using similarity-based feature clustering[J]. Computational intelligence, 2019, 35(1): 2–22.
[22] ALHELALI B, CHEN Qi, XUE Bing, et al. A new imputation method based on genetic programming and weighted KNN for symbolic regression with incomplete data[J]. Soft computing, 2021, 25(8): 5993–6012.
[23] SUN Lin, ZHANG Jiuxiao, DING Weiping, et al. Mixed measure-based feature selection using the Fisher score and neighborhood rough sets[J]. Applied intelligence, 2022, 52: 17264–17288.
[24] ZHANG Li, CHEN Xiaobo. Feature selection methods based on symmetric uncertainty coefficients and independent classification information[J]. IEEE access, 2021, 9: 13845–13856.
[25] LIN Xiaohui, LI Chao, REN Wenjie, et al. A new feature selection method based on symmetrical uncertainty and interaction gain[J]. Computational biology and chemistry, 2019, 83: 107149.
[26] RAHMANIAN M, MANSOORI E G. An unsupervised gene selection method based on multivariate normalized mutual information of genes[J]. Chemometrics and intelligent laboratory systems, 2022, 222: 104512.
[27] FAIVISHEVSKY L, GOLDBERGER J. Unsupervised feature selection based on non-parametric mutual information[C]// 2012 IEEE international workshop on machine learning for signal processing. Santander: IEEE, 2012: 1-6.
[28] SUN Xin, LIU Yanheng, WEI Da, et al. Selection of interdependent genes via dynamic relevance analysis for cancer diagnosis[J]. Journal of biomedical informatics, 2013, 46(2): 252–258.
[29] WANG Jun, WEI Jinmao, YANG Zhenglu, et al. Feature selection by maximizing independent classification information[J]. IEEE transactions on knowledge and data engineering, 2017, 29(4): 828–841.
[30] MACQUEEN J. B. Some methods for classification and analysis of multivariate observations[C]//Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability. Los Angeles: University of California Press, 1967: 281-297.
[31] KAUFMAN L, ROUSSEEUW P J. Finding groups in data: an introduction to cluster analysis[M]. Hoboken: Wiley Online Library, 1991.
[32] ESTER M, KRIEGEL H P, SANDER J, et al. A density-based algorithm for discovering clusters in large spatial databases with noise[C]//Proceedings of the 2th International Conference on Knowledge Discovery and Data Mining. Muenchen: AAAI Press, 1996: 226–231.
[33] ANKERST M, BREUNIG M M, KRIEGEL H P, et al. OPTICS: ordering points to identify the clustering structure [C]//Proceedings of ACM SIGMOD International Conference on Management of Data. New York: ACM, 1999: 49–60.
[34] ZENG Zilin, ZHANG Hongjun, ZHANG Rui, et al. A novel feature selection method considering feature interaction[J]. Pattern recognition, 2015, 48(8): 2656–2666.
[35] 刘杰, 张平, 高万夫. 基于条件相关的特征选择方法[J]. 吉林大学学报(工学版), 2018, 48(3): 874–881
LIU Jie, ZHANG Ping, GAO Wanfu. Feature selection method based on conditional relevance[J]. Journal of Jilin University (engineering and technology edition), 2018, 48(3): 874–881
[36] 孙林, 徐枫, 李硕, 等. 基于ReliefF和最大相关最小冗余的多标记特征选择[J]. 河南师范大学学报(自然科学版), 2023, 51(6): 21–29
SUN Lin, XU Feng, LI Shuo, et al. Multilabel feature selection algorithm using ReliefF and mRMR[J]. Journal of Henan Normal University (natural science edition), 2023, 51(6): 21–29
- 相似文献/References:
-
[1]孙正兴,张尧烨,李? 彬.基于线性规划分类器的相关反馈技术[J].智能系统学报,2007,2(3):34.
SUN Zheng-xing,ZHANG Yao-ye,LI Bin.Applying relevance feedback with a linear programming classifier[J].CAAI Transactions on Intelligent Systems,2007,2():34.
[2]张志飞,苗夺谦.基于粗糙集的文本分类特征选择算法[J].智能系统学报,2009,4(5):453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
ZHANG Zhi-fei,MIAO Duo-qian.Feature selection for text categorization based on rough set[J].CAAI Transactions on Intelligent Systems,2009,4():453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
[3]顾成杰,张顺颐,杜安源.结合粗糙集和禁忌搜索的网络流量特征选择[J].智能系统学报,2011,6(3):254.
GU Chengjie,ZHANG Shunyi,DU Anyuan.Feature selection of network traffic using a rough set and tabu search[J].CAAI Transactions on Intelligent Systems,2011,6():254.
[4]孙倩茹,王文敏,刘宏.视频序列的人体运动描述方法综述[J].智能系统学报,2013,8(3):189.
SUN Qianru,WANG Wenmin,LIU Hong.Study of human action representation in video sequences[J].CAAI Transactions on Intelligent Systems,2013,8():189.
[5]曹晋,张莉,李凡长.一种基于支持向量数据描述的特征选择算法[J].智能系统学报,2015,10(2):215.[doi:10.3969/j.issn.1673-4785.201405063]
CAO Jin,ZHANG Li,LI Fanzhang.A noval support vector data description-based feature selection method[J].CAAI Transactions on Intelligent Systems,2015,10():215.[doi:10.3969/j.issn.1673-4785.201405063]
[6]张佳骕,蒋亦樟,王士同.基于特征选择聚类方法的稀疏TSK模糊系统[J].智能系统学报,2015,10(4):583.[doi:10.3969/j.issn.1673-4785.201412001]
ZHANG Jiasu,JIANG Yizhang,WANG Shitong.Sparse TSK fuzzy system based on feature selection clustering method[J].CAAI Transactions on Intelligent Systems,2015,10():583.[doi:10.3969/j.issn.1673-4785.201412001]
[7]陈玉明,吴克寿,李向军.基因表达数据在邻域关系中的特征选择[J].智能系统学报,2014,9(2):210.[doi:10.3969/j.issn.1673-4785.201307014]
CHEN Yuming,WU Keshou,LI Xiangjun.Gene expression data feature selection with neighborhood relation[J].CAAI Transactions on Intelligent Systems,2014,9():210.[doi:10.3969/j.issn.1673-4785.201307014]
[8]郭雨萌,李国正.一种多标记数据的过滤式特征选择框架[J].智能系统学报,2014,9(3):292.[doi:10.3969/j.issn.1673-4785.201403064]
GUO Yumeng,LI Guozheng.A filtering framework for the multi-label feature selection[J].CAAI Transactions on Intelligent Systems,2014,9():292.[doi:10.3969/j.issn.1673-4785.201403064]
[9]滕旭阳,董红斌,孙静.面向特征选择问题的协同演化方法[J].智能系统学报,2017,12(1):24.[doi:10.11992/tis.201611029]
TENG Xuyang,DONG Hongbin,SUN Jing.Co-evolutionary algorithm for feature selection[J].CAAI Transactions on Intelligent Systems,2017,12():24.[doi:10.11992/tis.201611029]
[10]路子祥,屠黎阳,祖辰,等.基于脑连接网络的阿尔茨海默病临床变量值预测[J].智能系统学报,2017,12(3):355.[doi:10.11992/tis.201607020]
LU Zixiang,TU Liyang,ZU Chen,et al.Prediction of clinical variables in Alzheimer’s disease using brain connective networks[J].CAAI Transactions on Intelligent Systems,2017,12():355.[doi:10.11992/tis.201607020]
备注/Memo
收稿日期:2022-08-12。
基金项目:国家自然科学基金项目（62076089, 61772176, 61976082）；河南省科技攻关计划项目(212102210136).
作者简介:孙林，教授，博士生导师，博士，计算机学会会员，主要研究方向为粒计算、大数据挖掘和机器学习。发表学术论文60余篇。E-mail：sunlin@tust.edu.cn;梁娜，硕士研究生，主要研究方向为数据挖掘。E-mail：ms_liangna@126.com;徐久成，教授，博士生导师，博士，计算机学会高级会员，主要研究方向为粒计算、大数据挖掘和智能信息处理。E-mail：xjc@htu.edu.cn
通讯作者:孙林. E-mail：sunlin@tust.edu.cn
更新日期/Last Update:
1900-01-01