[1]黎建宇,詹志辉.面向大规模特征选择的自监督数据驱动粒子群优化算法[J].智能系统学报,2023,18(1):194-206.[doi:10.11992/tis.202206008]
LI Jianyu,ZHAN Zhihui.A self-supervised data-driven particle swarm optimization approach for large-scale feature selection[J].CAAI Transactions on Intelligent Systems,2023,18(1):194-206.[doi:10.11992/tis.202206008]
点击复制
LI Jianyu,ZHAN Zhihui.A self-supervised data-driven particle swarm optimization approach for large-scale feature selection[J].CAAI Transactions on Intelligent Systems,2023,18(1):194-206.[doi:10.11992/tis.202206008]
面向大规模特征选择的自监督数据驱动粒子群优化算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第1期
页码:
194-206
栏目:
吴文俊人工智能科学技术奖论坛
出版日期:
2023-01-05
- Title:
- A self-supervised data-driven particle swarm optimization approach for large-scale feature selection
- Keywords:
- feature selection; large-scale optimization; particle swarm optimization; evolutionary computation; swarm intelligence; data-driven; self-supervised learning; discrete region encoding
- 分类号:
- TP391
- 摘要:
- 大规模特征选择问题的求解通常面临两大挑战：一是真实标签不足，难以引导算法进行特征选择；二是搜索空间规模大，难以搜索到满意的高质量解。为此，提出了新型的面向大规模特征选择的自监督数据驱动粒子群优化算法。第一，提出了自监督数据驱动特征选择的新型算法框架，可不依赖于真实标签进行特征选择。第二，提出了基于离散区域编码的搜索策略，帮助算法在大规模搜索空间中找到更优解。第三，基于上述的框架和方法，提出了自监督数据驱动粒子群优化算法，实现对问题的求解。在大规模特征数据集上的实验结果显示，提出的算法与主流有监督算法表现相当，并比前沿无监督算法具有更高的特征选择效率。
- Abstract:
- Large-scale feature selection problems usually face two challenges: 1) Real labels are insufficient for guiding the algorithm to select features, and 2) a large-scale search space encumbers the search for a satisfactory high-quality solution. To this end, in this paper, a novel self-supervised data-driven particle swarm optimization algorithm is proposed for large-scale feature selection, including three contributions. First, a novel algorithmic framework named self-supervised data-driven feature selection is proposed, which can perform the feature selection without real labels. Second, a discrete region encoding-based search strategy is proposed, which helps the algorithm to find better solutions in a large-scale search space. Third, based on the above framework and method, a self-supervised data-driven particle swarm optimization algorithm is proposed to solve the large-scale feature selection problem. Experimental results on datasets with large-scale features show that the proposed algorithm performs comparably to the mainstream supervised algorithms and has higher feature selection efficiency than state-of-the-art unsupervised algorithms.
- 参考文献/References:
-
[1] XUE Bing, ZHANG Mengjie, BROWNE W N, et al. A survey on evolutionary computation approaches to feature selection[J]. IEEE transactions on evolutionary computation, 2016, 20(4): 606–626.
[2] ZHAN Zhihui, SHI Lin, TAN K C, et al. A survey on evolutionary computation for complex continuous optimization[J]. Artificial intelligence review, 2022, 55(1): 59–110.
[3] WANG Peng, XUE Bing, LIANG Jing, et al. Differential evolution based feature selection: a niching-based multi-objective approach[J]. IEEE transactions on evolutionary computation, PP(99): 1.
[4] ZHAN Zhihui, LI Jianyu, ZHANG Jun. Evolutionary deep learning: a survey[J]. Neurocomputing, 2022, 483: 42–58.
[5] CHENG Fan, CUI Junjie, WANG Qijun, et al. A variable granularity search based multi-objective feature selection algorithm for high-dimensional data classification[EB/OL].(2022–03–18)[2022–06–01].https://ieeexplore.ieee.org/abstract/document/9737335.
[6] LI Jianyu, ZHAN Zhihui, XU Jin, et al. Surrogate-assisted hybrid-model estimation of distribution algorithm for mixed-variable hyperparameters optimization in convolutional neural networks[EB/OL]. (2021–09–20)[2022–06–01].https://ieeexplore.ieee.org/document/9540902.
[7] SONG Xianfang, ZHANG Yong, GONG Dunwei, et al. Surrogate sample-assisted particle swarm optimization for feature selection on high-dimensional data[EB/OL]. (2022–05–18)[2022–06–01].https://ieeexplore.ieee.org/abstract/document/9775183
[8] 李永豪, 胡亮, 高万夫. 基于稀疏系数矩阵重构的多标记特征选择[J]. 计算机学报, 2022, 45(9): 1827–1841
LI Yonghao, HU Liang, GAO Wanfu. Multi-label feature selection based on sparse coefficient matrix reconstruction[J]. Chinese journal of computers, 2022, 45(9): 1827–1841
[9] LI Junyu, CHEN Jiazhou, QI Fei, et al. Two-dimensional unsupervised feature selection via sparse feature filter[EB/OL]. (2022–04–11)[2022–06–01].https://ieeexplore.ieee.org/abstract/document/9754711.
[10] WANG Peng, XUE Bing, LIANG Jing, et al. Multiobjective differential evolution for feature selection in classification[EB/OL]. (2021–12–07) [2022–06–01].https://pubmed.ncbi.nlm.nih.gov/34874881.
[11] 陈彤, 陈秀宏. 特征自表达和图正则化的鲁棒无监督特征选择[J]. 智能系统学报, 2022, 17(2): 286–294
CHEN Tong, CHEN Xiuhong. Feature self-representation and graph regularization for robust unsupervised feature selection[J]. CAAI transactions on intelligent systems, 2022, 17(2): 286–294
[12] LI Xiaoping, WANG Yadi, RUIZ R. A survey on sparse learning models for feature selection[J]. IEEE transactions on cybernetics, 2022, 52(3): 1642–1660.
[13] 李顺勇, 王改变. 一种新的最大相关最小冗余特征选择算法[J]. 智能系统学报, 2021, 16(4): 649–661
LI Shunyong, WANG Gaibian. New MRMR feature selection algorithm[J]. CAAI transactions on intelligent systems, 2021, 16(4): 649–661
[14] LIU Shulei, WANG Handing, PENG Wei, et al. A surrogate-assisted evolutionary feature selection algorithm with parallel random grouping for high-dimensional classification[J]. IEEE transactions on evolutionary computation, 2022, 26(5): 1087–1101.
[15] 曾毓菁, 姜勇. 一种融入注意力和预测的特征选择SLAM算法[J]. 智能系统学报, 2021, 16(6): 1039–1044
ZENG Yujing, JIANG Yong. Feature selection simultaneous localization and mapping algorithm incorporating attention and anticipation[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1039–1044
[16] ZHANG N, GUPTA A, CHEN Zefeng, et al. Evolutionary machine learning with minions: a case study in feature selection[J]. IEEE transactions on evolutionary computation, 2022, 26(1): 130–144.
[17] YANG Jiaquan, CHEN Chunhua, LI Jianyu, et al. Compressed-encoding particle swarm optimization with fuzzy learning for large-scale feature selection[J]. Symmetry, 2022, 14(6): 1142.
[18] CHEN Ke, XUE Bing, ZHANG Mengjie, et al. Correlation-guided updating strategy for feature selection in classification with surrogate-assisted particle swarm optimisation[EB/OL]. (2021–12–13)[2022–06–01].https://ieeexplore.ieee.org/abstract/document/9647020.
[19] WANG Zijia, JIAN Junrong, ZHAN Zhihui, et al. Gene targeting differential evolution: a simple and efficient method for large scale optimization[EB/OL]. (2022–06–23)[2022–06–23].https://ieeexplore.ieee.org/abstract/document/9804806.
[20] ZHANG Yong, WANG Yanhu, GONG Dunwei, et al. Clustering-guided particle swarm feature selection algorithm for high-dimensional imbalanced data with missing values[J]. IEEE transactions on evolutionary computation, 2022, 26(4): 616–630.
[21] WANG Yequn, LI Jianyu, CHEN Chunhua, et al. Scale adaptive fitness evaluation-based particle swarm optimisation for hyperparameter and architecture optimisation in neural networks and deep learning[EB/OL]. (2022–06–02)[2022–06–02].https://ietresearch.onlinelibrary.wiley.com/doi/full/10.1049/cit2.12106.
[22] HE Chunlin, ZHANG Yong, GONG Dunwei, et al. A multi-task bee colony band selection algorithm with variable-size clustering for hyperspectral images[EB/OL]. (2022–03–14) [2022–06–02].https://ieeexplore.ieee.org/abstract/document/9733922.
[23] 陈宗淦, 詹志辉. 面向多峰优化问题的双层协同差分进化算法[J]. 计算机学报, 2021, 44(9): 1806–1823
CHEN Zonggan, ZHAN Zhihui. Two-layer collaborative differential evolution algorithm for multimodal optimization problems[J]. Chinese journal of computers, 2021, 44(9): 1806–1823
[24] KENNEDY J, EBERHART R. Particle swarm optimization[C]//Proceedings of ICNN’95-International Conference on Neural Networks. Perth: IEEE, 1995: 1942–1948.
[25] KENNEDY J, EBERHART R C. A discrete binary version of the particle swarm algorithm[C]//1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation. Orlando: IEEE, 1997: 4104–4108.
[26] GU Shenkai, CHENG Ran, JIN Yaochu. Feature selection for high-dimensional classification using a competitive swarm optimizer[J]. Soft computing, 2018, 22(3): 811–822.
[27] TRAN Binh, XUE Bing, ZHANG Mengjie. Variable-length particle swarm optimization for feature selection on high-dimensional classification[J]. IEEE transactions on evolutionary computation, 2019, 23(3): 473–487.
[28] XUE Yu, TANG Tao, PANG Wei, et al. Self-adaptive parameter and strategy based particle swarm optimization for large-scale feature selection problems with multiple classifiers[J]. Applied soft computing, 2020, 88: 106031.
[29] LUO Chuan, WANG Sizhao, LI Tianrui, et al. Large-scale meta-heuristic feature selection based on BPSO assisted rough hypercuboid approach[EB/OL]. (2022–05–12) [2022–06–02].https://ieeexplore.ieee.org/abstract/document/9773310.
[30] JIAN Junrong, CHEN Zonggan, ZHAN Zhihui, et al. Region encoding helps evolutionary computation evolve faster: a new solution encoding scheme in particle swarm for large-scale optimization[J]. IEEE transactions on evolutionary computation, 2021, 25(4): 779–793.
[31] HE Xiaofei, YAN Shuicheng, HU Yuxiao, et al. Face recognition using Laplacianfaces[J]. IEEE transactions on pattern analysis and machine intelligence, 2005, 27(3): 328–340.
[32] CAI Deng, HE Xiaofei, HAN Jiawei, et al. Orthogonal laplacianfaces for face recognition[J]. IEEE transactions on image processing, 2006, 15(11): 3608–3614.
[33] CAI Deng, HE Xiaofei, HAN Jiawei, et al. Graph regularized nonnegative matrix factorization for data representation[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 33(8): 1548–1560.
[34] CHEN Ke, XUE Bing, ZHANG Mengjie, et al. An evolutionary multitasking-based feature selection method for high-dimensional classification[J]. IEEE transactions on cybernetics, 2022, 52(7): 7172–7186.
[35] JING Longlong, TIAN Yingli. Self-supervised visual feature learning with deep neural networks: a survey[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(11): 4037–4058.
[36] SARKAR P, ETEMAD A. Self-supervised ECG representation learning for emotion recognition[J]. IEEE transactions on affective computing, 2022, 13(3): 1541–1554.
[37] HSU W N, BOLTE B, TSAI Y H H, et al. HuBERT: self-supervised speech representation learning by masked prediction of hidden units[J]. IEEE/ACM transactions on audio, speech, and language processing, 2021, 29: 3451–3460.
[38] YUAN Aihong, YOU Mengbo, HE Dongjian, et al. Convex non-negative matrix factorization with adaptive graph for unsupervised feature selection[J]. IEEE transactions on cybernetics, 2022, 52(6): 5522–5534.
[39] YOU Mengbo, YUAN Aihong, ZOU Min, et al. Robust unsupervised feature selection via multi-group adaptive graph representation[EB/OL]. (2021–11–08)[2022–06–1].https://ieeexplore.ieee.org/abstract/document/9606609.
[40] ZHANG Rui, ZHANG Yunxing, LI Xuelong. Unsupervised feature selection via adaptive graph learning and constraint[J]. IEEE transactions on neural networks and learning systems, 2022, 33(3): 1355–1362.
[41] CHANG Heng, GUO Jun, ZHU Wenwu. Rethinking embedded unsupervised feature selection: a simple joint approach[EB/OL]. (2022–05–30)[2022–06–01].https://ieeexplore.ieee.org/abstract/document/9784919.
[42] SOLORIO-FERNáNDEZ S, CARRASCO-OCHOA J A, MARTíNEZ-TRINIDAD J F. A review of unsupervised feature selection methods[J]. Artificial intelligence review, 2020, 53(2): 907–948.
[43] LI Jundong, CHENG Kewei, WANG Suhang, et al. Feature selection: a data perspective[J]. ACM computing surveys, 2018, 50(6): 94.
- 相似文献/References:
-
[1]孙正兴,张尧烨,李? 彬.基于线性规划分类器的相关反馈技术[J].智能系统学报,2007,2(3):34.
SUN Zheng-xing,ZHANG Yao-ye,LI Bin.Applying relevance feedback with a linear programming classifier[J].CAAI Transactions on Intelligent Systems,2007,2():34.
[2]张志飞,苗夺谦.基于粗糙集的文本分类特征选择算法[J].智能系统学报,2009,4(5):453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
ZHANG Zhi-fei,MIAO Duo-qian.Feature selection for text categorization based on rough set[J].CAAI Transactions on Intelligent Systems,2009,4():453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
[3]顾成杰,张顺颐,杜安源.结合粗糙集和禁忌搜索的网络流量特征选择[J].智能系统学报,2011,6(3):254.
GU Chengjie,ZHANG Shunyi,DU Anyuan.Feature selection of network traffic using a rough set and tabu search[J].CAAI Transactions on Intelligent Systems,2011,6():254.
[4]孙倩茹,王文敏,刘宏.视频序列的人体运动描述方法综述[J].智能系统学报,2013,8(3):189.
SUN Qianru,WANG Wenmin,LIU Hong.Study of human action representation in video sequences[J].CAAI Transactions on Intelligent Systems,2013,8():189.
[5]曹晋,张莉,李凡长.一种基于支持向量数据描述的特征选择算法[J].智能系统学报,2015,10(2):215.[doi:10.3969/j.issn.1673-4785.201405063]
CAO Jin,ZHANG Li,LI Fanzhang.A noval support vector data description-based feature selection method[J].CAAI Transactions on Intelligent Systems,2015,10():215.[doi:10.3969/j.issn.1673-4785.201405063]
[6]张佳骕,蒋亦樟,王士同.基于特征选择聚类方法的稀疏TSK模糊系统[J].智能系统学报,2015,10(4):583.[doi:10.3969/j.issn.1673-4785.201412001]
ZHANG Jiasu,JIANG Yizhang,WANG Shitong.Sparse TSK fuzzy system based on feature selection clustering method[J].CAAI Transactions on Intelligent Systems,2015,10():583.[doi:10.3969/j.issn.1673-4785.201412001]
[7]陈玉明,吴克寿,李向军.基因表达数据在邻域关系中的特征选择[J].智能系统学报,2014,9(2):210.[doi:10.3969/j.issn.1673-4785.201307014]
CHEN Yuming,WU Keshou,LI Xiangjun.Gene expression data feature selection with neighborhood relation[J].CAAI Transactions on Intelligent Systems,2014,9():210.[doi:10.3969/j.issn.1673-4785.201307014]
[8]郭雨萌,李国正.一种多标记数据的过滤式特征选择框架[J].智能系统学报,2014,9(3):292.[doi:10.3969/j.issn.1673-4785.201403064]
GUO Yumeng,LI Guozheng.A filtering framework for the multi-label feature selection[J].CAAI Transactions on Intelligent Systems,2014,9():292.[doi:10.3969/j.issn.1673-4785.201403064]
[9]滕旭阳,董红斌,孙静.面向特征选择问题的协同演化方法[J].智能系统学报,2017,12(1):24.[doi:10.11992/tis.201611029]
TENG Xuyang,DONG Hongbin,SUN Jing.Co-evolutionary algorithm for feature selection[J].CAAI Transactions on Intelligent Systems,2017,12():24.[doi:10.11992/tis.201611029]
[10]路子祥,屠黎阳,祖辰,等.基于脑连接网络的阿尔茨海默病临床变量值预测[J].智能系统学报,2017,12(3):355.[doi:10.11992/tis.201607020]
LU Zixiang,TU Liyang,ZU Chen,et al.Prediction of clinical variables in Alzheimer’s disease using brain connective networks[J].CAAI Transactions on Intelligent Systems,2017,12():355.[doi:10.11992/tis.201607020]
备注/Memo
收稿日期:2022-06-06。
基金项目:国家重点研发计划项目(2019YFB2102102);国家自然科学基金面上项目资助(62176094).
作者简介:黎建宇,博士研究生,主要研究方向为人工智能、进化计算、群体智能、知识学习与数据驱动;詹志辉,教授,博士生导师,Elsevier中国高被引学者,主要研究方向为人工智能、进化计算、群体智能、云计算和大数据。荣获吴文俊人工智能优秀青年奖和IEEE计算智能学会杰出青年奖。目前已在国际期刊和国际会议发表(录用)论文共150余篇,其中IEEE Transactions系列的计算机领域顶尖国际期刊论文60余篇。论文近被国际同行引用超过一万次(Google Scholar),其中SCI引用超过5000次。11篇论文先后入选ESI高被引(全球影响力排名前百分之一)论文,包括1篇ESI热点(全球影响力排名前千分之一)论文
通讯作者:詹志辉.E-mail:zhanapollo@163.com
更新日期/Last Update:
1900-01-01