[1]过伶俐,陈秀宏.潜在多步马尔可夫概率的鲁棒无监督特征选择[J].智能系统学报,2023,18(5):1017-1029.[doi:10.11992/tis.202208013]
GUO Lingli,CHEN Xiuhong.Robust unsupervised feature selection via multistep Markov probability and latent representation[J].CAAI Transactions on Intelligent Systems,2023,18(5):1017-1029.[doi:10.11992/tis.202208013]
点击复制
GUO Lingli,CHEN Xiuhong.Robust unsupervised feature selection via multistep Markov probability and latent representation[J].CAAI Transactions on Intelligent Systems,2023,18(5):1017-1029.[doi:10.11992/tis.202208013]
潜在多步马尔可夫概率的鲁棒无监督特征选择
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第5期
页码:
1017-1029
栏目:
学术论文—机器学习
出版日期:
2023-09-05
- Title:
- Robust unsupervised feature selection via multistep Markov probability and latent representation
- Keywords:
- feature selection; latent representation learning; multistep Markov transition probability; unsupervised; non-negative matrix factorization; sparse regression; L2; 1-norm; dimensionality reduction
- 分类号:
- TP181
- 摘要:
- 无监督特征选择是机器学习和数据挖掘中的一种重要的降维技术。然而当前的无监督特征选择方法侧重于从数据的邻接矩阵中学习数据的流形结构，忽视非邻接数据对之间的关联。其次这些方法都假设数据实例具有独立同一性，但现实中的数据样本其来源是不同的，这样的假设就不成立。此外，在原始数据空间中特征重要性的衡量会受到数据和特征中的噪声影响。基于以上问题，本文提出了潜在多步马尔可夫概率的鲁棒无监督特征选择方法（unsupervised feature selection via multi-step Markov probability and latent representation, MMLRL），其思想是通过最大多步马尔可夫转移概率学习数据流形结构，然后通过对称非负矩阵分解模型学习数据的潜在表示，最后在数据的潜在表示空间中选择特征。同时在6个不同类型的数据集上验证了所提出算法的有效性。
- Abstract:
- Unsupervised feature selection is a significant dimensionality reduction technique in machine learning and data mining. However, current unsupervised feature selection methods primarily focus on learning the manifold structure of the data from the adjacency matrix, ignoring the association between non-adjacent data pairs. Second, these methods often assume that the data instances are independent and identically distributed, but in reality, the data samples originate from heterogeneous sources, and this assumption is often untenable. Additionally, the measure of feature importance in the original data space is affected by noise in the data and features. To address the aforementioned problems, this study proposes a robust unsupervised feature selection method based on multistep Markov probability and latent representation (MMLRL). The key idea is to learn the manifold structure between the data points through the maximum multistep Markov transition probability. Subsequently, a symmetric non-negative matrix factorization model was used to learn the latent representation of the data. Finally, the feature selection is performed in the latent representation space. At the same time, the proposed algorithm is evaluated on six different types of datasets to validate its effectiveness.
- 参考文献/References:
-
[1] BRUNETTI A, BUONGIORNO D, TROTTA G F, et al. Computer vision and deep learning techniques for pedestrian detection and tracking: a survey[J]. Neurocomputing, 2018, 300: 17-33.
[2] JORDAN M I, MITCHELL T M. Machine learning: trends, perspectives, and prospects[J]. Science, 2015, 349(6245): 255-260.
[3] LI H, HE X, TAO D, et al. Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning[J]. Pattern recognition, 2018, 79: 130-146.
[4] QURESHI R, UZAIR M, KHURSHID K, et al. Hyperspectral document image processing: applications, challenges and future prospects[J]. Pattern recognition, 2019, 90: 12-22.
[5] VADIM K. Overview of different approaches to solving problems of data mining[J]. Procedia computers, 2018, 123: 234-239.
[6] PENG H C, LONG F H, DING C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE transactions on pattern analysis and machine intelligence, 2005, 27(8): 1226-1238.
[7] 朱星宇, 陈秀宏. 联合不相关回归和非负谱分析的无监督特征选择[J]. 智能系统学报, 2022, 17(2): 303-313
ZHU X Y, CHEN X H. Joint uncorrelated regression and non-negative spectral analysis for unsupervised feature selection[J]. CAAI Transactions on Intelligent Systems, 2022, 17(2): 303-313
[8] 白圣子, 降爱莲. 基于特征正则稀疏关联的无监督特征选择方法[J]. 计算机工程与设计, 2022, 43(04): 969-976
BAI S Z, JIANG A L. Unsupervised feature selection method based on feature regularized sparse association[J]. Computer engineering and eesign, 2022, 43(04): 969-976
[9] PENG C, GAO X, WANG N, et al. Face recognition from multiple stylistic sketches: scenarios, datasets, and evaluation[J]. Pattern recognition, 2018, 84: 262-272.
[10] FU Y, YAN S, HUANG T S. Correlation metric for generalized feature extraction[J]. IEEE transactions on pattern analysis and machine intelligence, 2008, 30(12): 2229-2235.
[11] JAIN A, ZONGKER D. Feature selection: Evaluation, application, and small sample performance[J]. IEEE transactions on pattern analysis and machine intelligence, 1997, 19(2): 153-158.
[12] ZHAO J D, LU K, HE X F. Locality sensitive semi-supervised feature selection[J]. Neurocomputing, 2008, 71(10-12): 1842-1849.
[13] HOU C P, NIE F P, LI X L, et al. Joint embedding learning and sparse regression: a framework for unsupervised feature selection[J]. IEEE transactions on cybernetics, 2014, 44(6): 793-804.
[14] HE X F, CAI D, NIYOGI P. Laplacian Score for feature selection[C]//Advances in Neural Information Processing Systems. Vancouver: NIPS, 2005: 507?514.
[15] TABAKHI S, MORADI P, AKHLAGHIAN F. An unsupervised feature selection algorithm based on ant colony optimization[J]. Engineering applications of artificial intelligence, 2014, 32: 112-123.
[16] CAI D, ZHANG C Y, HE X F. Unsupervised feature selection for multi-cluster data[C]// Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Washington, DC: ACM, 2010, 333?342.
[17] NIE F P, WANG X Q, HUANG H. Clustering and projected clustering with adaptive neighbors[C]// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Arlington: ACM, 2014: 977?986.
[18] MOHSENZADEH Y, SHEIKHZADEH H, Reza A M, et al. The relevance sample-feature machine: A sparse bayesian learning approach to joint feature-sample selection[J]. IEEE transactions on cybernetics, 2014, 43(6): 2241-2254.
[19] ZHU P F, ZUO W M, ZHANG L, et al. Unsupervised feature selection by regularized self-representation[J]. Pattern recognition, 2015, 48(2): 438-446.
[20] NIE F P, ZHU W, LI X L. Unsupervised feature selection with structured graph optimization[C]// Proceedings of Thirtieth AAAI Conference on Artificial Intelligence. Phoenix: AAAI Press, 2016: 1302?1308.
[21] LI X L, ZHANG H, ZHANG R, et al. Generalized uncorrelated regression with adaptive graph for unsupervised feature selection[J]. IEEE transactions on neural network and learning systems, 2019, 30(5): 1587-1595.
[22] MIN Y, YE M, TIAN L, et al. Unsupervised feature selection via multi-step Markov probability relationship[J]. Neurocomputing, 2021, 453: 241-253.
[23] SZUMMER M, JAAKKOLA T. Partially labeled classification with markov random walks[C]//Proceedings of the 14th International Conference on Neural Information Processing System: Natural and Synthetic. Cambridge: MIT Press, 2001, 945?952.
[24] CAI J F, CANDES E J, SHEN Z W. A singular value thresholding algorithm for matrix completion[J]. SIAM journal on optimization, 2010, 20(4): 1956-1982.
[25] HE Z S, XIE S L, ZDUNEK R, et al. Symmetric nonnegative matrix factorization: algorithms and applications to probabilistic clustering[J]. IEEE transactions on neural networks, 2011, 22(12): 2117-2131.
[26] 徐慧敏, 陈秀宏. 图正则化稀疏判别非负矩阵分解[J]. 智能系统学报, 2019, 14(6): 1217-1224
XU H M, CHEN X H. Graph-regularized, sparse discriminant, non-negative matrix factorization[J]. CAAI Transactions on Intelligent Systems, 2019, 14(6): 1217-1224
[27] KUANG D, DING C, PARK H. Symmetric nonnegative matrix factorization for graph clustering[C]// Proceedings of the SIAM International Conference on Data Mining. Anaheim: SDM, 2012, 1(2): 106?117.
[28] BOYD S, PARIKH N, ERIC C, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and trends in machine learning, 2010, 3(1): 1-122.
[29] LONG B, ZHANG Z, YU P S. Co-clustering by block value decomposition[C]// Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Chicago: ACM, 2005: 635?640.
[30] LANGE K, HUNTER D R, YANG L. Optimization transfer using surrogate objective functions[J]. Journal of computational and graphical statistics, 2000, 9(1): 1-20.
[31] SAMARIA F S, HARTER A C. Parameterisation of a stochastic model for human face identification[C]// Proceedings of 1994 IEEE Workshop on Applications of Computer Vision. Sarasota: IEEE, 1994, 138?142.
[32] RATE C, RETRIEVAL C. Columbia object image library(COIL-20)[EB/OL]. (2011?12?12)[2020?01?01]. http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php.
[33] HULL J. A database for handwritten text recognition research[J]. IEEE transactions on pattern analysis and machine intelligence, 1994, 16(5): 550-554.
[34] FANTY M A, COLE R A. Spoken Letter Recognition[C]// Advances in Neural Information Processing Systems. Stroudsburg: Association for Computational Linguistics, 1990, 220?226.
[35] BHATTACHARJEE A, RICHARDS W G, STAUNTON J, et al. Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses[J]. Proceedings of the National Academy of Sciences of the United States of America, 2001, 98(24): 13790-13795.
[36] HASLINGER C, SCHWEIFER N, STILGENBAUER S, et al. Microarray gene expression profiling of B-cell chronic lymphocytic leukemia subgroups defined by genomic aberrations and VH mutation status[J]. Journal of clinical oncology, 2004, 22(19): 3937-49.
[37] TANG C, BIAN M, LIU X W, et al. Unsupervised feature selection via latent representation learning and manifold regularization[J]. Neural network, 2019, 117: 163-17.
- 相似文献/References:
-
[1]孙正兴,张尧烨,李? 彬.基于线性规划分类器的相关反馈技术[J].智能系统学报,2007,2(3):34.
SUN Zheng-xing,ZHANG Yao-ye,LI Bin.Applying relevance feedback with a linear programming classifier[J].CAAI Transactions on Intelligent Systems,2007,2():34.
[2]张志飞,苗夺谦.基于粗糙集的文本分类特征选择算法[J].智能系统学报,2009,4(5):453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
ZHANG Zhi-fei,MIAO Duo-qian.Feature selection for text categorization based on rough set[J].CAAI Transactions on Intelligent Systems,2009,4():453.[doi:10.3969/j.issn.1673-4785.2009.05.011]
[3]顾成杰,张顺颐,杜安源.结合粗糙集和禁忌搜索的网络流量特征选择[J].智能系统学报,2011,6(3):254.
GU Chengjie,ZHANG Shunyi,DU Anyuan.Feature selection of network traffic using a rough set and tabu search[J].CAAI Transactions on Intelligent Systems,2011,6():254.
[4]孙倩茹,王文敏,刘宏.视频序列的人体运动描述方法综述[J].智能系统学报,2013,8(3):189.
SUN Qianru,WANG Wenmin,LIU Hong.Study of human action representation in video sequences[J].CAAI Transactions on Intelligent Systems,2013,8():189.
[5]曹晋,张莉,李凡长.一种基于支持向量数据描述的特征选择算法[J].智能系统学报,2015,10(2):215.[doi:10.3969/j.issn.1673-4785.201405063]
CAO Jin,ZHANG Li,LI Fanzhang.A noval support vector data description-based feature selection method[J].CAAI Transactions on Intelligent Systems,2015,10():215.[doi:10.3969/j.issn.1673-4785.201405063]
[6]张佳骕,蒋亦樟,王士同.基于特征选择聚类方法的稀疏TSK模糊系统[J].智能系统学报,2015,10(4):583.[doi:10.3969/j.issn.1673-4785.201412001]
ZHANG Jiasu,JIANG Yizhang,WANG Shitong.Sparse TSK fuzzy system based on feature selection clustering method[J].CAAI Transactions on Intelligent Systems,2015,10():583.[doi:10.3969/j.issn.1673-4785.201412001]
[7]陈玉明,吴克寿,李向军.基因表达数据在邻域关系中的特征选择[J].智能系统学报,2014,9(2):210.[doi:10.3969/j.issn.1673-4785.201307014]
CHEN Yuming,WU Keshou,LI Xiangjun.Gene expression data feature selection with neighborhood relation[J].CAAI Transactions on Intelligent Systems,2014,9():210.[doi:10.3969/j.issn.1673-4785.201307014]
[8]郭雨萌,李国正.一种多标记数据的过滤式特征选择框架[J].智能系统学报,2014,9(3):292.[doi:10.3969/j.issn.1673-4785.201403064]
GUO Yumeng,LI Guozheng.A filtering framework for the multi-label feature selection[J].CAAI Transactions on Intelligent Systems,2014,9():292.[doi:10.3969/j.issn.1673-4785.201403064]
[9]滕旭阳,董红斌,孙静.面向特征选择问题的协同演化方法[J].智能系统学报,2017,12(1):24.[doi:10.11992/tis.201611029]
TENG Xuyang,DONG Hongbin,SUN Jing.Co-evolutionary algorithm for feature selection[J].CAAI Transactions on Intelligent Systems,2017,12():24.[doi:10.11992/tis.201611029]
[10]路子祥,屠黎阳,祖辰,等.基于脑连接网络的阿尔茨海默病临床变量值预测[J].智能系统学报,2017,12(3):355.[doi:10.11992/tis.201607020]
LU Zixiang,TU Liyang,ZU Chen,et al.Prediction of clinical variables in Alzheimer’s disease using brain connective networks[J].CAAI Transactions on Intelligent Systems,2017,12():355.[doi:10.11992/tis.201607020]
备注/Memo
收稿日期:2022-8-11。
基金项目:江苏省研究生科研与实践创新计划项目(KYCX22_2433).
作者简介:过伶俐，硕士研究生，主要研究方向为数字图像处理、模式识别;陈秀宏，教授，博士后，主要研究方向为数字图像处理、模式识别、优化理论与方法。发表学术论文 120 余篇
通讯作者:陈秀宏.E-mail:xiuhongc@jiangnan.edu.cn
更新日期/Last Update:
1900-01-01