[1]GUO Lingli,CHEN Xiuhong.Robust unsupervised feature selection via multistep Markov probability and latent representation[J].CAAI Transactions on Intelligent Systems,2023,18(5):1017-1029.[doi:10.11992/tis.202208013]
Copy

Robust unsupervised feature selection via multistep Markov probability and latent representation

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 5 Page number: 1017-1029 Column: 学术论文—机器学习 Public date: 2023-09-05
Title:
Robust unsupervised feature selection via multistep Markov probability and latent representation
Author(s):
GUO Lingli;  CHEN Xiuhong
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China
Keywords:
feature selection; latent representation learning; multistep Markov transition probability; unsupervised; non-negative matrix factorization; sparse regression; L2; 1-norm; dimensionality reduction
CLC:
TP181
DOI:
10.11992/tis.202208013
Abstract:
Unsupervised feature selection is a significant dimensionality reduction technique in machine learning and data mining. However, current unsupervised feature selection methods primarily focus on learning the manifold structure of the data from the adjacency matrix, ignoring the association between non-adjacent data pairs. Second, these methods often assume that the data instances are independent and identically distributed, but in reality, the data samples originate from heterogeneous sources, and this assumption is often untenable. Additionally, the measure of feature importance in the original data space is affected by noise in the data and features. To address the aforementioned problems, this study proposes a robust unsupervised feature selection method based on multistep Markov probability and latent representation (MMLRL). The key idea is to learn the manifold structure between the data points through the maximum multistep Markov transition probability. Subsequently, a symmetric non-negative matrix factorization model was used to learn the latent representation of the data. Finally, the feature selection is performed in the latent representation space. At the same time, the proposed algorithm is evaluated on six different types of datasets to validate its effectiveness.
References:
[1] BRUNETTI A, BUONGIORNO D, TROTTA G F, et al. Computer vision and deep learning techniques for pedestrian detection and tracking: a survey[J]. Neurocomputing, 2018, 300: 17-33.
[2] JORDAN M I, MITCHELL T M. Machine learning: trends, perspectives, and prospects[J]. Science, 2015, 349(6245): 255-260.
[3] LI H, HE X, TAO D, et al. Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning[J]. Pattern recognition, 2018, 79: 130-146.
[4] QURESHI R, UZAIR M, KHURSHID K, et al. Hyperspectral document image processing: applications, challenges and future prospects[J]. Pattern recognition, 2019, 90: 12-22.
[5] VADIM K. Overview of different approaches to solving problems of data mining[J]. Procedia computers, 2018, 123: 234-239.
[6] PENG H C, LONG F H, DING C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE transactions on pattern analysis and machine intelligence, 2005, 27(8): 1226-1238.
[7] 朱星宇, 陈秀宏. 联合不相关回归和非负谱分析的无监督特征选择[J]. 智能系统学报, 2022, 17(2): 303-313
ZHU X Y, CHEN X H. Joint uncorrelated regression and non-negative spectral analysis for unsupervised feature selection[J]. CAAI Transactions on Intelligent Systems, 2022, 17(2): 303-313
[8] 白圣子, 降爱莲. 基于特征正则稀疏关联的无监督特征选择方法[J]. 计算机工程与设计, 2022, 43(04): 969-976
BAI S Z, JIANG A L. Unsupervised feature selection method based on feature regularized sparse association[J]. Computer engineering and eesign, 2022, 43(04): 969-976
[9] PENG C, GAO X, WANG N, et al. Face recognition from multiple stylistic sketches: scenarios, datasets, and evaluation[J]. Pattern recognition, 2018, 84: 262-272.
[10] FU Y, YAN S, HUANG T S. Correlation metric for generalized feature extraction[J]. IEEE transactions on pattern analysis and machine intelligence, 2008, 30(12): 2229-2235.
[11] JAIN A, ZONGKER D. Feature selection: Evaluation, application, and small sample performance[J]. IEEE transactions on pattern analysis and machine intelligence, 1997, 19(2): 153-158.
[12] ZHAO J D, LU K, HE X F. Locality sensitive semi-supervised feature selection[J]. Neurocomputing, 2008, 71(10-12): 1842-1849.
[13] HOU C P, NIE F P, LI X L, et al. Joint embedding learning and sparse regression: a framework for unsupervised feature selection[J]. IEEE transactions on cybernetics, 2014, 44(6): 793-804.
[14] HE X F, CAI D, NIYOGI P. Laplacian Score for feature selection[C]//Advances in Neural Information Processing Systems. Vancouver: NIPS, 2005: 507?514.
[15] TABAKHI S, MORADI P, AKHLAGHIAN F. An unsupervised feature selection algorithm based on ant colony optimization[J]. Engineering applications of artificial intelligence, 2014, 32: 112-123.
[16] CAI D, ZHANG C Y, HE X F. Unsupervised feature selection for multi-cluster data[C]// Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Washington, DC: ACM, 2010, 333?342.
[17] NIE F P, WANG X Q, HUANG H. Clustering and projected clustering with adaptive neighbors[C]// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Arlington: ACM, 2014: 977?986.
[18] MOHSENZADEH Y, SHEIKHZADEH H, Reza A M, et al. The relevance sample-feature machine: A sparse bayesian learning approach to joint feature-sample selection[J]. IEEE transactions on cybernetics, 2014, 43(6): 2241-2254.
[19] ZHU P F, ZUO W M, ZHANG L, et al. Unsupervised feature selection by regularized self-representation[J]. Pattern recognition, 2015, 48(2): 438-446.
[20] NIE F P, ZHU W, LI X L. Unsupervised feature selection with structured graph optimization[C]// Proceedings of Thirtieth AAAI Conference on Artificial Intelligence. Phoenix: AAAI Press, 2016: 1302?1308.
[21] LI X L, ZHANG H, ZHANG R, et al. Generalized uncorrelated regression with adaptive graph for unsupervised feature selection[J]. IEEE transactions on neural network and learning systems, 2019, 30(5): 1587-1595.
[22] MIN Y, YE M, TIAN L, et al. Unsupervised feature selection via multi-step Markov probability relationship[J]. Neurocomputing, 2021, 453: 241-253.
[23] SZUMMER M, JAAKKOLA T. Partially labeled classification with markov random walks[C]//Proceedings of the 14th International Conference on Neural Information Processing System: Natural and Synthetic. Cambridge: MIT Press, 2001, 945?952.
[24] CAI J F, CANDES E J, SHEN Z W. A singular value thresholding algorithm for matrix completion[J]. SIAM journal on optimization, 2010, 20(4): 1956-1982.
[25] HE Z S, XIE S L, ZDUNEK R, et al. Symmetric nonnegative matrix factorization: algorithms and applications to probabilistic clustering[J]. IEEE transactions on neural networks, 2011, 22(12): 2117-2131.
[26] 徐慧敏, 陈秀宏. 图正则化稀疏判别非负矩阵分解[J]. 智能系统学报, 2019, 14(6): 1217-1224
XU H M, CHEN X H. Graph-regularized, sparse discriminant, non-negative matrix factorization[J]. CAAI Transactions on Intelligent Systems, 2019, 14(6): 1217-1224
[27] KUANG D, DING C, PARK H. Symmetric nonnegative matrix factorization for graph clustering[C]// Proceedings of the SIAM International Conference on Data Mining. Anaheim: SDM, 2012, 1(2): 106?117.
[28] BOYD S, PARIKH N, ERIC C, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and trends in machine learning, 2010, 3(1): 1-122.
[29] LONG B, ZHANG Z, YU P S. Co-clustering by block value decomposition[C]// Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Chicago: ACM, 2005: 635?640.
[30] LANGE K, HUNTER D R, YANG L. Optimization transfer using surrogate objective functions[J]. Journal of computational and graphical statistics, 2000, 9(1): 1-20.
[31] SAMARIA F S, HARTER A C. Parameterisation of a stochastic model for human face identification[C]// Proceedings of 1994 IEEE Workshop on Applications of Computer Vision. Sarasota: IEEE, 1994, 138?142.
[32] RATE C, RETRIEVAL C. Columbia object image library(COIL-20)[EB/OL]. (2011?12?12)[2020?01?01]. http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php.
[33] HULL J. A database for handwritten text recognition research[J]. IEEE transactions on pattern analysis and machine intelligence, 1994, 16(5): 550-554.
[34] FANTY M A, COLE R A. Spoken Letter Recognition[C]// Advances in Neural Information Processing Systems. Stroudsburg: Association for Computational Linguistics, 1990, 220?226.
[35] BHATTACHARJEE A, RICHARDS W G, STAUNTON J, et al. Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses[J]. Proceedings of the National Academy of Sciences of the United States of America, 2001, 98(24): 13790-13795.
[36] HASLINGER C, SCHWEIFER N, STILGENBAUER S, et al. Microarray gene expression profiling of B-cell chronic lymphocytic leukemia subgroups defined by genomic aberrations and VH mutation status[J]. Journal of clinical oncology, 2004, 22(19): 3937-49.
[37] TANG C, BIAN M, LIU X W, et al. Unsupervised feature selection via latent representation learning and manifold regularization[J]. Neural network, 2019, 117: 163-17.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems