[1]ZHOU Hongbiao,QIAO Junfei.Feature selection method based on high dimensional k-nearest neighbors mutual information[J].CAAI Transactions on Intelligent Systems,2017,12(5):595-600.[doi:10.11992/tis.201609020]
Copy

Feature selection method based on high dimensional k-nearest neighbors mutual information

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 12 Number of periods: 2017 5 Page number: 595-600 Column: 学术论文—人工智能基础 Public date: 2017-10-25
Title:
Feature selection method based on high dimensional k-nearest neighbors mutual information
Author(s):
ZHOU Hongbiao1; 2; 3;  QIAO Junfei1; 2
1. Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China;
2. Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, China;
3. Faculty of Automation, Huaiyin Institute of Technology, Huai’an 223003, China
Keywords:
feature selection; mutual information; k-nearest neighbor; high-dimensional mutual information; multilayer perceptron
CLC:
TP183
DOI:
10.11992/tis.201609020
Abstract:
Feature selection plays an important role in the modeling and forecast of multivariate series. In this paper, we propose a feature selection method based on data-driven high-dimensional k-nearest neighbor mutual information. First, this method extends the k-nearest neighbor method to estimate the amount of mutual information among high-dimensional feature variables. Next, optimal sorting of all these features is achieved by adopting a forward accumulation strategy in which irrelevant features are eliminated according to a preset number. Then, redundant features are located and removed using a backward cross strategy. Lastly, this method obtains optimal subsets that feature a strong correlation. Using Friedman data, housing data, and actual effluent total-phosphorus forecast data from wastewater treatment plant as examples, we performed a simulation experiment by adopting a neural network forecast model with multilayer perception. The simulation results demonstrate the feasibility of the proposed method.
References:
[1] OSELEDETS I V, TYRTYSHNIKOV E E. Breaking the curse of dimensionality, or how to use SVD in many dimensions[J]. SIAM journal on scientific computing, 2009, 31(5):3744-3759.
[2] GHAMISI P, BENEDIKTSSON J A. Feature selection based on hybridization of genetic algorithm and particle swarm optimization[J]. IEEE geoscience and remote sensing letters, 2015, 12(2):309-313.
[3] RAUBER T W, ASSIS BOLDT de F, VAREJ?O F M. Heterogeneous feature models and feature selection applied to bearing fault diagnosis[J]. IEEE transactions on industrial electronics, 2015, 62(1):637-646.
[4] WOLD S, SJ?STR?M M, ERIKSSON L. PLS-regression:a basic tool of chemometrics[J]. Chemometrics and intelligent laboratory systems, 2001, 58(2):109-130.
[5] SONG Q, SHEPPERD M. Predicting software project effort:a grey relational analysis based method[J]. Expert systems with applications, 2011, 38(6):7302-7316.
[6] FENG J, JIAO L, LIU F, et al. Mutual-information-based semi-supervised hyperspectral band selection with high discrimination, high information, and low redundancy[J]. IEEE transactions on geoscience and remote sensing, 2015, 53(5):2956-2969.
[7] BENNASAR M, HICKS Y, SETCHI R. Feature selection using joint mutual information maximisation[J]. Expert systems with applications, 2015, 42(22):8520-8532.
[8] SHANNON C E. A mathematical theory of communication[J]. ACM sigmobile mobile computing and communications review, 2001, 5(1):3-55.
[9] BATTITI R. Using mutual information for selecting features in supervised neural net learning[J]. IEEE transactions on neural networks, 1994, 5(4):537-550.
[10] PENG H, LONG F, DING C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE transactions on pattern analysis and machine intelligence, 2005, 27(8):1226-1238.
[11] FLEURET F. Fast binary feature selection with conditional mutual information[J]. Journal of machine learning research, 2004, 5:1531-1555.
[12] YANG H H, MOODY J E. Data visualization and feature selection:new algorithms for nongaussian data[C]//Advances in Neural Information Processing Systems. Cambridge,Britain, 1999:687-693.
[13] BROWN G. A new perspective for information theoretic feature selection[C]//Proceedings of the 12th International Conference on Artificial Intelligence and Statistics. Florida, USA, 2009:49-56.
[14] 韩敏, 刘晓欣. 基于互信息的分步式输入变量选择多元序列预测研究[J]. 自动化学报, 2012, 38(6):999-1005.HAN Min, LIU Xiaoxin. Stepwise input variable selection based on mutual information for multivariate forecasting[J]. ACTA automatica sinica, 2012, 38(6):999-1005.
[15] BEIRLANT J, DUDEWICZ E J, GY?RFI L, et al. Nonparametric entropy estimation:an overview[J]. International journal of mathematical and statistical sciences, 1997, 6(1):17-39.
[16] MOON Y I, RAJAGOPALAN B, LALL U. Estimation of mutual information using kernel density estimators[J]. Physical review E, 1995, 52(3):2318-2321.
[17] KRASKOV A, ST?GBAUER H, GRASSBERGER P. Estimatingmutual information[J]. Physical review E, 2004, 69(6):1-16.
[18] ST?GBAUER H, KRASKOV A, ASTAKHOV S A, et al. Least-pendent-component analysis based on mutual information[J]. Physical review E, 2004, 70(6):1-17.
[19] 霍军周, 王亚杰, 欧阳湘宇, 等. 基于BP神经网络的TBM主轴承载荷谱预测[J]. 哈尔滨工程大学学报, 2015, 36(7):965-969.HUO Junzhou, WANG Yajie, OUYANG Xiangyu, et al.Load spectrum prediction of the main drive bearing of a tunnel boring machine based on BP neural networks[J]. Journal of Harbin Engineering University, 2015, 36(7):965-969.
[20] 乔俊飞, 周红标. 基于自组织模糊神经网络的出水总磷预测[J]. 控制理论与应用, 2017, 34(2):224-232. QIAO Junfei, ZHOU Hongbiao. Prediction of effluent total phosphorus based on self-organizing fuzzy neural network[J]. Control theory and applications, 2017, 34(2):224-232.
Similar References:

Memo

-

Last Update: 2017-10-25

Copyright © CAAI Transactions on Intelligent Systems