[1]吴晗,王士同.不完整数据分类与缺失信息重要性识别特权LSSVM[J].智能系统学报,2023,18(4):743-753.[doi:10.11992/tis.202202026]
WU Han,WANG Shitong.Privileged LSSVM for classification and simultaneous importance identification of missing information on incomplete data[J].CAAI Transactions on Intelligent Systems,2023,18(4):743-753.[doi:10.11992/tis.202202026]
点击复制
WU Han,WANG Shitong.Privileged LSSVM for classification and simultaneous importance identification of missing information on incomplete data[J].CAAI Transactions on Intelligent Systems,2023,18(4):743-753.[doi:10.11992/tis.202202026]
不完整数据分类与缺失信息重要性识别特权LSSVM
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第4期
页码:
743-753
栏目:
学术论文—机器感知与模式识别
出版日期:
2023-07-15
- Title:
- Privileged LSSVM for classification and simultaneous importance identification of missing information on incomplete data
- Keywords:
- least squares support vector machines; learning using privileged information; additional kernel; missing data; k-nearest neighbor; sample space; privileged space; data quality
- 分类号:
- TP181
- 摘要:
- 针对直接移除缺失数据的样本可能会导致因样本数量规模的减少从而降低了分类性能的问题，本文基于同时处理缺失数据与构建模式分类模型的策略，提出使用特权信息学习(learning using privileged information, LUPI)的特权最小二乘支持向量机 (privileged least squares support vector machine, P-LSSVM)，从而达到既能改进其分类性能，又能在保证无偏的情况下确定缺失特征的重要性。本文的基本思想是将完整数据的训练作为特权信息，以此来引导面向整个不完全数据的最小二乘支持向量机(least squares support vector machine, LSSVM)的学习，通过可加性核表达每个特征（含缺失特征）的重要性，推导完整数据的训练的特权信息，并以此构建P-LSSVM，运用所提出的留一交叉验证方法完成无偏的缺失特征重要性识别。实验结果表明，本文提出的方法不但在平均测试精度上优于对比算法，还能同时确定缺失特征的重要性。
- Abstract:
- While handling missing data classification tasks, the commonly-used removal strategy of missing data may perhaps degrade the classifier’s performance, due to very insufficient perfect data. Based on the strategy of processing missing data and constructing classification model simultaneously, we develop a novel privileged LSSVM (P-LSSVM), which learns using privilaged information. It can not only improve its classification performance, but also determines the importance of missing features without bias. The basic idea is to take the trained classifier of the available perfect data as the privileged information to guide the learning of LSSVM for the whole incomplete data, express the importance of each feature including missing features through the additivity kernel, then deduce the privilaged information of complete data after training, based on which P-LSSVM is constructed. Finally, the unbiased missing feature importance recognition is completed by the proposed leaving-one cross-validation method. Experimental results show that the proposed method can achieve better testing accuracies, with the importance identification of missing features.
- 参考文献/References:
-
[1] GARCíA-LAENCINA P J, SANCHO-GóMEZ J L, FIGUEIRAS-VIDAL A R. Pattern classification with missing data: a review[J]. Neural computing and applications, 2010, 19(2): 263–282.
[2] WANG Guanjin, DENG Zhaohong, CHOI K S. Tackling missing data in community health studies using additive LS-SVM classifier[J]. IEEE journal of biomedical and health informatics, 2016, 22(2): 579–587.
[3] BATISTA G E A P A, MONARD M C. An analysis of four missing data treatment methods for supervised learning[J]. Applied artificial intelligence, 2003, 17(5/6): 519–533.
[4] MOON T K. The expectation-maximization algorithm[J]. IEEE signal processing magazine, 1996, 13(6): 47?60.
[5] DEMPSTER A P, LAIRD N M, RUBIN D B. Maximum likelihood from incomplete data via the EM algorithm[J]. Journal of the royal statistical society series B: statistical methodology, 1977, 39(1): 1?22.
[6] HORTON N J, KLEINMAN K P. Much ado about nothing[J]. The American statistician, 2007, 61(1): 79?90.
[7] IBRAHIM J G, CHEN M H, LIPSITZ S R. Monte Carlo EM for missing covariates in parametric regression models[J]. Biometrics, 1999, 55(2): 591?596.
[8] 仝伯兵, 王士同, 梅向东. 稀疏条件下的两层分类算法[J]. 智能系统学报, 2015, 10(1): 27–36
TONG Bobing, WANG Shitong, MEI Xiangdong. Sparsity-inspired two-level classification algorithm[J]. CAAI transactions on intelligent systems, 2015, 10(1): 27–36
[9] GARCíA-LAENCINA P J, SERRANO J, FIGUEIRAS-VIDAL A R, et al. Multi-task neural networks for dealing with missing inputs[M]. Berlin: Springer Berlin Heidelberg, 2007: 282?291.
[10] TROYANSKAYA O, CANTOR M, SHERLOCK G, et al. Missing value estimation methods for DNA microarrays[J]. Bioinformatics (Oxford, England), 2001, 17(6): 520?525.
[11] JUSZCZAK P, DUIN R P W. Combining one-class classifiers to classify missing data[M]. Berlin: Springer Berlin Heidelberg, 2004: 92?101.
[12] JIANG Kai, CHEN Haixia, YUAN Senmiao. Classification for incomplete data using classifier ensembles[C]//2005 International Conference on Neural Networks and Brain. Piscataway: IEEE, 2006: 559?563.
[13] KRAUSE S, POLIKAR R. An ensemble of classifiers approach for the missing feature problem[C]//Proceedings of the International Joint Conference on Neural Networks. Piscataway: IEEE, 2003: 553?558.
[14] CHOUDHURY S J, PALN R. Imputation of missing data with neural networks for classification[J]. Knowledge-based systems, 2019, 182: 104838
[15] 卞则康, 王士同. 基于混合距离学习的鲁棒的模糊C均值聚类算法[J]. 智能系统学报, 2017, 12(4): 450–458
BIAN Zekang, WANG Shitong. Robust FCM clustering algorithm based on hybrid-distance learning[J]. CAAI transactions on intelligent systems, 2017, 12(4): 450–458
[16] VAPNIK V, VASHIST A. A new learning paradigm: learning using privileged information[J]. Neural networks, 2009, 22(5/6): 544?557.
[17] SUYKENS J A K, VANDEWALLE J. Chaos control using least-squares support vector machines[J]. International journal of circuit theory and applications, 1999, 27(6): 605?615.
[18] PELCKMANS K, GOETHALS I, BRABANTER J D, et al. Componentwise least squares support vector machines[M]. Berlin: Springer Berlin Heidelberg, 2005: 77?98.
[19] 李欢, 王士同. 支持向量机的多观测样本二分类算法[J]. 智能系统学报, 2014, 9(4): 392–400
LI Huan, WANG Shitong. Binary-class classification algorithm with multiple-access acquired objects based on the SVM[J]. CAAI transactions on intelligent systems, 2014, 9(4): 392–400
[20] MAJI S, BERG A C, MALIK J. Efficient classification for additive kernel SVMs[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 35(1): 66?77.
[21] DEMIR B, BRUZZONE L. Fast and accurate image classification with histogram based features and additive kernel SVM[C]//2015 IEEE International Geoscience and Remote Sensing Symposium. Piscataway: IEEE, 2015: 2350?2353.
[22] 王旭凤. 基于可加性核的快速支持向量机分类算法的研究[D]. 西安：西安电子科技大学, 2017.
WANG Xufeng. Fast support vector machine classification algorithm with additive kernel [D]. Xi’an: Xi’an University of Electronic Science and technology, 2017.
[23] XUE Li. Robust learning with imperfect privileged information[J]. Artificial intelligence, 2020, 282: 103246.
[24] XU Xinxing, LI Wen, XU Dong. Distance metric learning using privileged information for face verification and person re-identification[J]. IEEE transactions on neural networks and learning systems, 2015, 26(12): 3150?3162.
[25] PAL A, KHEMCHANDANI R R N. Learning TWSVM using privilege information[C]//2018 IEEE Symposium Series on Computational Intelligence Piscataway: IEEE, 2019: 1548?1554.
- 相似文献/References:
-
[1]向峥嵘,陈庆伟.基于小波和LSSVM的软测量建模方法[J].智能系统学报,2010,5(1):63.
XIANG Zheng-rong,CHEN Qing-wei.An approach to soft sensor modeling based onwavelets and a least square support vector machine[J].CAAI Transactions on Intelligent Systems,2010,5():63.
[2]钱晓山,阳春华.基于GEP的最小二乘支持向量机模型参数选择[J].智能系统学报,2012,7(3):225.
QIAN Xiaoshan,YANG Chunhua.A parameter selection method of a least squares support vector machine based on gene expression programming[J].CAAI Transactions on Intelligent Systems,2012,7():225.
备注/Memo
收稿日期:2022-02-27。
基金项目:国家自然科学基金项目（61972181）.
作者简介:吴晗，硕士研究生，主要研究方向为人工智能、模式识别;王士同，教授，博士生导师，全国优秀教师、国务院政府特贴获得者、省部级有突出贡献中青年专家，主要研究方向为人工智能与模式识别。主持及参与国家自然科学基金项目6项，获教育部、中船总公司、湖南省等省部级科技进步奖10项。发表学术论文50余篇。
通讯作者:王士同.E-mail:wxwangst@aliyun.com
更新日期/Last Update:
1900-01-01