[1]WANG Yibin,PEI Gensheng,CHENG Yusheng.Multi-label learning algorithm of an elastic net kernel extreme learning machine[J].CAAI Transactions on Intelligent Systems,2019,14(4):831-842.[doi:10.11992/tis.201806005]
Copy

Multi-label learning algorithm of an elastic net kernel extreme learning machine

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 14 Number of periods: 2019 4 Page number: 831-842 Column: 学术论文—机器学习 Public date: 2019-07-02
Title:
Multi-label learning algorithm of an elastic net kernel extreme learning machine
Author(s):
WANG Yibin1; 2;  PEI Gensheng1;  CHENG Yusheng1; 2
1. School of Computer and Information, Anqing Normal University, Anqing 246011, China;
2. The University Key Laboratory of Intelligent Perception and Computing of Anhui Province, Anqing 246011, China
Keywords:
multi-label learning; kernel extreme learning machine; regularization; elastic net; radial basis function; coordinate descent
CLC:
TP391
DOI:
10.11992/tis.201806005
Abstract:
Regularized extreme learning machine or kernel extreme learning machine theory was applied to multi-label classification, which improves the stability of the algorithm to a certain extent. However, the regularization terms added by these algorithms for loss functions are all based on L2 regularization, which leads to the lack of sparse expression of the model. Simultaneously, elastic net regularization guarantees both model robustness and model sparse learning. Nevertheless, there is insufficient research on how to solve multi-label learning problems by combining elastic net kernel extreme learning machines. Based on this hypothesis, this paper proposes a multi-label learning algorithm that adds elastic network regularization to kernel extreme learning machines. It first uses radial basis function mapping for feature spacing of multi-label; subsequently, it applies the elastic net regularization to the loss function of kernel extreme learning machine. Finally, it uses the coordinate descent method to iteratively solve the output weights to get the final prediction labels. Through comparative experiments and statistical analyses, the proposed method demonstrates better performance.
References:
[1] ZHANG Minling, ZHOU Zhihua. A review on multi-label learning algorithms[J]. IEEE transactions on knowledge and data engineering, 2014, 26(8):1819-1837.
[2] SCHAPIRE R E, SINGER Y. BoosTexter:a boosting-based system for text categorization[J]. Machine learning, 2000, 39(2/3):135-168.
[3] AGRAWAL S, AGRAWAL J, KAUR S, et al. A comparative study of fuzzy PSO and fuzzy SVD-based RBF neural network for multi-label classification[J]. Neural computing and applications, 2018, 29(1):245-256.
[4] BOUTELL M R, LUO Jiebo, SHEN Xipeng, et al. Learning multi-label scene classification[J]. Pattern recognition, 2004, 37(9):1757-1771.
[5] GUAN Renchu, WANG Xu, YANG M Q, et al. Multi-label deep learning for gene function annotation in cancer pathways[J]. Scientific reports, 2018, 8(1):267.
[6] TOMAR D, AGARWAL S. Multi-label classifier for emotion recognition from music[C]//Proceedings of the 3rd International Conference on Advanced Computing, Networking and Informatics. New Delhi, India:Springer, 2016:111-123.
[7] READ J, PFAHRINGER B, HOLMES G. Multi-label classification using ensembles of pruned sets[C]//Proceedings of the Eighth IEEE International Conference on Data Mining. Pisa:IEEE, 2008:995-1000.
[8] READ J. A pruned problem transformation method for multi-label classification[C]//Proceedings of 2008 New Zealand Computer Science Research Student Conference. Christchurch, New Zealand:NZCSRS, 2008:143-150.
[9] TSOUMAKAS G, VLAHAVAS I. Random k-labelsets:an ensemble method for multilabel classification[C]//Proceedings of the 18th European Conference on Machine Learning. Warsaw, Poland:Springer, 2007:406-417.
[10] ZHANG Minling, ZHOU Zhihua. ML-KNN:a lazy learning approach to multi-label learning[J]. Pattern recognition, 2007, 40(7):2038-2048.
[11] ZHANG Minling, PE?A J M, ROBLES V. Feature selection for multi-label naive Bayes classification[J]. Information sciences, 2009, 179(19):3218-3229.
[12] ELISSEEFF A, WESTON J. A kernel method for multi-labelled classification[C]//Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic. Vancouver, Canada:MIT Press, 2001:681-687.
[13] ZHANG Minling. ML-RBF:RBF neural networks for multi-label learning[J]. Neural processing letters, 2009, 29(2):61-74.
[14] HUANG Guangbin, ZHU Qinyu, SIEW C K. Extreme learning machine:theory and applications[J]. Neurocomputing, 2006, 70(1/2/3):489-501.
[15] 邓万宇, 郑庆华, 陈琳, 等. 神经网络极速学习方法研究[J]. 计算机学报, 2010, 33(2):279-287 DENG Wanyu, ZHENG Qinghua, CHEN Lin, et al. Research on extreme learning of neural networks[J]. Chinese journal of computers, 2010, 33(2):279-287
[16] MICHE Y, VAN HEESWIJK M, BAS P, et al. TROP-ELM:a double-regularized ELM using LARS and Tikhonov regularization[J]. Neurocomputing, 2011, 74(16):2413-2421.
[17] HUANG Guangbin, ZHOU Hongming, DING Xiaojian, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE transactions on systems, man, and cybernetics, part B (cybernetics), 2012, 42(2):513-529.
[18] ER M J, VENKATESAN R, WANG Ning. A high speed multi-label classifier based on extreme learning machines[C]//Proceedings of ELM-2015 Volume 2:Theory, Algorithms and Applications. Cham:Springer International Publishing, 2016.
[19] SUN Xia, XU Jingting, JIANG Changmeng, et al. Extreme learning machine for multi-label classification[J]. Entropy, 2016, 18(6):225.
[20] ZHANG Nan, DING Shifei, ZHANG Jian. Multi layer ELM-RBF for multi-label learning[J]. Applied soft computing, 2016, 43:535-545.
[21] LUO Fangfang, GUO Wenzhong, YU Yuanlong, et al. A multi-label classification algorithm based on kernel extreme learning machine[J]. Neurocomputing, 2017, 260:313-320.
[22] HAN Yahong, WU Fei, ZHUANG Yueting, et al. Multi-label transfer learning with sparse representation[J]. IEEE transactions on circuits and systems for video technology, 2010, 20(8):1110-1121.
[23] ZOU Hui, HASTIE T. Regularization and variable selection via the elastic net[J]. Journal of the royal statistical society:series B (statistical methodology), 2005, 67(2):301-320.
[24] FRIEDMAN J, HASTIE T, TIBSHIRANI R. Regularization paths for generalized linear models via coordinate descent[J]. Journal of statistical software, 2010, 33(1):1-22.
[25] ZHOU Zhihua, ZHANG Minling. Multi-label learning[M]//SAMMUT C, WEBB G I. Encyclopedia of Machine Learning and Data Mining. Boston, MA:Springer, 2017:875-881
[26] WRIGHT S J. Coordinate descent algorithms[J]. Mathematical programming, 2015, 151(1):3-34.
[27] DONOHO D L. De-noising by soft-thresholding[J]. IEEE transactions on information theory, 1995, 41(3):613-627.
[28] FRIEDMAN J, HASTIE T, H?FLING H, et al. Pathwise coordinate optimization[J]. The annals of applied statistics, 2007, 1(2):302-332.
[29] DEMSAR J. Statistical comparisons of classifiers over multiple data sets[J]. Journal of machine learning research, 2006, 7:1-30.
Similar References:

Memo

-

Last Update: 2019-08-25

Copyright © CAAI Transactions on Intelligent Systems