[1]王一宾,裴根生,程玉胜.弹性网络核极限学习机的多标记学习算法[J].智能系统学报,2019,14(04):831-842.[doi:10.11992/tis.201806005]
 WANG Yibin,PEI Gensheng,CHENG Yusheng.Multi-label learning algorithm of an elastic net kernel extreme learning machine[J].CAAI Transactions on Intelligent Systems,2019,14(04):831-842.[doi:10.11992/tis.201806005]
点击复制

弹性网络核极限学习机的多标记学习算法(/HTML)
分享到:

《智能系统学报》[ISSN:1673-4785/CN:23-1538/TP]

卷:
第14卷
期数:
2019年04期
页码:
831-842
栏目:
出版日期:
2019-07-02

文章信息/Info

Title:
Multi-label learning algorithm of an elastic net kernel extreme learning machine
作者:
王一宾12 裴根生1 程玉胜12
1. 安庆师范大学 计算机与信息学院, 安徽 安庆 246011;
2. 安徽省高校智能感知与计算重点实验室, 安徽 安庆 246011
Author(s):
WANG Yibin12 PEI Gensheng1 CHENG Yusheng12
1. School of Computer and Information, Anqing Normal University, Anqing 246011, China;
2. The University Key Laboratory of Intelligent Perception and Computing of Anhui Province, Anqing 246011, China
关键词:
多标记学习核极限学习机正则化弹性网络径向基函数坐标下降法
Keywords:
multi-label learningkernel extreme learning machineregularizationelastic netradial basis functioncoordinate descent
分类号:
TP391
DOI:
10.11992/tis.201806005
摘要:
将正则化极限学习机或者核极限学习机理论应用到多标记分类中,一定程度上提高了算法的稳定性。但目前这些算法关于损失函数添加的正则项都基于L2正则,导致模型缺乏稀疏性表达。同时,弹性网络正则化既保证模型鲁棒性且兼具模型稀疏化学习,但结合弹性网络的极限学习机如何解决多标记问题鲜有研究。基于此,本文提出一种对核极限学习机添加弹性网络正则化的多标记学习算法。首先,对多标记数据特征空间使用径向基核函数映射;随后,对核极限学习机损失函数施加弹性网络正则项;最后,采用坐标下降法迭代求解输出权值以得到最终预测标记。通过对比试验和统计分析表明,提出的算法具有更好的性能表现。
Abstract:
Regularized extreme learning machine or kernel extreme learning machine theory was applied to multi-label classification, which improves the stability of the algorithm to a certain extent. However, the regularization terms added by these algorithms for loss functions are all based on L2 regularization, which leads to the lack of sparse expression of the model. Simultaneously, elastic net regularization guarantees both model robustness and model sparse learning. Nevertheless, there is insufficient research on how to solve multi-label learning problems by combining elastic net kernel extreme learning machines. Based on this hypothesis, this paper proposes a multi-label learning algorithm that adds elastic network regularization to kernel extreme learning machines. It first uses radial basis function mapping for feature spacing of multi-label; subsequently, it applies the elastic net regularization to the loss function of kernel extreme learning machine. Finally, it uses the coordinate descent method to iteratively solve the output weights to get the final prediction labels. Through comparative experiments and statistical analyses, the proposed method demonstrates better performance.

参考文献/References:

[1] ZHANG Minling, ZHOU Zhihua. A review on multi-label learning algorithms[J]. IEEE transactions on knowledge and data engineering, 2014, 26(8):1819-1837.
[2] SCHAPIRE R E, SINGER Y. BoosTexter:a boosting-based system for text categorization[J]. Machine learning, 2000, 39(2/3):135-168.
[3] AGRAWAL S, AGRAWAL J, KAUR S, et al. A comparative study of fuzzy PSO and fuzzy SVD-based RBF neural network for multi-label classification[J]. Neural computing and applications, 2018, 29(1):245-256.
[4] BOUTELL M R, LUO Jiebo, SHEN Xipeng, et al. Learning multi-label scene classification[J]. Pattern recognition, 2004, 37(9):1757-1771.
[5] GUAN Renchu, WANG Xu, YANG M Q, et al. Multi-label deep learning for gene function annotation in cancer pathways[J]. Scientific reports, 2018, 8(1):267.
[6] TOMAR D, AGARWAL S. Multi-label classifier for emotion recognition from music[C]//Proceedings of the 3rd International Conference on Advanced Computing, Networking and Informatics. New Delhi, India:Springer, 2016:111-123.
[7] READ J, PFAHRINGER B, HOLMES G. Multi-label classification using ensembles of pruned sets[C]//Proceedings of the Eighth IEEE International Conference on Data Mining. Pisa:IEEE, 2008:995-1000.
[8] READ J. A pruned problem transformation method for multi-label classification[C]//Proceedings of 2008 New Zealand Computer Science Research Student Conference. Christchurch, New Zealand:NZCSRS, 2008:143-150.
[9] TSOUMAKAS G, VLAHAVAS I. Random k-labelsets:an ensemble method for multilabel classification[C]//Proceedings of the 18th European Conference on Machine Learning. Warsaw, Poland:Springer, 2007:406-417.
[10] ZHANG Minling, ZHOU Zhihua. ML-KNN:a lazy learning approach to multi-label learning[J]. Pattern recognition, 2007, 40(7):2038-2048.
[11] ZHANG Minling, PEÑA J M, ROBLES V. Feature selection for multi-label naive Bayes classification[J]. Information sciences, 2009, 179(19):3218-3229.
[12] ELISSEEFF A, WESTON J. A kernel method for multi-labelled classification[C]//Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic. Vancouver, Canada:MIT Press, 2001:681-687.
[13] ZHANG Minling. ML-RBF:RBF neural networks for multi-label learning[J]. Neural processing letters, 2009, 29(2):61-74.
[14] HUANG Guangbin, ZHU Qinyu, SIEW C K. Extreme learning machine:theory and applications[J]. Neurocomputing, 2006, 70(1/2/3):489-501.
[15] 邓万宇, 郑庆华, 陈琳, 等. 神经网络极速学习方法研究[J]. 计算机学报, 2010, 33(2):279-287 DENG Wanyu, ZHENG Qinghua, CHEN Lin, et al. Research on extreme learning of neural networks[J]. Chinese journal of computers, 2010, 33(2):279-287
[16] MICHE Y, VAN HEESWIJK M, BAS P, et al. TROP-ELM:a double-regularized ELM using LARS and Tikhonov regularization[J]. Neurocomputing, 2011, 74(16):2413-2421.
[17] HUANG Guangbin, ZHOU Hongming, DING Xiaojian, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE transactions on systems, man, and cybernetics, part B (cybernetics), 2012, 42(2):513-529.
[18] ER M J, VENKATESAN R, WANG Ning. A high speed multi-label classifier based on extreme learning machines[C]//Proceedings of ELM-2015 Volume 2:Theory, Algorithms and Applications. Cham:Springer International Publishing, 2016.
[19] SUN Xia, XU Jingting, JIANG Changmeng, et al. Extreme learning machine for multi-label classification[J]. Entropy, 2016, 18(6):225.
[20] ZHANG Nan, DING Shifei, ZHANG Jian. Multi layer ELM-RBF for multi-label learning[J]. Applied soft computing, 2016, 43:535-545.
[21] LUO Fangfang, GUO Wenzhong, YU Yuanlong, et al. A multi-label classification algorithm based on kernel extreme learning machine[J]. Neurocomputing, 2017, 260:313-320.
[22] HAN Yahong, WU Fei, ZHUANG Yueting, et al. Multi-label transfer learning with sparse representation[J]. IEEE transactions on circuits and systems for video technology, 2010, 20(8):1110-1121.
[23] ZOU Hui, HASTIE T. Regularization and variable selection via the elastic net[J]. Journal of the royal statistical society:series B (statistical methodology), 2005, 67(2):301-320.
[24] FRIEDMAN J, HASTIE T, TIBSHIRANI R. Regularization paths for generalized linear models via coordinate descent[J]. Journal of statistical software, 2010, 33(1):1-22.
[25] ZHOU Zhihua, ZHANG Minling. Multi-label learning[M]//SAMMUT C, WEBB G I. Encyclopedia of Machine Learning and Data Mining. Boston, MA:Springer, 2017:875-881
[26] WRIGHT S J. Coordinate descent algorithms[J]. Mathematical programming, 2015, 151(1):3-34.
[27] DONOHO D L. De-noising by soft-thresholding[J]. IEEE transactions on information theory, 1995, 41(3):613-627.
[28] FRIEDMAN J, HASTIE T, HÖFLING H, et al. Pathwise coordinate optimization[J]. The annals of applied statistics, 2007, 1(2):302-332.
[29] DEMSAR J. Statistical comparisons of classifiers over multiple data sets[J]. Journal of machine learning research, 2006, 7:1-30.

相似文献/References:

[1]刘杨磊,梁吉业,高嘉伟,等.基于Tri-training的半监督多标记学习算法[J].智能系统学报,2013,8(05):439.[doi:10.3969/j.issn.1673-4785.201305033]
 LIU Yanglei,LIANG Jiye,GAO Jiawei,et al.Semi-supervised multi-label learning algorithm based on Tri-training[J].CAAI Transactions on Intelligent Systems,2013,8(04):439.[doi:10.3969/j.issn.1673-4785.201305033]
[2]杨文元.多标记学习自编码网络无监督维数约简[J].智能系统学报,2018,13(05):808.[doi:10.11992/tis.201804051]
 YANG Wenyuan.Unsupervised dimensionality reduction of multi-label learning via autoencoder networks[J].CAAI Transactions on Intelligent Systems,2018,13(04):808.[doi:10.11992/tis.201804051]
[3]余鹰,王乐为,吴新念,等.基于改进卷积神经网络的多标记分类算法[J].智能系统学报,2019,14(03):566.[doi:10.11992/tis.201804056]
 YU Ying,WANG Lewei,WU Xinnian,et al.A multi-label classification algorithm based on an improved convolutional neural network[J].CAAI Transactions on Intelligent Systems,2019,14(04):566.[doi:10.11992/tis.201804056]

备注/Memo

备注/Memo:
收稿日期:2018-06-02。
基金项目:安徽省高校重点科研项目(KJ2017A352);安徽省高校重点实验室基金项目(ACAIM160102).
作者简介:王一宾,男,1970年生,教授,主要研究方向为多标记学习、机器学习、软件安全。主持安徽省教育厅重点项目多项,发表学术论文20余篇;裴根生,男,1992年生,硕士研究生,主要研究方向为机器学习、数据挖掘、统计;程玉胜,男,1969年生,教授,博士,主要研究方向为数据挖掘、机器学习。主持省自然科学基金项目1项、省教育厅项目多项,发表学术论文50余篇。
通讯作者:程玉胜.E-mail:chengyshaq@163.com
更新日期/Last Update: 2019-08-25