[1]王一宾,裴根生,程玉胜.弹性网络核极限学习机的多标记学习算法[J].智能系统学报,2019,14(4):831-842.[doi:10.11992/tis.201806005]
WANG Yibin,PEI Gensheng,CHENG Yusheng.Multi-label learning algorithm of an elastic net kernel extreme learning machine[J].CAAI Transactions on Intelligent Systems,2019,14(4):831-842.[doi:10.11992/tis.201806005]
点击复制
WANG Yibin,PEI Gensheng,CHENG Yusheng.Multi-label learning algorithm of an elastic net kernel extreme learning machine[J].CAAI Transactions on Intelligent Systems,2019,14(4):831-842.[doi:10.11992/tis.201806005]
弹性网络核极限学习机的多标记学习算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
14
期数:
2019年第4期
页码:
831-842
栏目:
学术论文—机器学习
出版日期:
2019-07-02
- Title:
- Multi-label learning algorithm of an elastic net kernel extreme learning machine
- Keywords:
- multi-label learning; kernel extreme learning machine; regularization; elastic net; radial basis function; coordinate descent
- 分类号:
- TP391
- 摘要:
- 将正则化极限学习机或者核极限学习机理论应用到多标记分类中，一定程度上提高了算法的稳定性。但目前这些算法关于损失函数添加的正则项都基于L2正则，导致模型缺乏稀疏性表达。同时，弹性网络正则化既保证模型鲁棒性且兼具模型稀疏化学习，但结合弹性网络的极限学习机如何解决多标记问题鲜有研究。基于此，本文提出一种对核极限学习机添加弹性网络正则化的多标记学习算法。首先，对多标记数据特征空间使用径向基核函数映射；随后，对核极限学习机损失函数施加弹性网络正则项；最后，采用坐标下降法迭代求解输出权值以得到最终预测标记。通过对比试验和统计分析表明，提出的算法具有更好的性能表现。
- Abstract:
- Regularized extreme learning machine or kernel extreme learning machine theory was applied to multi-label classification, which improves the stability of the algorithm to a certain extent. However, the regularization terms added by these algorithms for loss functions are all based on L2 regularization, which leads to the lack of sparse expression of the model. Simultaneously, elastic net regularization guarantees both model robustness and model sparse learning. Nevertheless, there is insufficient research on how to solve multi-label learning problems by combining elastic net kernel extreme learning machines. Based on this hypothesis, this paper proposes a multi-label learning algorithm that adds elastic network regularization to kernel extreme learning machines. It first uses radial basis function mapping for feature spacing of multi-label; subsequently, it applies the elastic net regularization to the loss function of kernel extreme learning machine. Finally, it uses the coordinate descent method to iteratively solve the output weights to get the final prediction labels. Through comparative experiments and statistical analyses, the proposed method demonstrates better performance.
- 参考文献/References:
-
[1] ZHANG Minling, ZHOU Zhihua. A review on multi-label learning algorithms[J]. IEEE transactions on knowledge and data engineering, 2014, 26(8):1819-1837.
[2] SCHAPIRE R E, SINGER Y. BoosTexter:a boosting-based system for text categorization[J]. Machine learning, 2000, 39(2/3):135-168.
[3] AGRAWAL S, AGRAWAL J, KAUR S, et al. A comparative study of fuzzy PSO and fuzzy SVD-based RBF neural network for multi-label classification[J]. Neural computing and applications, 2018, 29(1):245-256.
[4] BOUTELL M R, LUO Jiebo, SHEN Xipeng, et al. Learning multi-label scene classification[J]. Pattern recognition, 2004, 37(9):1757-1771.
[5] GUAN Renchu, WANG Xu, YANG M Q, et al. Multi-label deep learning for gene function annotation in cancer pathways[J]. Scientific reports, 2018, 8(1):267.
[6] TOMAR D, AGARWAL S. Multi-label classifier for emotion recognition from music[C]//Proceedings of the 3rd International Conference on Advanced Computing, Networking and Informatics. New Delhi, India:Springer, 2016:111-123.
[7] READ J, PFAHRINGER B, HOLMES G. Multi-label classification using ensembles of pruned sets[C]//Proceedings of the Eighth IEEE International Conference on Data Mining. Pisa:IEEE, 2008:995-1000.
[8] READ J. A pruned problem transformation method for multi-label classification[C]//Proceedings of 2008 New Zealand Computer Science Research Student Conference. Christchurch, New Zealand:NZCSRS, 2008:143-150.
[9] TSOUMAKAS G, VLAHAVAS I. Random k-labelsets:an ensemble method for multilabel classification[C]//Proceedings of the 18th European Conference on Machine Learning. Warsaw, Poland:Springer, 2007:406-417.
[10] ZHANG Minling, ZHOU Zhihua. ML-KNN:a lazy learning approach to multi-label learning[J]. Pattern recognition, 2007, 40(7):2038-2048.
[11] ZHANG Minling, PE?A J M, ROBLES V. Feature selection for multi-label naive Bayes classification[J]. Information sciences, 2009, 179(19):3218-3229.
[12] ELISSEEFF A, WESTON J. A kernel method for multi-labelled classification[C]//Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic. Vancouver, Canada:MIT Press, 2001:681-687.
[13] ZHANG Minling. ML-RBF:RBF neural networks for multi-label learning[J]. Neural processing letters, 2009, 29(2):61-74.
[14] HUANG Guangbin, ZHU Qinyu, SIEW C K. Extreme learning machine:theory and applications[J]. Neurocomputing, 2006, 70(1/2/3):489-501.
[15] 邓万宇, 郑庆华, 陈琳, 等. 神经网络极速学习方法研究[J]. 计算机学报, 2010, 33(2):279-287 DENG Wanyu, ZHENG Qinghua, CHEN Lin, et al. Research on extreme learning of neural networks[J]. Chinese journal of computers, 2010, 33(2):279-287
[16] MICHE Y, VAN HEESWIJK M, BAS P, et al. TROP-ELM:a double-regularized ELM using LARS and Tikhonov regularization[J]. Neurocomputing, 2011, 74(16):2413-2421.
[17] HUANG Guangbin, ZHOU Hongming, DING Xiaojian, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE transactions on systems, man, and cybernetics, part B (cybernetics), 2012, 42(2):513-529.
[18] ER M J, VENKATESAN R, WANG Ning. A high speed multi-label classifier based on extreme learning machines[C]//Proceedings of ELM-2015 Volume 2:Theory, Algorithms and Applications. Cham:Springer International Publishing, 2016.
[19] SUN Xia, XU Jingting, JIANG Changmeng, et al. Extreme learning machine for multi-label classification[J]. Entropy, 2016, 18(6):225.
[20] ZHANG Nan, DING Shifei, ZHANG Jian. Multi layer ELM-RBF for multi-label learning[J]. Applied soft computing, 2016, 43:535-545.
[21] LUO Fangfang, GUO Wenzhong, YU Yuanlong, et al. A multi-label classification algorithm based on kernel extreme learning machine[J]. Neurocomputing, 2017, 260:313-320.
[22] HAN Yahong, WU Fei, ZHUANG Yueting, et al. Multi-label transfer learning with sparse representation[J]. IEEE transactions on circuits and systems for video technology, 2010, 20(8):1110-1121.
[23] ZOU Hui, HASTIE T. Regularization and variable selection via the elastic net[J]. Journal of the royal statistical society:series B (statistical methodology), 2005, 67(2):301-320.
[24] FRIEDMAN J, HASTIE T, TIBSHIRANI R. Regularization paths for generalized linear models via coordinate descent[J]. Journal of statistical software, 2010, 33(1):1-22.
[25] ZHOU Zhihua, ZHANG Minling. Multi-label learning[M]//SAMMUT C, WEBB G I. Encyclopedia of Machine Learning and Data Mining. Boston, MA:Springer, 2017:875-881
[26] WRIGHT S J. Coordinate descent algorithms[J]. Mathematical programming, 2015, 151(1):3-34.
[27] DONOHO D L. De-noising by soft-thresholding[J]. IEEE transactions on information theory, 1995, 41(3):613-627.
[28] FRIEDMAN J, HASTIE T, H?FLING H, et al. Pathwise coordinate optimization[J]. The annals of applied statistics, 2007, 1(2):302-332.
[29] DEMSAR J. Statistical comparisons of classifiers over multiple data sets[J]. Journal of machine learning research, 2006, 7:1-30.
- 相似文献/References:
-
[1]刘杨磊,梁吉业,高嘉伟,等.基于Tri-training的半监督多标记学习算法[J].智能系统学报,2013,8(5):439.[doi:10.3969/j.issn.1673-4785.201305033]
LIU Yanglei,LIANG Jiye,GAO Jiawei,et al.Semi-supervised multi-label learning algorithm based on Tri-training[J].CAAI Transactions on Intelligent Systems,2013,8():439.[doi:10.3969/j.issn.1673-4785.201305033]
[2]杨文元.多标记学习自编码网络无监督维数约简[J].智能系统学报,2018,13(5):808.[doi:10.11992/tis.201804051]
YANG Wenyuan.Unsupervised dimensionality reduction of multi-label learning via autoencoder networks[J].CAAI Transactions on Intelligent Systems,2018,13():808.[doi:10.11992/tis.201804051]
[3]余鹰,王乐为,吴新念,等.基于改进卷积神经网络的多标记分类算法[J].智能系统学报,2019,14(3):566.[doi:10.11992/tis.201804056]
YU Ying,WANG Lewei,WU Xinnian,et al.A multi-label classification algorithm based on an improved convolutional neural network[J].CAAI Transactions on Intelligent Systems,2019,14():566.[doi:10.11992/tis.201804056]
[4]黄琴,钱文彬,王映龙,等.代价敏感数据的多标记特征选择算法[J].智能系统学报,2019,14(5):929.[doi:10.11992/tis.201807027]
HUANG Qin,QIAN Wenbin,WANG Yinglong,et al.Multi-label feature selection algorithm for cost-sensitive data[J].CAAI Transactions on Intelligent Systems,2019,14():929.[doi:10.11992/tis.201807027]
[5]查思明,鲍庆森,骆健,等.自适应标记关联与实例关联诱导的缺失多视图弱标记学习[J].智能系统学报,2022,17(4):670.[doi:10.11992/tis.202106017]
ZHA Siming,BAO Qingsen,LUO Jian,et al.Adaptive label correlation and instance correlation guided incomplete multiview weak label learning[J].CAAI Transactions on Intelligent Systems,2022,17():670.[doi:10.11992/tis.202106017]
[6]秦天,滕齐发,贾修一.结合局部标记序关系的弱监督标记分布学习[J].智能系统学报,2023,18(1):47.[doi:10.11992/tis.202204018]
QIN Tian,TENG Qifa,JIA Xiuyi.Weakly supervised label distribution learning by maintaining local label ranking[J].CAAI Transactions on Intelligent Systems,2023,18():47.[doi:10.11992/tis.202204018]
[7]胡军,王海峰.基于加权信息粒化的多标记数据特征选择算法[J].智能系统学报,2023,18(3):619.[doi:10.11992/tis.202111058]
HU Jun,WANG Haifeng.Feature selection algorithm of multi-labeled data based on weighted information granulation[J].CAAI Transactions on Intelligent Systems,2023,18():619.[doi:10.11992/tis.202111058]
备注/Memo
收稿日期:2018-06-02。
基金项目:安徽省高校重点科研项目（KJ2017A352）；安徽省高校重点实验室基金项目（ACAIM160102）.
作者简介:王一宾,男,1970年生,教授,主要研究方向为多标记学习、机器学习、软件安全。主持安徽省教育厅重点项目多项,发表学术论文20余篇;裴根生,男,1992年生,硕士研究生,主要研究方向为机器学习、数据挖掘、统计;程玉胜,男,1969年生,教授,博士,主要研究方向为数据挖掘、机器学习。主持省自然科学基金项目1项、省教育厅项目多项,发表学术论文50余篇。
通讯作者:程玉胜.E-mail:chengyshaq@163.com
更新日期/Last Update:
2019-08-25