[1]吕佳,邱小龙.基于K-means聚类和特征空间增强的噪声标签深度学习算法[J].智能系统学报,2024,19(2):267-277.[doi:10.11992/tis.202303014]
LYU Jia,QIU Xiaolong.A noisy label deep learning algorithm based on K-means clustering and feature space augmentation[J].CAAI Transactions on Intelligent Systems,2024,19(2):267-277.[doi:10.11992/tis.202303014]
点击复制
LYU Jia,QIU Xiaolong.A noisy label deep learning algorithm based on K-means clustering and feature space augmentation[J].CAAI Transactions on Intelligent Systems,2024,19(2):267-277.[doi:10.11992/tis.202303014]
基于K-means聚类和特征空间增强的噪声标签深度学习算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第2期
页码:
267-277
栏目:
学术论文—机器学习
出版日期:
2024-03-05
- Title:
- A noisy label deep learning algorithm based on K-means clustering and feature space augmentation
- Keywords:
- noisy label learning; deep learning; semisupervised learning; machine learning; neural network; K-means clustering; feature space augmentation; mix-up algorithm
- 分类号:
- TP181
- 文献标志码:
- 2023-12-25
- 摘要:
- 深度学习中神经网络的性能依赖于高质量的样本，然而噪声标签会降低网络的分类准确率。为降低噪声标签对网络性能的影响，噪声标签学习算法被提出。该算法首先将训练样本集划分成干净样本集和噪声样本集，然后使用半监督学习算法对噪声样本集赋予伪标签。然而，错误的伪标签以及训练样本数量不足的问题仍然限制着噪声标签学习算法性能的提升。为解决上述问题，提出基于K-means聚类和特征空间增强的噪声标签深度学习算法。首先，该算法利用K-means聚类算法对干净样本集进行标签聚类，并根据噪声样本集与聚类中心的距离大小筛选出难以分类的噪声样本，以提高训练样本的质量；其次，使用mixup算法扩充干净样本集和噪声样本集，以增加训练样本的数量；最后，采用特征空间增强算法抑制mixup算法新生成的噪声样本，从而提高网络的分类准确率。并在CIFAR10、CIFAR100、MNIST和ANIMAL-10共4个数据集上试验验证了该算法的有效性。
- Abstract:
- The performance of neural networks in deep learning relies on high-quality samples. However, the presence of noisy labels reduces the classification accuracy of the network. To reduce the impact of noisy labels, we propose a learning algorithm that categorizes training samples into clean and noisy subsets, assigning pseudo-labels to the noisy samples using a semisupervised learning algorithm. Despite these measures, the performance of the noisy label learning algorithm can be hindered by inaccurate pseudo-labels and a lack of sufficient training samples. To address the aforementioned problems, we propose a noisy label deep learning algorithm that leverages K-means clustering and feature space augmentation. First, the algorithm applies the K-means clustering algorithm to cluster the clean samples based on their labels. It then selects noisy samples that are difficult to classify according to the distance between the noisy samples and the cluster center. This process enhances the quality of the training samples. Second, the mix-up algorithm is used to expand both the clean and noisy samples, thereby increasing the number of training samples. Finally, a feature space augmentation algorithm is used to suppress the noise samples generated by the mix-up algorithm, leading to improved network classification accuracy. The effectiveness of the proposed algorithm has been validated on four data sets: CIFAR10, CIFAR100, MNIST, and ANIMAL-10.
- 参考文献/References:
-
[1] ZHANG Jing, WU Xindong, SHENG V S. Learning from crowdsourced labeled data: a survey[J]. Artificial intelligence review, 2016, 46(4): 543–576.
[2] 伏博毅, 彭云聪, 蓝鑫, 等. 基于深度学习的标签噪声学习算法综述[J]. 计算机应用, 2023, 43(3): 674–684
FU Boyi, PENG Yuncong, LAN Xin, et al. Survey of label noise learning algorithms based on deep learning[J]. Journal of computer applications, 2023, 43(3): 674–684
[3] HAN Bo, YAO Quanming, YU Xingrui, et al. Co-teaching: robust training of deep neural networks with extremely noisy labels[C]//Advances in Neural Information Processing Systems. Montreal: NIPS, 2018: 1602-1613.
[4] YU Xingrui, HAN Bo, YAO Jiangchao, et al. How does disagreement help generalization against label corruption [EB/OL]. (2019-01-14)[2022-12-25]. https://arxiv.org/abs/1901.04215.pdf.
[5] WEI Hongxin, FENG Lei, CHEN Xiangyu, et al. Combating noisy labels by agreement: a joint training method with co-regularization[C]//2020 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE, 2020: 13723-13732.
[6] LI Junnan, SOCHER R, HOI S C H. DivideMix: learning with noisy labels as semi-supervised learning[EB/OL]. (2020-02-18)[2022-12-25]. https://arxiv.org/abs/2002.07394.pdf.
[7] CORDEIRO F R, SACHDEVA R, BELAGIANNIS V, et al. LongReMix: robust learning with high confidence samples in a noisy label environment[J]. Pattern recognition, 2023, 133(1): 565–581.
[8] KARIM N, KHALID U, ESMAEILI A, et al. CNLL: a semi-supervised approach for continual noisy label learning[C]//2022 IEEE Conference on Computer Vision and Pattern Recognition Workshops. New Orleans: IEEE, 2022: 3877-3887.
[9] ZAHEER M Z, LEE Jinha, ASTRID M, et al. Cleaning label noise with clusters for minimally supervised anomaly detection[EB/OL]. (2021-04-30)[2022-12-25]. https://arxiv.org/abs/2104.14770.pdf.
[10] CHENG Hao, ZHU Zhaowei, LI Xingyu, et al. Learning with instance-dependent label noise: a sample sieve appro-ach[EB/OL]. (2020-10-05)[2022-12-25]. https://arxiv.org/abs/2010.02347.pdf.
[11] ZHANG Hongyi, CISSE M, DAUPHIN Y N, et al. Mixup: beyond empirical risk minimization[EB/OL]. (2018-04-27)[2022-12-25]. https://arxiv.org/abs/1710.09412.pdf.
[12] LI Boyi, WU F, LIM S N, et al. On feature normalization and data augmentation[C]//2021 IEEE Conference on Computer Vision and Pattern Recognition. Kuala Lumpur: IEEE, 2021: 12378-12387.
[13] BERTHELOT D, RAFFEL C, ROY A, et al. Understanding and improving interpolation in autoencoders via an adversarial regularizer[EB/OL]. (2018-07-23)[2022-12-.25]. https://arxiv.org/abs/1807.07543.pdf.
[14] SOHN K, BERTHELOT D, CARLINI N, et al. Fixmatch: simplifying semi-supervised learning with consistency and confidence[C]//Advances in Neural Information Processing Systems. Addis Ababa: NIPS, 2020: 596-608.
[15] LIU Defu, ZHAO Jiayi, WU Jinzhao, et al. Multi-category classification with label noise by robust binary loss[J]. Neurocomputing, 2022, 482(16): 14–26.
[16] WU Songhua, XIA Xiaobo, LIU Tongliang, et al. Class2Simi: a noise reduction perspective on learning with noisy labels[C]//International Conference on Machine Learning. London: ACM, 2021: 11285-11295.
[17] SHARMA N, JAIN V, MISHRA A. An analysis of convolutional neural networks for image classification[J]. Procedia computer science, 2018, 132(9): 377–384.
[18] DENG L. The mnist database of handwritten digit images for machine learning research[J]. IEEE signal processing magazine, 2012, 29(6): 141–142.
[19] TAN C, XIA J, WU L, et al. Co-learning: Learning from noisy labels with self-supervision[C]//Proceedings of the 29th ACM International Conference on Multimedia. Chengdu: ACM, 2021: 1405-1413.
[20] PATRINI G, ROZZA A, MENON A K, et al. Making deep neural networks robust to label noise: a loss correction approach[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2233-2241.
[21] ZHANG Z, SABUNCU M. Generalized cross entropy loss fortraining deep neural networks with noisy labels[C]//Avances in Neural Information Processing Systems. Montreal: NIPS, 2018: 11400-11411.
[22] ARAZO E, ORTEGO D, ALBERT P, et al. Unsupervised label noise modeling and loss correction[C]//International Conference on Machine Learning. Los Angeles: ACM, 2019: 312-321.
[23] WANG Zhuowei, JIANG Jing, HAN Bo, et al. SemiNLL: a framework of noisy-label learning by semi-supervised learning[EB/OL]. (2020-11-02)[2022-12-27]. https://arxiv.org/abs/2012.00925.pdf.
[24] ZHOU Xiong, LIU Xianming, WANG Chenyang, et al. Learning with noisy labels via sparse regularization[C]//2021 IEEE International Conference on Computer Vision. Montreal: IEEE, 2022: 72-81.
[25] FENG Lei, SHU Senlin, LIN Zhuoyi, et al. Can cross entropy loss be robust to label noise[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. Yokohama: ACM, 2020: 2206-2212.
[26] YI Li, LIU Sheng, SHE Qi, et al. On learning contrastive representations for learning with noisy labels[C]//2022 IEEE Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 16661-16670.
[27] MENON A K, RAWAT A S, REDDI S J, et al. Can gradient clipping mitigate label noise[C]//International Conference on Learning Representations. Addis Ababa: ACM, 2020: 6204-6231.
[28] SONG H, KIM M, LEE J G. Selfie: Refurbishing unclean samples for robust deep learning[C]//International Conference on Machine Learning, Los Angeles: ACM, 2019: 5907-5915.
[29] ZHANG Yikai, ZHENG Songzhu, WU Pengxiang, et al. Learning with feature dependent label noise: a progressiv-e approach[EB/OL]. (2021-05-27)[2022-12-25]. https://ar-xiv.org/abs/2103.07756.pdf.
[30] CHEN Yingyi, SHEN Xi, HU S X, et al. Boosting co-teaching with compression regularization for label noise[C]//2021 IEEE Conference on Computer Vision and Pattern Recognition Kuala Lumpur, IEEE, 2021: 2682-2686.
[31] RIPPEL O, GELBART M, ADAMS R. Learning ordered representations with nested dropout[C]//International Conference on Machine Learning. Beijing: ACM, 2014: 1746-1754.
- 相似文献/References:
-
[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[4]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[5]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[6]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[7]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
[8]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[9]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(6):894.[doi:10.11992/tis.201706037]
YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12():894.[doi:10.11992/tis.201706037]
[10]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
备注/Memo
收稿日期:2023-03-07。
基金项目:国家自然科学基金重大项目(11991024)；重庆市教委“成渝地区双城经济圈建设”科技创新项目(KJCX2020024)；重庆市高校创新研究群体资助项目(CXQT20015)；重庆市教委科研项目重点项目(KJZD-K202200511).
作者简介:吕佳，教授，博士，主要研究方向为机器学习、数据挖掘，中国计算机学会会员。主持或参与的国家级、省部级科研项目20项，发表学术论文70余篇。E-mail：lvjia@cqnu.edu.cn;邱小龙，硕士研究生，主要研究方向为机器学习、数据挖掘、噪声标签学习算法。E-mail：2021210516067@stu.cqnu.edu.cn
通讯作者:吕佳. E-mail: lvjia@cqnu.edu.cn
更新日期/Last Update:
1900-01-01