[1]吕佳,邱鸿波,肖锋.基于动态阈值和差异性检验的自训练算法[J].智能系统学报,2024,19(4):839-852.[doi:10.11992/tis.202306047]
LYU Jia,QIU Hongbo,XIAO Feng.Self-training algorithm based on dynamic threshold and difference test[J].CAAI Transactions on Intelligent Systems,2024,19(4):839-852.[doi:10.11992/tis.202306047]
点击复制
LYU Jia,QIU Hongbo,XIAO Feng.Self-training algorithm based on dynamic threshold and difference test[J].CAAI Transactions on Intelligent Systems,2024,19(4):839-852.[doi:10.11992/tis.202306047]
基于动态阈值和差异性检验的自训练算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第4期
页码:
839-852
栏目:
学术论文—机器学习
出版日期:
2024-07-05
- Title:
- Self-training algorithm based on dynamic threshold and difference test
- Keywords:
- self-training algorithm; mislabeled samples; high-confidence samples; dynamic threshold; difference test; local outlier factor; contrast membership; dense distance
- 分类号:
- TP181
- 摘要:
- 针对自训练算法在迭代训练分类器的过程中存在难以有效选取高置信度样本以及误标记样本错误累积的问题，本文提出了基于动态阈值和差异性检验的自训练算法。引入样本的局部离群因子，据此剔除有标签样本中的离群点以及分类标注无标签样本，依据标注分批次处理无标签样本，以使模型更易选取到高置信度的无标签样本；根据新增伪标签样本的数量和对比隶属度的变化，设计一种动态隶属度阈值函数，提升高置信度样本的质量；定义密集距离度量样本间的差异性，分别计算伪标签样本与同类和不同类样本之间的密集距离之和，从而找出不确定度高的伪标签样本，并将此类样本并入下轮训练的无标签样本集中，缓解误标记样本错误累积的问题。实验结果表明，该算法在12个UCI基准数据集上均取得理想效果。
- Abstract:
- In the process of iterative training of the classifier by a self-training algorithm, it is difficult to effectively select high-confidence samples and there exists mislabeled samples error accumulation. To address the above issues, this paper proposes a self-training algorithm based on dynamic threshold and difference test.
- 参考文献/References:
-
[1] LAI C I, CHUANG Y S, LEE H Y, et al. Semi-supervised spoken language understanding via self-supervised speech and language model pretraining[C]//ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2021: 7468–7472.
[2] LIU Quande, YU Lequan, LUO Luyang, et al. Semi-supervised medical image classification with relation-driven self-ensembling model[J]. IEEE transactions on medical imaging, 2020, 39(11): 3429–3440.
[3] LI Changchun, LI Ximing, OUYANG Jihong. Semi-supervised text classification with balanced deep representation distributions[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2021: 5044–5053.
[4] VAN ENGELEN J E, HOOS H H. A survey on semi-supervised learning[J]. Machine learning, 2020, 109(2): 373–440.
[5] YANG Xiangli, SONG Zixing, KING I, et al. A survey on deep semi-supervised learning[J]. IEEE transactions on knowledge and data engineering, 2023, 35(9): 8934–8954.
[6] DUARTE J M, BERTON L. A review of semi-supervised learning for text classification[J]. Artificial intelligence review, 2023, 56(9): 9401–9469.
[7] SOHN K, BERTHELOT D, LI Chunliang, et al. FixMatch: simplifying semi-supervised learning with consistency and confidence[EB/OL]. (2020-01-21)[2023-03-25]. https://arxiv.org/abs/2001.07685.pdf.
[8] KIM J, MIN Y, KIM D, et al. ConMatch: semi-supervised learning with confidence-guided consistency regularization[C]//European Conference on Computer Vision. Cham: Springer, 2022: 674–690.
[9] ZHANG Zhiwu, JING Xiaoyuan, WANG Tiejian. Label propagation based semi-supervised learning for software defect prediction[J]. Automated software engineering, 2017, 24(1): 47–69.
[10] ISCEN A, TOLIAS G, AVRITHIS Y, et al. Label propagation for deep semi-supervised learning[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 5070–5079.
[11] ZHAO Zhen, ZHOU Luping, WANG Lei, et al. LaSSL: label-guided self-training for semi-supervised learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(8): 9208–9216.
[12] XU Qiantong, BAEVSKI A, LIKHOMANENKO T, et al. Self-training and pre-training are complementary for speech recognition[C]//ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2021: 3030–3034.
[13] 程康明, 熊伟丽. 一种自训练框架下的三优选半监督回归算法[J]. 智能系统学报, 2020, 15(3): 568–577
CHENG Kangming, XIONG Weili. Three-optimal semi-supervised regression algorithm under self-training framework[J]. CAAI transactions on intelligent systems, 2020, 15(3): 568–577
[14] CHEN Baixu, JIANG Junguang, WANG Ximei, et al. Debiased Self-Training for Semi-Supervised Learning[C]//Proceedings of the Annual Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2022: 92–100.
[15] WANG Yidong, CHEN Hao, HENG Qiang, et al. FreeMatch: self-adaptive thresholding for semi-supervised learning[EB/OL]. (2022-05-15)[2023-05-12].https://arxiv.org/abs/2205.07246.
[16] PARK S, KIM G, OH Y, et al. Self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation[J]. Nature communications, 2022, 13: 3848.
[17] GAN Haitao, SANG Nong, HUANG Rui, et al. Using clustering analysis to improve semi-supervised classification[J]. Neurocomputing, 2013, 101: 290–298.
[18] WU Di, SHANG Mingsheng, LUO Xin, et al. Self-training semi-supervised classification based on density peaks of data[J]. Neurocomputing, 2018, 275: 180–191.
[19] ZOU Yang, YU Zhiding, LIU Xiaofeng, et al. Confidence regularized self-training[C]//2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 5982–5991.
[20] MUKHERJEE S, AWADALLAH A H. Uncertainty-aware self-training for text classification with few labels[EB/OL]. (2020-06-27)[2023-05-12].https://arxiv.org/abs/2006.15315.
[21] WEI Danni, YANG Youlong, QIU Haiquan. Improving self-training with density peaks of data and cut edge weight statistic[J]. Soft computing, 2020, 24(20): 15595–15610.
[22] 吕佳, 刘强, 李帅军. 结合密度峰值和改进自然邻居的自训练算法[J]. 南京大学学报(自然科学版), 2022, 58(5): 805–815
LYU Jia, LIU Qiang, LI Shuaijun. Self-training method based on density peaks and improved natural neighbor[J]. Journal of Nanjing university (natural science edition), 2022, 58(5): 805–815
[23] LI Shuaijun, LYU Jia. Self-training algorithm based on density peaks combining globally adaptive multi-local noise filter[J]. Intelligent data analysis, 2023, 27(2): 323–343.
[24] CHEN Hao, TAO Ran, FAN Yue, et al. SoftMatch: addressing the quantity-quality trade-off in semi-supervised learning[EB/OL]. (2023-01-26)[2023-05-12].https://arxiv.org/abs/2301.10921.
[25] OUYANG Boya, SONG Yu, LI Yuhai, et al. EBOD: an ensemble-based outlier detection algorithm for noisy datasets[J]. Knowledge-based systems, 2021, 231: 107400.
[26] BREUNIG M M, KRIEGEL H P, NG R T, et al. LOF: identifying density-based local outliers[C]//Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. New York: ACM, 2000: 93–104.
[27] GUO Lanzhe, ZHANG Zhenyu, JIANG Yuan, et al. Safe deep semi-supervised learning for unseen-class unlabeled data[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ACM, 2020: 3897–3906.
[28] XU Yi, SHANG Lei, YE Jinxing, et al. Dash: semi-supervised learning with dynamic thresholding[EB/OL]. (2021-09-01)[2023-05-12].https://arxiv.org/abs/2109.00650.
[29] GUO Lanzhe, LI Yufeng. Class-imbalanced semi-supervised learning with adaptive thresholding[C]//Proceedings of International Conference on Machine Learning. New York: ACM, 2022: 8082–8094.
[30] YAO Huifeng, HU Xiaowei, LI Xiaomeng. Enhancing pseudo label quality for semi-supervised domain-generalized medical image segmentation[EB/OL]. (2022-01-21)[2023-05-12].https://arxiv.org/abs/2201.08657.
[31] 刘学文, 王继奎, 杨正国, 等. 密度峰值隶属度优化的半监督Self-Training算法[J]. 计算机科学与探索, 2022, 16(9): 2078–2088
LIU Xuewen, WANG Jikui, YANG Zhengguo, et al. Semi-supervised self-training algorithm for density peak membership optimization[J]. Journal of frontiers of computer science and technology, 2022, 16(9): 2078–2088
[32] VERMA V, KAWAGUCHI K, LAMB A, et al. Interpolation consistency training for semi-supervised learning[J]. Neural networks, 2022, 145: 90–106.
[33] REN Zhongzheng, YEH R A, SCHWING A G. Not all unlabeled data are equal: learning to weight data in semi-supervised learning[EB/OL]. (2020-07-02)[2023-05-16].https://arxiv.org/abs/2007.01293.
[34] LI Ming, ZHOU Zhihua. SETRED: self-training with editing[C]//Proceedings of the 9th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining. New York: ACM, 2005: 611–621.
[35] WEI Zhihua, WANG Hanli, ZHAO Rui. Semi-supervised multi-label image classification based on nearest neighbor editing[J]. Neurocomputing, 2013, 119: 462–468.
[36] LI Junnan, ZHU Qingsheng, WU Quanwang. A self-training method based on density peaks and an extended parameter-free local noise filter for k nearest neighbor[J]. Knowledge-based systems, 2019, 184: 104895.
[37] HOULE M E, KRIEGEL H P, KR?GER P, et al. NII Technical Reports[R]. [S.l.: s.n.], 2009: 1–29.
[38] ZIMEK A, SCHUBERT E, KRIEGEL H P. A survey on unsupervised outlier detection in high-dimensional numerical data[J]. Statistical analysis and data mining: the ASA data science journal, 2012, 5(5): 363–387.
[39] 许子微, 陈秀宏. 自步稀疏最优均值主成分分析[J]. 智能系统学报, 2021, 16(3): 416–424
XU Ziwei, CHEN Xiuhong. Sparse optimal mean principal component analysis based on self-paced learning[J]. CAAI transactions on intelligent systems, 2021, 16(3): 416–424
[40] 陆宇,赵凌云,白斌雯,等. 基于改进的半监督聚类的不平衡分类算法[J]. 计算机应用, 2022, 42(12): 3750–3755
LU Yu, ZHAO Lingyun, BAI Binwen, et al. Imbalanced classification algorithm based on improved semi-supervised clustering[J]. Applied science and technology, 2022, 42(12): 3750–3755
[41] 陈波,朱英韬. 基于噪声估计的钢材表面图像增强与缺陷检测[J]. 应用科技, 2023, 50(3): 116–121
CHEN Bo, ZHU Yingtao. Image enhancement and defect detection of steel surface based on noise estimation[J]. Applied science and technology, 2023, 50(3): 116–121
[42] 高玉森,朱昌明,岳闻. 一种改进的增强组合特征判别性的典型相关分析[J]. 应用科技, 2022, 49(2): 119–126
GAO Yusen, ZHU Changming, YUE Wen. An improved canonical correlation analysis to enhance the discriminability of combined features[J]. Applied science and technology, 2022, 49(2): 119–126
[43] 王立国, 马骏宇, 李阳. 联合多种空间信息的高光谱半监督分类方法[J]. 哈尔滨工程大学学报, 2021, 42(2): 280–285
WANG Liguo, MA Junyu, LI Yang. Hyperspectral semi-supervised classification algorithm considering multiple spatial information[J]. Journal of Harbin Engineering University, 2021, 42(2): 280–285
备注/Memo
收稿日期:2023-06-26。
基金项目:国家自然科学基金重大项目（11991024）；重庆市教委“成渝地区双城经济圈建设”科技创新项目（KJCX2020024）；重庆市高校创新研究群体资助项目（CXQT20015）.
作者简介:吕佳，教授，博士，主要研究方向为机器学习、数据挖掘。主持或参与国家级、省部级科研项目共20项，发表学术论文70余篇。E-mail：lvjia@cqnu.edu.cn;邱鸿波，硕士研究生，主要研究方向为机器学习、凸优化算法、噪声标签学习算法。E-mail：2021110516007@cqnu.edu.cn;肖锋，硕士研究生，主要研究方向为机器学习、数据挖掘、数据流算法。E-mail：2021210516083@cqnu.edu.cn
通讯作者:吕佳. E-mail：lvjia@cqnu.edu.cn
更新日期/Last Update:
1900-01-01