[1]胡峰,李路正,代劲,等.结合聚类边界采样的主动学习[J].智能系统学报,2024,19(2):482-492.[doi:10.11992/tis.202205020]
HU Feng,LI Luzheng,DAI Jin,et al.Active learning combined with clustering boundary sampling[J].CAAI Transactions on Intelligent Systems,2024,19(2):482-492.[doi:10.11992/tis.202205020]
点击复制
HU Feng,LI Luzheng,DAI Jin,et al.Active learning combined with clustering boundary sampling[J].CAAI Transactions on Intelligent Systems,2024,19(2):482-492.[doi:10.11992/tis.202205020]
结合聚类边界采样的主动学习
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第2期
页码:
482-492
栏目:
人工智能院长论坛
出版日期:
2024-03-05
- Title:
- Active learning combined with clustering boundary sampling
- Keywords:
- active learning; machine learning; cluster boundary; density peak clustering; geometric sampling; entropy; version space; active clustering
- 分类号:
- TP301
- 文献标志码:
- 2023-11-20
- 摘要:
- 主动学习是一种机器学习方法，需要选择最有价值的样本进行标注。目前，主动学习在应用时面临着一些挑战，其依赖分类器的先验假设，这容易导致分类器性能意外下降，同时需要一定规模的样本作为启动条件。聚类可以降低问题规模，是主动学习的一种有效手段。为此，结合密度聚类边界采样，开展主动学习方法的研究。针对容易产生分类错误的聚类边界区域，通过计算样本密度，提出一种密度峰值聚类边界点采样方法；在此基础上，给出密度熵的定义，并利用密度熵对聚类边界区域进行启发式搜索，提出一种基于聚类边界采样的主动学习方法。试验结果表明，与文献中的5种主动学习算法相比，该算法能够以更少标记量获得同等甚至更高的分类性能，是一种有效的主动学习算法；在标记不足，无标签样本总量20%的情况下，算法在Accuracy、F-score等指标上取得较好的结果。
- Abstract:
- Active learning is a machine learning method that requires the selection of the most valuable samples for labeling. Currently, active learning encounters certain challenges in its practical application. It relies on prior assumptions of the classifier, which can lead to unexpected declines in classifier performance and requires a specific number of samples as an initial condition. Clustering, which can reduce the complexity of a problem, serves as an effective tool in active learning. Based on density clustering boundary sampling, this study focuses on active learning methods. First, a method of sampling boundary points in density peak clustering is introduced. This method calculates the sample density for a clustering boundary region that is prone to classification errors. Subsequently, with a specified definition of density entropy, an active learning method based on cluster boundary sampling is proposed. This method employs density entropy for the heuristic search of cluster boundary regions. The experimental results show that the proposed algorithm, compared with the five active learning algorithms referenced in the literature, can achieve equal or even higher classification performance with fewer markers. This proves that it is an effective active learning algorithm. When the number of labeled samples is less than 20% of the total number of unlabeled samples, the algorithm achieves better results in the accuracy and F-score metrics.
- 参考文献/References:
-
[1] SHAHRAKI A, ABBASI M, TAHERKORDI A, et al. Active learning for network traffic classification: a technical study[J]. IEEE transactions on cognitive communications and networking, 2021, 8(1): 422–439.
[2] NATH V, YANG Dong, LANDMAN B A, et al. Diminishing uncertainty within the training pool: active learning for medical image segmentation[J]. IEEE transactions on medical imaging, 2021, 40(10): 2534–2547.
[3] 陈立伟, 房赫, 朱海峰. 多视图主动学习的多样性样本选择方法研究[J]. 智能系统学报, 2021, 16(6): 1007–1014
CHEN Liwei, FANG He, ZHU Haifeng. Diversity sample selection method of multiview active learning classification[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1007–1014
[4] CARCILLO F, LE BORGNE Y A, CAELEN O, et al. Streaming active learning strategies for real-life credit card fraud detection: assessment and visualization[J]. International journal of data science and analytics, 2018, 5(4): 285–300.
[5] OWOYELE O, PAL P, VIDAL TORREIRA A. An automated machine learning-genetic algorithm framework with active learning for design optimization[J]. Journal of energy resources technology, 2021, 143(8): 082305.
[6] AGGARWAL C C, KONG X, GU Q, et al. Active learning: A survey [M]. [S. l. ]: Algorithms and Applications, 2014: 571-605.
[7] LEWIS D D, GALE W A. A sequential algorithm for training text classifiers[C]//Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.[S.l.]: SIGIR 1994, 29: 3-12.
[8] KEE S, DEL CASTILLO E, RUNGER G. Query-by-committee improvement with diversity and density in batch active learning[J]. Information sciences, 2018, 454/455: 401–418.
[9] SHAO Hao. Query by diverse committee in transfer active learning[J]. Frontiers of computer science, 2019, 13(2): 280–291.
[10] SETTLES B, CRAVEN M. An analysis of active learning strategies for sequence labeling tasks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. New York: ACM, 2008: 1070-1079.
[11] MIN Fan, ZHANG Shiming, CIUCCI D, et al. Three-way active learning through clustering selection[J]. International journal of machine learning and cybernetics, 2020, 11(5): 1033–1046.
[12] YAO Yiyu. Three-way decisions with probabilistic rough sets[J]. Information sciences, 2010, 180(3): 341–353.
[13] HOI S C H, JIN Rong, ZHU Jianke, et al. Semi-supervised SVM batch mode active learning for image retrieval[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage: IEEE, 2008: 1-7.
[14] HUANG Shengjun, JIN Rong, ZHOU Zhihua. Active learning by querying informative and representative examples[J]. IEEE transactions on pattern analysis and machine intelligence, 2014, 36(10): 1936–1949.
[15] DONG Shi. Multi class SVM algorithm with active learning for network traffic classification[J]. Expert systems with applications, 2021, 176: 114885.
[16] SIDDIQUI Y, VALENTIN J, NIESSNER M. ViewAL: active learning with viewpoint entropy for semantic segmentation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 9430-9440.
[17] CAO Xiaofeng. A structured perspective of volumes on active learning[J]. Neurocomputing, 2020, 377: 200–212.
[18] CAO Xiaofeng. A divide-and-conquer approach to geometric sampling for active learning[J]. Expert systems with applications, 2020, 140: 112907.
[19] RODRIGUEZ A, LAIO A. Clustering by fast search and find of density peaks[J]. Science, 2014, 344(6191): 1492–1496.
[20] SETTIES B. Curious machines: active learning with structured instances[J]. Journal of chemical information and modeling, 2013, 53(9): 1689–1699.
[21] XIA Chenyi, HSU W, LEE M L, et al. BORDER: efficient computation of boundary points[J]. IEEE transactions on knowledge and data engineering, 2006, 18(3): 289–303.
[22] QIU Baozhi, CAO Xiaofeng. Clustering boundary detection for high dimensional space based on space inversion and Hopkins statistics[J]. Knowledge-based systems, 2016, 98: 216–225.
[23] AGGARWAL C C. An introduction to outlier analysis[M]//Outlier Analysis. Cham: Springer International Publishing, 2016: 1-34.
[24] KONYUSHKOVA K, SZNITMAN R, FUA P. Learning active learning from data[J]. Conference and workshop on neural information processing systems, 2017, 31(12): 4226–4236.
[25] HE Deniu, YU Hong, WANG Guoyin, et al. A two-stage clustering-based cold-start method for active learning[J]. Intelligent data analysis, 2021, 25(5): 1169–1185.
[26] KARAMCHETI S, KRISHNA R, LI Feifei, et al. Mind your outliers! investigating the negative impact of outliers on active learning for visual question answering[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2021: 7265-7281.
- 相似文献/References:
-
[1]叶志飞,文益民,吕宝粮.不平衡分类问题研究综述[J].智能系统学报,2009,4(2):148.
YE Zhi-fei,WEN Yi-min,LU Bao-liang.A survey of imbalanced pattern classification problems[J].CAAI Transactions on Intelligent Systems,2009,4():148.
[2]刘奕群,张 敏,马少平.基于非内容信息的网络关键资源有效定位[J].智能系统学报,2007,2(1):45.
LIU Yi-qun,ZHANG Min,MA Shao-ping.Web key resource page selection based on non-content inf o rmation[J].CAAI Transactions on Intelligent Systems,2007,2():45.
[3]马世龙,眭跃飞,许 可.优先归纳逻辑程序的极限行为[J].智能系统学报,2007,2(4):9.
MA Shi-long,SUI Yue-fei,XU Ke.Limit behavior of prioritized inductive logic programs[J].CAAI Transactions on Intelligent Systems,2007,2():9.
[4]古丽娜孜,孙铁利,伊力亚尔,等.一种基于主动学习支持向量机哈萨克文文本分类方法[J].智能系统学报,2011,6(3):261.
GU Linazi,SUN Tieli,YI Liyaer,et al.An approach to the text categorization of the Kazakh language based on an active learning support vector machine[J].CAAI Transactions on Intelligent Systems,2011,6():261.
[5]姚伏天,钱沄涛.高斯过程及其在高光谱图像分类中的应用[J].智能系统学报,2011,6(5):396.
YAO Futian,QIAN Yuntao.Gaussian process and its applications in hyperspectral image classification[J].CAAI Transactions on Intelligent Systems,2011,6():396.
[6]文益民,强保华,范志刚.概念漂移数据流分类研究综述[J].智能系统学报,2013,8(2):95.[doi:10.3969/j.issn.1673-4785.201208012]
WEN Yimin,QIANG Baohua,FAN Zhigang.A survey of the classification of data streams with concept drift[J].CAAI Transactions on Intelligent Systems,2013,8():95.[doi:10.3969/j.issn.1673-4785.201208012]
[7]杨成东,邓廷权.综合属性选择和删除的属性约简方法[J].智能系统学报,2013,8(2):183.[doi:10.3969/j.issn.1673-4785.201209056]
YANG Chengdong,DENG Tingquan.An approach to attribute reduction combining attribute selection and deletion[J].CAAI Transactions on Intelligent Systems,2013,8():183.[doi:10.3969/j.issn.1673-4785.201209056]
[8]胡小生,钟勇.基于加权聚类质心的SVM不平衡分类方法[J].智能系统学报,2013,8(3):261.
HU Xiaosheng,ZHONG Yong.Support vector machine imbalanced data classification based on weighted clustering centroid[J].CAAI Transactions on Intelligent Systems,2013,8():261.
[9]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[10]孔庆超,毛文吉,张育浩.社交网站中用户评论行为预测[J].智能系统学报,2015,10(3):349.[doi:10.3969/j.issn.1673-4785.201403019]
KONG Qingchao,MAO Wenji,ZHANG Yuhao.User comment behavior prediction in social networking sites[J].CAAI Transactions on Intelligent Systems,2015,10():349.[doi:10.3969/j.issn.1673-4785.201403019]
备注/Memo
收稿日期:2022-05-17。
基金项目:国家重点研发计划项目（2018YFC0832102）；重庆市教委重点合作项目（HZ2021008）；重庆市自然科学基金项目（cstc2021jcyj-msxmX0849）.
作者简介:胡峰，教授，博士，主要研究方向为粗糙集、粒计算、数据挖掘。主持和参与国家自然科学基金项目4项，参与科技部重点研发计划项目3项，作为参与者获吴文俊人工智能科学技术奖、重庆市自然科学奖各1项，发表学术论文40余篇。E-mail：hufeng@ cqupt.edu.cn;李路正，硕士研究生，主要研究方向为数据挖掘、主动学习。E-mail：isluzheng.li@foxmail.com;代劲，教授，博士，重庆邮电大学软件学院副院长。主要研究方向为大数据知识工程、智能信息处理。先后承担和完成省部级科研项目4项，出版专著1部，发表学术论文20余篇。E-mail：331545392@qq.com
通讯作者:胡峰. E-mail：hufeng@cqupt.edu.cn
更新日期/Last Update:
1900-01-01