[1]冷强奎,孙薛梓,孟祥福.一种基于KNN和随机仿射的边界样本合成过采样方法[J].智能系统学报,2025,20(2):329-343.[doi:10.11992/tis.202311038]
LENG Qiangkui,SUN Xuezi,MENG Xiangfu.A borderline sample synthesis oversampling method based on KNN and random affine transformation[J].CAAI Transactions on Intelligent Systems,2025,20(2):329-343.[doi:10.11992/tis.202311038]
点击复制
LENG Qiangkui,SUN Xuezi,MENG Xiangfu.A borderline sample synthesis oversampling method based on KNN and random affine transformation[J].CAAI Transactions on Intelligent Systems,2025,20(2):329-343.[doi:10.11992/tis.202311038]
一种基于KNN和随机仿射的边界样本合成过采样方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
20
期数:
2025年第2期
页码:
329-343
栏目:
学术论文—机器学习
出版日期:
2025-03-05
- Title:
- A borderline sample synthesis oversampling method based on KNN and random affine transformation
- Keywords:
- K-nearest neighbor; linear interpolation; borderline sample; natural distribution; oversampling; three nearest neighbor theory; random affine transformation; imbalanced classification
- 分类号:
- TP391
- 摘要:
- 过采样是处理不平衡数据分类问题的有效策略。本文提出了一种基于K近邻(K-nearest neighbor, KNN)和随机仿射的边界样本合成过采样方法，用于改进现有过采样方法的种子样本选择阶段和合成样本生成阶段。首先，引入三近邻理论，建立样本间有效的内在近邻关系，并去除数据集中的噪声，以降低后续分类器的过拟合风险。其次，准确识别那些难以学习且包含丰富信息的少数类边界样本，并将其用作采样种子。最后，利用局部随机仿射代替线性插值机制，在原始数据的近似流形中均匀地生成合成样本。相比于传统过采样方法，本文方法能更充分挖掘数据集中的重要边界信息，从而为分类器提供更多辅助以改善其分类性能。在18个基准数据集上，与8种经典采样方法(结合4种不同分类器)进行了大量对比实验。结果表明，本文所提方法获得了更高的F1分数和几何均值(G-mean)，可以更为有效地解决不平衡数据分类问题。此外，统计分析也证实该方法具有更高的弗里德曼排名(Friedman ranking)。
- Abstract:
- Oversampling is a proven strategy for addressing imbalanced data classification challenges. This paper introduces a borderline sample synthesis oversampling method based on K-nearest neighbor (KNN) and random affine transformation to improve both the seed sample selection stage and synthetic sample generation stages of existing oversampling methods. Initially, the three nearest neighbor theory is applied to establish an effective intrinsic neighborhood relationship between samples and remove noise from the dataset. This step helps reduce the risk of overfitting by subsequent classifiers. Next, the minority-class borderline samples that are difficult to learn but contain rich information are accurately identified and treated as sampling seeds. Finally, the method replaces traditional linear interpolation with local random affine transformation, uniformly generating synthetic samples within the approximate manifold of the original data. Compared with traditional oversampling methods, the proposed method more effectively leverages important borderline information within datasets, thereby enhancing classifier performance. Extensive comparative experiments were conducted on 18 benchmark datasets, comparing the proposed method against 8 classic sampling methods, each combined with 4 different classifiers. The results show that this method achieves higher F1 scores and geometric means (G-mean), addressing the imbalanced data classification problem more effectively. Furthermore, statistical analysis confirms that the method has a higher Friedman ranking.
- 参考文献/References:
-
[1] GUZMáN-PONCE A, SáNCHEZ J S, VALDOVINOS R M, et al. DBIG-US: a two-stage under-sampling algorithm to face the class imbalance problem[J]. Expert systems with applications, 2021, 168: 114301.
[2] WANG Qingyong, ZHOU Yun, ZHANG Weiming, et al. Adaptive sampling using self-paced learning for imbalanced cancer data pre-diagnosis[J]. Expert systems with applications, 2020, 152: 113334.
[3] SHEN Feng, ZHAO Xingchao, KOU Gang, et al. A new deep learning ensemble credit risk evaluation model with an improved synthetic minority oversampling technique[J].Applied soft computing, 2021, 98: 106852.
[4] RATHORE S S, CHOUHAN S S, JAIN D K, et al. Generative oversampling methods for handling imbalanced data in software fault prediction[J]. IEEE transactions on reliability, 2022, 71(2): 747-76.
[5] WEI Guoliang, MU Weimeng, SONG Yan, et al. An improved and random synthetic minority oversampling technique for imbalanced data[J]. Knowledge-based systems, 2022, 248: 108839.
[6] GUO Haixiang, LI Yijing, SHANG J, et al. Learning from class-imbalanced data: review of methods and applications[J]. Expert systems with applications, 2017, 73: 220-239.
[7] BAO Feng, DENG Yue, KONG Youyong, et al. Learning deep landmarks for imbalanced classification[J]. IEEE transactions on neural networks and learning systems, 2019, 31(8): 2691-2704.
[8] TAO Xinmin, LI Qing, GUO Wenjie, et al. Adaptive weighted over-sampling for imbalanced datasets based on density peaks clustering with heuristic filtering[J]. Information sciences, 2020, 519: 43-73.
[9] EPENDI U, ROCHIM A F, WIBOWO A. A hybrid sampling approach for improving the classification of imbalanced data using ROS and NCL methods[J]. International journal of intelligent engineering and systems, 2023, 16(3): 345-361.
[10] CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: synthetic minority over-sampling technique[J]. Journal of artificial intelligence research, 2002, 16(1): 321-357.
[11] TAO Xinmin, ZHENG Yujia, CHEN Wei, et al. SVDD-based weighted oversampling technique for imbalanced and overlapped dataset learning[J]. Information sciences, 2022, 588: 13-51.
[12] HAN Hui, WANG Wenyuan, MAO Binghuan. Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning[C]//International Conference on Intelligent Computing. Berlin: Springer, 2005: 878-887.
[13] HE Haibo, BAI Yang, GARCIA E A, et al. ADASYN: Adaptive synthetic sampling approach for imbalanced learning[C]//2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). Hong Kong: IEEE, 2008: 1322-1328.
[14] GAO Xin, JIA Xin, LIU Jing, et al. An ensemble contrastive classification framework for imbalanced learning with sample-neighbors pair construction[J]. Knowledge-based systems, 2022, 249: 109007.
[15] THEJAS G S, HARIPRASAD Y, IYENGAR S S, et al. An extension of synthetic minority oversampling technique based on Kalman filter for imbalanced datasets[J]. Machine learning with applications, 2022, 8: 100267.
[16] 周晶雨, 王士同. 对不平衡目标域的多源在线迁移学习[J]. 智能系统学报, 2022, 17(2): 248-256.
ZHOU Jingyu, WANG Shitong. Multi-source online transfer learning for imbalanced target domains[J]. CAAI transactions on intelligent systems, 2022, 17(2): 248-256.
[17] KOZIARSKI M. Radial-based undersampling for imbalanced data classification[J]. Pattern recognition, 2020, 102: 107262.
[18] 陶佳晴, 贺作伟, 冷强奎等. 基于Tomek链的边界少数类样本合成过采样方法[J]. 计算机应用研究, 2023, 40(2): 463-469.
TAO Jiaqing, HE Zuowei, LENG Qiangkui, et al. Synthetic oversampling method for boundary minority samples based on Tomek links[J]. Application research of computers, 2023, 40(2): 463-469.
[19] LENG Qiangkui, GUO Jiamei, JIAO Erjie, et al. NanBDOS: adaptive and parameter-free borderline oversampling via natural neighbor search for class-imbalance learning[J]. Knowledge-based systems, 2023, 274: 110665.
[20] HE Zuowei, TAO Jiaqing, LENG Qiangkui, et al. HS-Gen: a hypersphere-constrained generation mechanism to improve synthetic minority oversampling for imbalanced classification[J]. Complex & intelligent systems, 2023, 9(4): 3971-3988.
[21] BELLINGER C, DRUMMOND C, JAPKOWICZ N. Manifold-based synthetic oversampling with manifold conformance estimation[J]. Machine learning, 2018, 107(3): 605-637.
[22] KOZIARSKI M, KRAWCZYK B, WO?NIAK M. Radial-based oversampling for noisy imbalanced data classification[J]. Neurocomputing, 2019, 343: 19-33.
[23] DOUZAS G, BACAO F. Geometric SMOTE a geometrically enhanced drop-in replacement for SMOTE[J]. Information sciences, 2019, 501: 118-135.
[24] YE Xiucai, LI Hongmin, IMAKURA A, et al. An oversampling framework for imbalanced classification based on Laplacian eigenmaps[J]. Neurocomputing, 2020, 399: 107-116.
[25] BEJ S, DAVTYAN N, WOLFIEN M, et al. LoRAS: an oversampling approach for imbalanced datasets[J]. Machine learning, 2021, 110(2): 279-301.
[26] SA?LAM F, ALI CENGIZ M. A novel SMOTE-based resampling technique trough noise detection and the boosting procedure[J]. Expert systems with applications, 2022, 200: 117023.
[27] SáEZ J A, LUENGO J, STEFANOWSKI J, et al. SMOTE–IPF: addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering[J]. Information sciences, 2015, 291: 184-203.
[28] WANG Xinyue, XU Jian, ZENG Tieyong, et al. Local distribution-based adaptive minority oversampling for imbalanced data classification[J]. Neurocomputing, 2021, 422: 200-213.
[29] KELLY M, LONGJOHN R, NOTTINGHAM K. Machine learning repository[EB/OL]. (1988-07-01)[2023-11-24]. https://archive.ics.uci.edu.
[30] CORTES C, VAPNIK V. Support-vector networks[J]. Machine learning, 1995, 20(3): 273-297.
[31] YANG Kaixiang, YU Zhiwen, WEN Xin, et al. Hybrid classifier ensemble for imbalanced data[J]. IEEE transactions on neural networks and learning systems, 2020, 31(4): 1387-1400.
[32] XIE Yuxi, PENG Lizhi, CHEN Zhenxiang, et al. Generative learning for imbalanced data using the Gaussian mixed model[J]. Applied soft computing, 2019, 79: 439-451.
[33] DEM?AR J. Statistical comparisons of classifiers over multiple data sets[J]. Journal of machine learning research, 2006, 7: 1-30.
- 相似文献/References:
-
[1]张一鸣,欧宗瑛,王 虹.基于C均值K近邻算法的面部表情识别[J].智能系统学报,2008,3(1):57.
ZHANG Yi-ming,OU Zong-ying,WANG Hong.Facial expression recognition based on Cmeans and K-nearest neighbor algorithms[J].CAAI Transactions on Intelligent Systems,2008,3():57.
[2]林琴,李卫军,董肖莉,等.基于双目视觉的人脸三维重建[J].智能系统学报,2018,13(4):534.[doi:10.11992/tis.201701020]
LIN Qin,LI Weijun,DONG Xiaoli,et al.Face reconstruction based on binocular stereo vision[J].CAAI Transactions on Intelligent Systems,2018,13():534.[doi:10.11992/tis.201701020]
[3]郑静,熊伟丽.基于互信息的多块k近邻故障监测及诊断[J].智能系统学报,2021,16(4):717.[doi:10.11992/tis.202007035]
ZHENG Jing,XIONG Weili.Multiblock k-nearest neighbor fault monitoring and diagnosis based on mutual information[J].CAAI Transactions on Intelligent Systems,2021,16():717.[doi:10.11992/tis.202007035]
[4]何罗苏阳,熊伟丽.助训练框架下的半监督软测量建模方法[J].智能系统学报,2023,18(2):231.[doi:10.11992/tis.202111019]
HE Luosuyang,XIONG Weili.Semi-supervised soft sensor modeling method under the help-training framework[J].CAAI Transactions on Intelligent Systems,2023,18():231.[doi:10.11992/tis.202111019]
[5]孙乐,王士同.不确定成对约束的双对抗流形传播方法[J].智能系统学报,2023,18(2):270.[doi:10.11992/tis.202202025]
SUN Le,WANG Shitong.Doubly adversarial manifold propagation on uncertain pairwise constraints[J].CAAI Transactions on Intelligent Systems,2023,18():270.[doi:10.11992/tis.202202025]
[6]赵嘉,马清,肖人彬,等.面向流形数据的共享近邻密度峰值聚类算法[J].智能系统学报,2023,18(4):719.[doi:10.11992/tis.202209026]
ZHAO Jia,MA Qing,XIAO Renbin,et al.Density peaks clustering based on shared nearest neighbor for manifold datasets[J].CAAI Transactions on Intelligent Systems,2023,18():719.[doi:10.11992/tis.202209026]
[7]曾振宇,程雨夏,陶颖,等.数字报版面布局自动生成方法[J].智能系统学报,2024,19(3):679.[doi:10.11992/tis.202207020]
ZENG Zhenyu,CHENG Yuxia,TAO Ying,et al.Automated generation method of digital newspaper layout[J].CAAI Transactions on Intelligent Systems,2024,19():679.[doi:10.11992/tis.202207020]
[8]余紫康,董红斌.具有混合策略的樽海鞘群特征选择算法[J].智能系统学报,2024,19(3):757.[doi:10.11992/tis.202209040]
YU Zikang,DONG Hongbin.Salp swarm feature selection algorithm with a hybrid strategy[J].CAAI Transactions on Intelligent Systems,2024,19():757.[doi:10.11992/tis.202209040]
[9]陈威,吕莉,肖人彬,等.面向混合数据的对称邻域和微簇合并密度峰值聚类算法[J].智能系统学报,2025,20(1):172.[doi:10.11992/tis.202311005]
CHEN Wei,LYU Li,XIAO Renbin,et al.Density peak clustering algorithm based on symmetric neighborhood and micro-cluster merging for mixed datasets[J].CAAI Transactions on Intelligent Systems,2025,20():172.[doi:10.11992/tis.202311005]
备注/Memo
收稿日期:2023-11-24。
基金项目:国家自然科学基金青年项目（61602056）；国家自然科学基金面上项目（61772249）；辽宁省教育厅项目（JYTMS20230819）；辽宁工程技术大学博士科研启动基金项目（21-1043）.
作者简介:冷强奎，教授，博士生导师，博士，中国计算机学会高级会员。主要研究方向为人工智能与机器学习。主持国家自然科学基金青年项目1项、辽宁省博士科研启动基金项目1项、辽宁省自然科学基金项目1项、辽宁省教育厅科研项目2项。发表学术论文30余篇。E-mail：qkleng@126.com;孙薛梓，硕士研究生，主要研究方向为人工智能与机器学习。E-mail：980048119@qq.com;孟祥福，教授，博士生导师，博士，中国计算机学会高级会员。主要研究方向为时空大数据分析、医学影像分析、人工智能。主持国家自然科学基金项目2项、辽宁省高校优秀学校杰出青年学者成长计划项目1项、辽宁省教育厅一般项目2项。获发明专利授权5项、软件著作权10项，发表学术论文80余篇，出版专著2部。E-mail：marxi@126.com。
通讯作者:冷强奎. E-mail：qkleng@126.com
更新日期/Last Update:
2025-03-05