[1]高海洋,张明川,葛泉波,等.基于点集匹配的缺陷样本图像生成方法[J].智能系统学报,2023,18(5):1030-1038.[doi:10.11992/tis.202209028]
GAO Haiyang,ZHANG Mingchuan,GE Quanbo,et al.Method of defect sample image generation based on point set matching[J].CAAI Transactions on Intelligent Systems,2023,18(5):1030-1038.[doi:10.11992/tis.202209028]
点击复制
GAO Haiyang,ZHANG Mingchuan,GE Quanbo,et al.Method of defect sample image generation based on point set matching[J].CAAI Transactions on Intelligent Systems,2023,18(5):1030-1038.[doi:10.11992/tis.202209028]
基于点集匹配的缺陷样本图像生成方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第5期
页码:
1030-1038
栏目:
学术论文—机器学习
出版日期:
2023-09-05
- Title:
- Method of defect sample image generation based on point set matching
- 关键词:
- 工业; 缺陷检测; 小样本问题; 点集匹配; 样本扩充; 缺陷样本生成; 有效训练; 循环生成对抗网络模型; 矢量化变分自动编码器
- Keywords:
- industrial; defect detection; small-sample problem; point set matching; sample expansion; generation of the defect sample; effective training; Cycle GAN model; VQ-VAE
- 分类号:
- TP181
- 摘要:
- 针对工业缺陷检测中存在的由产品次品率过低、产品迭代更新过快、缺陷种类难以覆盖全部以及缺陷样本高质量标注难度较高导致的小样本问题，使用基于点集匹配的缺陷样本图像生成方法来对缺陷数据进行扩充。将缺陷部位从多特征角度进行变换，使用单张样本进行扩充得到不同特征的缺陷图像，解决小样本条件下深度学习方法难以生成高质量缺陷图像的问题。通过图像评估与实验验证，该方法生成的图像具有更好的视觉效果，并且对缺陷与分割模型有着高效的提升。该方法可应用于样本较少的深度学习模型训练过程中，达到扩充样本提高训练效果的目的。
- Abstract:
- The paper presents a novel approach for generating defect sample images using point set matching, which addresses the challenges posed by small-sample in industrial defect detection. These challenges arise due to low defective rates of products, rapid iterative updating of products, limited coverage of defect types, and difficulty in obtaining high-quality labeled defect samples. The proposed method transforms defects from a multifeature perspective and applies a single-sample expansion technique to generate defect images with diverse characteristics. This method solves the problem of the difficult generation of a high-quality defect image by deep learning under small-sample conditions. Through image evaluation and experimental verification, this method can produce images with superior visual effects and can effectively improve defect segmentation and detection. This method can be applied to the training process of the deep learning model with few samples for sample expansion and improvement of the training effect.
- 参考文献/References:
-
[1] 于重重, 萨良兵, 马先钦, 等. 基于度量学习的小样本零器件表面缺陷检测[J]. 仪器仪表学报, 2020, 41(7): 214-223
YU Chongchong, SA Liangbing, MA Xianqin, et al. Few-shot parts surface defect detection based on the metric learning[J]. Chinese journal of scientific instrument, 2020, 41(7): 214-223
[2] CHEN Yajun, DING Yuanyuan, ZHAO Fan, et al. Surface defect detection methods for industrial products: a review[J]. Applied sciences, 2021, 11(16): 7657.
[3] GAO Yiping, LI Xinyu, WANG X V, et al. A review on recent advances in vision-based defect recognition towards industrial intelligence[J]. Journal of manufacturing systems, 2022, 62: 753-766.
[4] 胡文杰, 吴晓波, 李波, 等. 基于Self-Attention的单样本ConSinGAN模型的工业缺陷样本图像生成[J]. 中南民族大学学报(自然科学版), 2022, 41(3): 356-364
HU Wenjie, WU Xiaobo, LI Bo, et al. Single sample image generation of industrial defect samples based on self-attention ConSinGAN[J]. Journal of south-central Minzu university (natural science edition), 2022, 41(3): 356-364
[5] KINGMA D P, WELLING M. Auto-encoding variational bayes[J]. Advances in neural information processing systems, 2014: 2002-2009.
[6] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the acm, 2020, 63(11): 139-144.
[7] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000?6010.
[8] HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[J]. Advances in neural information processing systems, 2020, 33: 6840-6851.
[9] SHI Jie, WU Chenfei, LIANG Jian, et al. Divae: Photorealistic images synthesis with denoising diffusion decoder[EB/OL].(2022?06?01)[2022?09?10]. https://arxiv.org/abs/2206.00386.
[10] RAZAVI A, VAN DEN OORD A, VINYALS O. Generating diverse high-fidelity images with vq-vae-2[J]. Advances in Neural Information Processing Systems, 2019: 14866-14876.
[11] LIU Mingyu, BREUEL T, KAUTZ J. Unsupervised image-to-image translation networks[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 700?708.
[12] 余艳杰, 孙嘉琪, 葛思擘, 等. CycleGAN-SN: 结合谱归一化和CycleGAN的图像风格化算法[J]. 西安交通大学学报, 2020, 54(5): 133-141
YU Yanjie, SUN Jiaqi, GE Siqing. CycleGAN-SN: image stylization algorithm combining spectral normalization and CycleGAN[J]. Journal of Xi’an Jiaotong University, 2020, 54(5): 133-141
[13] 叶亚男, 迟静, 于志平, 等. 基于改进CycleGan模型和区域分割的表情动画合成[J]. 计算机科学, 2020, 47(9): 142-149
YE Yanan, CHI Jing, YU Zhiping, et al. Expression animation synthesis based on improved CycleGan model and region segmentation[J]. Computer science, 2020, 47(9): 142-149
[14] K?KSAL A, LU Shijian. RF-GAN: a light and reconfigurable network for unpaired image-to-image translation[C]//ISHIKAWA H, LIU C L, PAJDLA T, et al. Asian Conference on Computer Vision. Cham: Springer, 2021: 542?559.
[15] YEH R A, CHEN Chen, LIM T Y, et al. Semantic image inpainting with deep generative models[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6882?6890.
[16] YU Jiahui, LIN Zhe, YANG Jimei, et al. Generative image inpainting with contextual attention[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 5505?5514.
[17] 崔克彬, 潘锋. 用于绝缘子故障检测的CycleGAN小样本库扩增方法研究[J]. 计算机工程与科学, 2022, 44(3): 509-515
CUI Kebin, PAN Feng. A CycleGAN small sample library amplification method for faulty insulator detection[J]. Computer engineering & science, 2022, 44(3): 509-515
[18] LIU Zirong, LAI Zhihui, GAO Can. Multi-scale defective samples synthesis for surface defect detection[C]//IEEE 7th International Conference on Cloud Computing and Intelligent Systems. Xi’an: IEEE, 2022: 224?229.
[19] SINGH R, GARG R, PATEL N S, et al. Generative adversarial networks for synthetic defect generation in assembly and test manufacturing[C]//31st Annual SEMI Advanced Semiconductor Manufacturing Conference. Saratoga Springs: IEEE, 2020: 1?5.
[20] YANG Benyi, LIU Zhenyu, DUAN Guifang, et al. Mask2Defect: a prior knowledge-based data augmentation method for metal surface defect inspection[J]. IEEE transactions on industrial informatics, 2022, 18(10): 6743-6755.
[21] CHOI J, KIM S, JEONG Y, et al. ILVR: Conditioning method for denoising diffusion probabilistic models[EB/OL]. (2021-08-06)[2022-09-10] . https://arxiv.org/abs/2108.02938.
[22] LUGMAYR A, DANELLJAN M, ROMERO A, et al. RePaint: inpainting using denoising diffusion probabilistic models[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 11451?11461.
[23] MYRONENKO A, SONG Xubo. Point set registration: coherent point drift[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 32(12): 2262-2275.
[24] TANG Te, WANG Changhao, TOMIZUKA M. A framework for manipulating deformable linear objects by coherent point drift[J]. IEEE robotics and automation letters, 2018, 3(4): 3426-3433.
[25] SOTIRAS A, DAVATZIKOS C, PARAGIOS N. Deformable medical image registration: a survey[J]. IEEE transactions on medical imaging, 2013, 32(7): 1153-1190.
[26] MA Jiayi, ZHOU Huabing, ZHAO Ji, et al. Robust feature matching for remote sensing image registration via locally linear transforming[J]. IEEE transactions on geoscience and remote sensing, 2015, 53(12): 6469-6481.
[27] XU Xingpeng, MATKOWSKI W M, KONG A W K. A portrait photo-to-tattoo transform based on digital tattooing[J]. Multimedia tools and applications, 2020, 79(33): 24367-24392.
[28] BERGMANN P, FAUSER M, SATTLEGGER D, et al. MVTec AD—a comprehensive real-world dataset for unsupervised anomaly detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 9584?9592.
- 相似文献/References:
-
[1]吴家伟,严京旗,方志宏,等.基于图像显著性特征的铸坯表面缺陷检测[J].智能系统学报,2012,7(1):75.
WU Jiawei,YAN Jingqi,FANG Zhihong,et al.Defect detection on a steel slab surface based on the characteristics of an image’s saliency region[J].CAAI Transactions on Intelligent Systems,2012,7():75.
[2]逄增治,郑修楠,李金屏.全钢子午线轮胎X光图像的缺陷检测研究现状[J].智能系统学报,2019,14(4):793.[doi:10.11992/tis.201806014]
PANG Zengzhi,ZHENG Xiunan,LI Jinping.Research status of defect detection in X-ray images of all-steel radial tires[J].CAAI Transactions on Intelligent Systems,2019,14():793.[doi:10.11992/tis.201806014]
[3]王凯旋,任福继,倪红军,等.面向电力设备红外图像的温度值识别算法[J].智能系统学报,2022,17(3):617.[doi:10.11992/tis.202105043]
WANG Kaixuan,REN Fuji,NI Hongjun,et al.Temperature value recognition algorithm for the infrared image of power equipment[J].CAAI Transactions on Intelligent Systems,2022,17():617.[doi:10.11992/tis.202105043]
[4]戚银城,耿劭锋,赵振兵,等.基于特征迁移的螺栓图像超分辨率处理方法[J].智能系统学报,2023,18(4):858.[doi:10.11992/tis.202201009]
QI Yincheng,GENG Shaofeng,ZHAO Zhenbing,et al.A method for super resolution processing of bolt image based on feature transfer[J].CAAI Transactions on Intelligent Systems,2023,18():858.[doi:10.11992/tis.202201009]
[5]王延春秋,葛泉波,刘华平.石板材表面缺陷检测的无监督学习方法[J].智能系统学报,2023,18(6):1344.[doi:10.11992/tis.202212006]
WANG Yanchunqiu,GE Quanbo,LIU Huaping.Unsupervised learning method for surface defect detection of slate materials[J].CAAI Transactions on Intelligent Systems,2023,18():1344.[doi:10.11992/tis.202212006]
[6]吴瑞林,葛泉波,刘华平.基于YOLOX的类增量印刷电路板缺陷检测方法[J].智能系统学报,2024,19(4):1061.[doi:10.11992/tis.202309044]
WU Ruilin,GE Quanbo,LIU Huaping.Class-incremental printed circuit board defect detection method based on YOLOX[J].CAAI Transactions on Intelligent Systems,2024,19():1061.[doi:10.11992/tis.202309044]
备注/Memo
收稿日期:2022-9-15。
基金项目:国家自然科学基金重点项目（62033010）；中原科技创新领军人才项目（224200510004）.
作者简介:高海洋，硕士研究生，主要研究方向为图像生成、缺陷检测;张明川，教授，博士生导师，主要研究方向为新型网络、工业互联网、智能信息处理、智慧医疗、机器学习。发表学术论文60余篇;刘华平，教授，博士生导师，中国人工智能学会理事、中国人工智能学会认知系统与信息处理专业委员会秘书长，吴文俊人工智能科学技术奖获得者，主要研究方向为机器人感知、学习与控制、多模态信息融合。主持国家自然科学基金重点项目2项。发表 学术论文100余篇
通讯作者:刘华平.E-mail：hpliu@tsinghua.edu.cn
更新日期/Last Update:
1900-01-01