[1]GAO Haiyang,ZHANG Mingchuan,GE Quanbo,et al.Method of defect sample image generation based on point set matching[J].CAAI Transactions on Intelligent Systems,2023,18(5):1030-1038.[doi:10.11992/tis.202209028]
Copy

Method of defect sample image generation based on point set matching

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 5 Page number: 1030-1038 Column: 学术论文—机器学习 Public date: 2023-09-05
Title:
Method of defect sample image generation based on point set matching
Author(s):
GAO Haiyang1;  ZHANG Mingchuan1;  GE Quanbo2;  LIU Huaping3
1. School of Information Engineering, Henan University of Science and Technology, Luoyang 471023, China;
2. School of Automation, Nanjing University of Information Science and Technology, Nanjing 210044, China;
3. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
Keywords:
industrial; defect detection; small-sample problem; point set matching; sample expansion; generation of the defect sample; effective training; Cycle GAN model; VQ-VAE
CLC:
TP181
DOI:
10.11992/tis.202209028
Abstract:
The paper presents a novel approach for generating defect sample images using point set matching, which addresses the challenges posed by small-sample in industrial defect detection. These challenges arise due to low defective rates of products, rapid iterative updating of products, limited coverage of defect types, and difficulty in obtaining high-quality labeled defect samples. The proposed method transforms defects from a multifeature perspective and applies a single-sample expansion technique to generate defect images with diverse characteristics. This method solves the problem of the difficult generation of a high-quality defect image by deep learning under small-sample conditions. Through image evaluation and experimental verification, this method can produce images with superior visual effects and can effectively improve defect segmentation and detection. This method can be applied to the training process of the deep learning model with few samples for sample expansion and improvement of the training effect.
References:
[1] 于重重, 萨良兵, 马先钦, 等. 基于度量学习的小样本零器件表面缺陷检测[J]. 仪器仪表学报, 2020, 41(7): 214-223
YU Chongchong, SA Liangbing, MA Xianqin, et al. Few-shot parts surface defect detection based on the metric learning[J]. Chinese journal of scientific instrument, 2020, 41(7): 214-223
[2] CHEN Yajun, DING Yuanyuan, ZHAO Fan, et al. Surface defect detection methods for industrial products: a review[J]. Applied sciences, 2021, 11(16): 7657.
[3] GAO Yiping, LI Xinyu, WANG X V, et al. A review on recent advances in vision-based defect recognition towards industrial intelligence[J]. Journal of manufacturing systems, 2022, 62: 753-766.
[4] 胡文杰, 吴晓波, 李波, 等. 基于Self-Attention的单样本ConSinGAN模型的工业缺陷样本图像生成[J]. 中南民族大学学报(自然科学版), 2022, 41(3): 356-364
HU Wenjie, WU Xiaobo, LI Bo, et al. Single sample image generation of industrial defect samples based on self-attention ConSinGAN[J]. Journal of south-central Minzu university (natural science edition), 2022, 41(3): 356-364
[5] KINGMA D P, WELLING M. Auto-encoding variational bayes[J]. Advances in neural information processing systems, 2014: 2002-2009.
[6] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the acm, 2020, 63(11): 139-144.
[7] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000?6010.
[8] HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[J]. Advances in neural information processing systems, 2020, 33: 6840-6851.
[9] SHI Jie, WU Chenfei, LIANG Jian, et al. Divae: Photorealistic images synthesis with denoising diffusion decoder[EB/OL].(2022?06?01)[2022?09?10]. https://arxiv.org/abs/2206.00386.
[10] RAZAVI A, VAN DEN OORD A, VINYALS O. Generating diverse high-fidelity images with vq-vae-2[J]. Advances in Neural Information Processing Systems, 2019: 14866-14876.
[11] LIU Mingyu, BREUEL T, KAUTZ J. Unsupervised image-to-image translation networks[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 700?708.
[12] 余艳杰, 孙嘉琪, 葛思擘, 等. CycleGAN-SN: 结合谱归一化和CycleGAN的图像风格化算法[J]. 西安交通大学学报, 2020, 54(5): 133-141
YU Yanjie, SUN Jiaqi, GE Siqing. CycleGAN-SN: image stylization algorithm combining spectral normalization and CycleGAN[J]. Journal of Xi’an Jiaotong University, 2020, 54(5): 133-141
[13] 叶亚男, 迟静, 于志平, 等. 基于改进CycleGan模型和区域分割的表情动画合成[J]. 计算机科学, 2020, 47(9): 142-149
YE Yanan, CHI Jing, YU Zhiping, et al. Expression animation synthesis based on improved CycleGan model and region segmentation[J]. Computer science, 2020, 47(9): 142-149
[14] K?KSAL A, LU Shijian. RF-GAN: a light and reconfigurable network for unpaired image-to-image translation[C]//ISHIKAWA H, LIU C L, PAJDLA T, et al. Asian Conference on Computer Vision. Cham: Springer, 2021: 542?559.
[15] YEH R A, CHEN Chen, LIM T Y, et al. Semantic image inpainting with deep generative models[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6882?6890.
[16] YU Jiahui, LIN Zhe, YANG Jimei, et al. Generative image inpainting with contextual attention[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 5505?5514.
[17] 崔克彬, 潘锋. 用于绝缘子故障检测的CycleGAN小样本库扩增方法研究[J]. 计算机工程与科学, 2022, 44(3): 509-515
CUI Kebin, PAN Feng. A CycleGAN small sample library amplification method for faulty insulator detection[J]. Computer engineering & science, 2022, 44(3): 509-515
[18] LIU Zirong, LAI Zhihui, GAO Can. Multi-scale defective samples synthesis for surface defect detection[C]//IEEE 7th International Conference on Cloud Computing and Intelligent Systems. Xi’an: IEEE, 2022: 224?229.
[19] SINGH R, GARG R, PATEL N S, et al. Generative adversarial networks for synthetic defect generation in assembly and test manufacturing[C]//31st Annual SEMI Advanced Semiconductor Manufacturing Conference. Saratoga Springs: IEEE, 2020: 1?5.
[20] YANG Benyi, LIU Zhenyu, DUAN Guifang, et al. Mask2Defect: a prior knowledge-based data augmentation method for metal surface defect inspection[J]. IEEE transactions on industrial informatics, 2022, 18(10): 6743-6755.
[21] CHOI J, KIM S, JEONG Y, et al. ILVR: Conditioning method for denoising diffusion probabilistic models[EB/OL]. (2021-08-06)[2022-09-10] . https://arxiv.org/abs/2108.02938.
[22] LUGMAYR A, DANELLJAN M, ROMERO A, et al. RePaint: inpainting using denoising diffusion probabilistic models[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 11451?11461.
[23] MYRONENKO A, SONG Xubo. Point set registration: coherent point drift[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 32(12): 2262-2275.
[24] TANG Te, WANG Changhao, TOMIZUKA M. A framework for manipulating deformable linear objects by coherent point drift[J]. IEEE robotics and automation letters, 2018, 3(4): 3426-3433.
[25] SOTIRAS A, DAVATZIKOS C, PARAGIOS N. Deformable medical image registration: a survey[J]. IEEE transactions on medical imaging, 2013, 32(7): 1153-1190.
[26] MA Jiayi, ZHOU Huabing, ZHAO Ji, et al. Robust feature matching for remote sensing image registration via locally linear transforming[J]. IEEE transactions on geoscience and remote sensing, 2015, 53(12): 6469-6481.
[27] XU Xingpeng, MATKOWSKI W M, KONG A W K. A portrait photo-to-tattoo transform based on digital tattooing[J]. Multimedia tools and applications, 2020, 79(33): 24367-24392.
[28] BERGMANN P, FAUSER M, SATTLEGGER D, et al. MVTec AD—a comprehensive real-world dataset for unsupervised anomaly detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 9584?9592.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems