[1]刘嘉轩,胡非易,张辉,等.上下文空间与实例信息的皮肤镜图像自监督分类[J].智能系统学报,2023,18(4):783-792.[doi:10.11992/tis.202211010]
LIU Jiaxuan,HU Feiyi,ZHANG Hui,et al.Dermoscopic images classification based on context and instance-level feature of self-supervised learning[J].CAAI Transactions on Intelligent Systems,2023,18(4):783-792.[doi:10.11992/tis.202211010]
点击复制
LIU Jiaxuan,HU Feiyi,ZHANG Hui,et al.Dermoscopic images classification based on context and instance-level feature of self-supervised learning[J].CAAI Transactions on Intelligent Systems,2023,18(4):783-792.[doi:10.11992/tis.202211010]
上下文空间与实例信息的皮肤镜图像自监督分类
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第4期
页码:
783-792
栏目:
学术论文—智能系统
出版日期:
2023-07-15
- Title:
- Dermoscopic images classification based on context and instance-level feature of self-supervised learning
- Keywords:
- skin disease diagnosis; self-supervised learning; feature extraction; image classification; transfer learning; convolutional neural network; medical image processing; auxiliary diagnosis
- 分类号:
- TP391
- 摘要:
- 针对基于深度学习的皮肤疾病检测方法受限于皮肤镜图像标注难度大、成本高、耗时耗力等问题，本文提出了一种针对皮肤镜图像的自监督学习方法，利用无标注的皮肤镜图像来提高皮肤疾病分类准确率。根据皮肤镜图像空间结构相似度高、信息单一的特点，使用基于旋转预测自监督代理任务，通过模型预测图像旋转的角度，约束模型对图像病变区域的关注，学习上下文空间特征；同时使用个体判别任务，利用正负样本对比学习，获得皮肤镜图像实例信息，为皮肤镜图像的分类提供全局分类信息指导；融合上下文空间特征与实例语义信息得到初始的皮肤疾病检测模型。实验结果表明：本文所提的自监督学习方法，从无标记的皮肤镜样本挖掘出重要信息，更有效地关注皮肤镜图像病变区域，通过融合皮肤镜图像特征信息，使分类结果更加准确。
- Abstract:
- The skin disease detection method based on deep learning is limited by the difficulty of annotations, high cost, time consuming and labor-intensive labeling of dermoscopy images. This paper proposes a self-supervised learning method for dermoscopy images, which uses unmarked dermoscopy images to improve the classification accuracy of skin diseases. According to the characteristics of high similarity and single information of the spatial structure of dermoscopic images, a rotation prediction pretext task is designed to predict the rotation angle of the image through the model, so as to constrain the model’s attention to pathological areas of the image, and learn the context feature. At the same time, the paper presents an instance discrimination task and comparative learning of positive and negative instances to obtain the instance information of dermatoscope images, which provides guidance for global classification information in the classification of dermatoscope images. The initial skin disease detection model is obtained by fusing context features and instance semantic information. The experimental results show that the self-supervised learning method proposed in this paper can mine important information from unmarked dermoscopy samples, pay more attention to the lesion area of dermoscopy images, and make the classification results more accurate by fusing the characteristic information of dermoscopy images.
- 参考文献/References:
-
[1] MIRBEIK-SABZEVARI A, TAVASSOLIAN N. Ultrawideband, stable normal and cancer skin tissue phantoms for millimeter-wave skin cancer imaging[J]. IEEE transactions on bio-medical engineering, 2019, 66(1): 176–186.
[2] HOSNY K M, KASSEM M A, FOAUD M M. Skin cancer classification using deep learning and transfer learning[C]//2018 9th Cairo International Biomedical Engineering Conference. Piscataway: IEEE, 2018: 90-93.
[3] 李晨, 张辉, 张邹铨, 等. 融合多尺度特征与全局上下文信息的X光违禁物品检测[J]. 中国图象图形学报, 2022, 27(10): 3043–3057
LI Chen, ZHANG Hui, ZHANG Zouquan, et al. Integrated multi-scale features and global context in X-ray detection for prohibited items[J]. Chinese journal of image and graphics, 2022, 27(10): 3043–3057
[4] SHEN Dinggang, WU Guorong, SUK H I. Deep learning in medical image analysis[J]. Annual review of biomedical engineering, 2017, 19: 221–248.
[5] SHIN H C, ROTH H R, GAO Mingchen, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning[J]. IEEE transactions on medical imaging, 2016, 35(5): 1285–1298.
[6] SADEGHI M, LEE T K, MCLEAN D, et al. Detection and analysis of irregular streaks in dermoscopic images of skin lesions[J]. IEEE transactions on medical imaging, 2013, 32(5): 849–861.
[7] MA Yuhui, LIU Jiang, LIU Yonghuai, et al. Structure and illumination constrained GAN for medical image enhancement[J]. IEEE transactions on medical imaging, 2021, 40(12): 3955–3967.
[8] BAILO O, HAM D, SHIN Y M. Red blood cell image generation for data augmentation using conditional generative adversarial networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2019: 1039-1048.
[9] JING Longlong, TIAN Yingli. Self-supervised visual feature learning with deep neural networks: a survey[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(11): 4037–4058.
[10] 莫宏伟, 傅智杰. 基于迁移学习的无监督跨域人脸表情识别[J]. 智能系统学报, 2021, 16(3): 397–406
MO Hongwei, FU Zhijie. Unsupervised cross-domain expression recognition based on transfer learning[J]. CAAI transactions on intelligent systems, 2021, 16(3): 397–406
[11] 杜航原, 张晶, 王文剑. 一种深度自监督聚类集成算法[J]. 智能系统学报, 2020, 15(6): 1113–1120
DU Hangyuan, ZHANG Jing, WANG Wenjian. A deep self-supervised clustering ensemble algorithm[J]. CAAI transactions on intelligent systems, 2020, 15(6): 1113–1120
[12] WU Zhirong, XIONG Yuanjun, YU S X, et al. Unsupervised feature learning via non-parametric instance discrimination[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 3733-3742.
[13] FENG Zeyu, XU Chang, TAO Dacheng. Self-supervised representation learning by rotation feature decoupling[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 10356-10366.
[14] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2016: 770-778.
[15] HE Kaiming, FAN Haoqi, WU Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 9726-9735.
[16] ZIMMERMANN R S , SHARMA Y, SCHNEIDER S, et al. Contrastive learning inverts the data generating process[EB/OL]. (2021?02?17)[2022?11?10]. https://arxiv.org/abs/2102.08850.
[17] ROTEMBERG V, KURTANSKY N, BETZ-STABLEIN B, et al. A patient-centric dataset of images and metadata for identifying melanomas using clinical context[J]. Scientific data, 2021, 8(1): 34.
[18] TSCHANDL P, ROSENDAHL C, KITTLER H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions[J]. Scientific data, 2018, 5: 180161.
[19] CHEN Ting, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ACM, 2020: 1597?1607.
[20] GIDARIS S, SINGH P, KOMODAKIS N. Unsupervised representation learning by predicting image rotations[EB/OL]. (2018?03?21)[2022?11?10].https://arxiv.org/abs/1803.07728.
[21] GRILL J B, STRUB F, ALTCHé F, et al. Bootstrap your own latent a new approach to self-supervised learning[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 21271?21284.
[22] MISRA I, VAN DER MAATEN L. Self-supervised learning of pretext-invariant representations[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2020: 6706-6716.
[23] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2009: 248-255.
[24] ROBBINS H, MONRO S. A stochastic approximation method[J]. The annals of mathematical statistics, 1951, 22(3): 400–407.
[25] RODRIGUEZ J D, PEREZ A, LOZANO J A. Sensitivity analysis of k-fold cross validation in prediction error estimation[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 32(3): 569–575.
[26] JIANG Pengtao, ZHANG Changbin, HOU Qibin, et al. Layercam: exploring hierarchical class activation maps for localization[J]. IEEE transactions on image processing, 2021, 30: 5875–5888.
- 相似文献/References:
-
[1]商显震,韩萌,王少峰,等.融合迁移学习和神经网络的皮肤病诊断方法[J].智能系统学报,2020,15(3):452.[doi:10.11992/tis.201811015]
SHANG Xianzhen,HAN Meng,WANG Shaofeng,et al.A skin diseases diagnosis method combining transfer learning and neural networks[J].CAAI Transactions on Intelligent Systems,2020,15():452.[doi:10.11992/tis.201811015]
[2]杜航原,张晶,王文剑.一种深度自监督聚类集成算法[J].智能系统学报,2020,15(6):1113.[doi:10.11992/tis.202006050]
DU Hangyuan,ZHANG Jing,WANG Wenjian.A deep self-supervised clustering ensemble algorithm[J].CAAI Transactions on Intelligent Systems,2020,15():1113.[doi:10.11992/tis.202006050]
[3]陈丽,马楠,逄桂林,等.多视角数据融合的特征平衡YOLOv3行人检测研究[J].智能系统学报,2021,16(1):57.[doi:10.11992/tis.202010003]
CHEN Li,MA Nan,PANG Guilin,et al.Research on multi-view data fusion and balanced YOLOv3 for pedestrian detection[J].CAAI Transactions on Intelligent Systems,2021,16():57.[doi:10.11992/tis.202010003]
[4]丁维昌,施俊,王骏.自监督对比特征学习的多模态乳腺超声诊断[J].智能系统学报,2023,18(1):66.[doi:10.11992/tis.202111052]
DING Weichang,SHI Jun,WANG Jun.Multi-modality ultrasound diagnosis of the breast with self-supervised contrastive feature learning[J].CAAI Transactions on Intelligent Systems,2023,18():66.[doi:10.11992/tis.202111052]
[5]黎建宇,詹志辉.面向大规模特征选择的自监督数据驱动粒子群优化算法[J].智能系统学报,2023,18(1):194.[doi:10.11992/tis.202206008]
LI Jianyu,ZHAN Zhihui.A self-supervised data-driven particle swarm optimization approach for large-scale feature selection[J].CAAI Transactions on Intelligent Systems,2023,18():194.[doi:10.11992/tis.202206008]
备注/Memo
收稿日期:2022-11-10。
基金项目:国家重大研究计划重点支持项目（92148204）；科技创新2030“新一代人工智能”重大项目（2021ZD0114503）；国家自然科学基金项目 (61971071，62027810，62133005)；湖南省杰出青年科学基金项目 (2021JJ10025) ；湖南省重点研发计划（2021GK4011，2022GK2011）；长沙市科技重大专项 (kh2003026)；机器人学国家重点实验室联合开放基金项目（2021-KF-22-17）；中国高校产学研创新基金项目（2020HYA06006）.
作者简介:刘嘉轩，硕士研究生，主要研究方向为医学图像分析和计算机视觉;胡非易，硕士研究生，主要研究方向为图像识别、医学图像分析;张辉，教授，博士生导师，主要研究方向为计算机视觉。获省部级科学技术奖励一等奖8项，获2022年湖南省第十三届教学成果特等奖等，主持科技创新2030—“新一代人工智能”重大项目、国家自然科学基金共融机器人重大研究计划重点项目、国家重点研发计划子课题、国家科技支撑计划项目子课题等20余项，授权发明专利38项。发表学术论文50余篇。
通讯作者:张辉.E-mail:zhanghuihby@126.com
更新日期/Last Update:
1900-01-01