[1]DING Weichang,SHI Jun,WANG Jun.Multi-modality ultrasound diagnosis of the breast with self-supervised contrastive feature learning[J].CAAI Transactions on Intelligent Systems,2023,18(1):66-74.[doi:10.11992/tis.202111052]
Copy

Multi-modality ultrasound diagnosis of the breast with self-supervised contrastive feature learning

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 1 Page number: 66-74 Column: 学术论文—智能系统 Public date: 2023-01-05
Title:
Multi-modality ultrasound diagnosis of the breast with self-supervised contrastive feature learning
Author(s):
DING Weichang;  SHI Jun;  WANG Jun
School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China
Keywords:
self-supervised learning; contrastive learning; ultrasound image; flexible ultrasound; B-mode ultrasound; multi-modality; breast cancer; computer-aided diagnosis; deep learning
CLC:
TP391
DOI:
10.11992/tis.202111052
Abstract:
An automatic ultrasound-based diagnosis of breast cancer has important clinical value. However, high-precision automatic diagnosis methods are very difficult to construct because many labeled data are missing. Recently, self-supervised contrastive learning has shown great potential in using unlabeled natural images to generate discriminative and highly generalized features. However, this approach is not applicable to using natural images to construct positive and negative samples in the field of breast ultrasound. To this end, this work introduces the elastography ultrasound (EUS) image and proposes a self-supervised contrastive learning method integrating multimodal information based on the multimodal features of an EUS image. Specifically, positive and negative samples are constructed using multi-modality ultrasound images collected from the same and different patients, respectively. We construct the object learning criterion of contrastive learning based on modal consistency, rotation invariance, and sample separation. The EUS information is integrated into the model by learning the unified feature representation for both modalities in the embedding space, which improves model performance in the downstream B-mode ultrasound classification task. The experimental results show that our method can fully mine the high-level semantic features from unlabeled multimodal breast ultrasound images, thereby effectively improving the diagnosis accuracy of breast cancer.
References:
[1] CHENG H D, SHAN Juan, JU Wen, et al. Automated breast cancer detection and classification using ultrasound images: a survey[J]. Pattern recognition, 2010, 43(1): 299–317.
[2] 龚勋, 杨菲, 杜章锦, 等. 甲状腺、乳腺超声影像自动分析技术综述[J]. 软件学报, 2020, 31(7): 2245–2282
GONG Xun, YANG Fei, DU Zhangjin, et al. Survey of automatic ultrasonographic analysis for thyroid and breast[J]. Journal of software, 2020, 31(7): 2245–2282
[3] HUANG Qinghua, ZHANG Fan, LI Xuelong. Machine learning in ultrasound computer-aided diagnostic systems: a survey[J]. BioMed research international, 2018, 2018: 5137904.
[4] 郑光远, 刘峡壁, 韩光辉. 医学影像计算机辅助检测与诊断系统综述[J]. 软件学报, 2018, 29(5): 1471–1514
ZHENG Guangyuan, LIU Xiabi, HAN Guanghui. Survey on medical image computer aided detection and diagnosis systems[J]. Journal of software, 2018, 29(5): 1471–1514
[5] SIGRIST R M S, LIAU J, KAFFAS A E, et al. Ultrasound elastography: review of techniques and clinical applications[J]. Theranostics, 2017, 7(5): 1303–1329.
[6] 刘飞, 张俊然, 杨豪. 基于深度学习的医学图像识别研究进展[J]. 中国生物医学工程学报, 2018, 37(01): 86–94
LIU Fei, ZHANG Junran, YANGHao. Advances in medical image recognition based on deep learning[J]. Journal of biomedical engineering, 2018, 37(01): 86–94
[7] 高歌, 马帅, 王霄英. 计算机辅助诊断在医学影像诊断中的基本原理和应用进展[J]. 放射学实践, 2016, 31(12): 1127–1129
GAO Ge, MA Shuai, WANG Xiaoying. Basic principle and application progress of computer aided diagnosis in medical image diagnosis[J]. Radiologic practice, 2016, 31(12): 1127–1129
[8] 周涛, 陆惠玲, 陈志强, 等. 多模态医学影像融合识别技术研究进展[J]. 生物医学工程学杂志, 2013, 30(05): 1117–1122
ZHOU Tao, LU Huiling, CHEN Zhiqiang, et al. Research progress of multimodal medical image fusion recognition technology[J]. Journal of Biomedical Engineering, 2013, 30(05): 1117–1122
[9] JING Longlong, TIAN Yingli. Self-supervised visual feature learning with deep neural networks: a survey[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(11): 4037–4058.
[10] HE Kaiming, FAN Haoqi, WU Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 9726?9735.
[11] CHEN Ting, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[C]//ICML’20: Proceedings of the 37th International Conference on Machine Learning. New York: ACM, 2020: 1597-1607.
[12] MISRA I, VAN DER MAATEN L. Self-supervised learning of pretext-invariant representations[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6706?6716.
[13] GRILL J B, STRUB F, ALTCHé F, et al. Bootstrap your own latent-a new approach to self-supervised learning[J]. Advances in neural information processing systems, 2020(33): 21271–21284.
[14] GIDARIS S, SINGH P, KOMODAKIS N. Unsupervised representation learning by predicting image rotations[EB/OL]. (2018?03?21)[2021?11?29].https://arxiv.org/abs/1803.07728.
[15] PATHAK D, KR?HENBüHL P, DONAHUE J, et al. Context encoders: feature learning by inpainting[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2536?2544.
[16] NOROOZI M, FAVARO P. Unsupervised learning of visual representations by solving jigsaw puzzles[M]//Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016: 69?84.
[17] ZHUANG Xinrui, LI Yuexiang, HU Yifan, et al. Self-supervised feature learning for 3D medical images by playing a rubik’s cube[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2019: 420?428.
[18] BAI Wenjia, CHEN Chen, TARRONI G, et al. Self-supervised learning for cardiac MR image segmentation by anatomical position prediction[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2019: 541?549.
[19] BLENDOWSKI M, NICKISCH H, HEINRICH M P. How to learn from unlabeled volume data: self-supervised 3D context feature learning[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2019: 649?657.
[20] SPITZER H, KIWITZ K, AMUNTS K, et al. Improving cytoarchitectonic segmentation of human brain areas with self-supervised Siamese networks[M]//Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Cham: Springer International Publishing, 2018: 663-671.
[21] HADSELL R, CHOPRA S, LECUN Y. Dimensionality reduction by learning an invariant mapping[C]//2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2006: 1735?1742.
[22] DOSOVITSKIY A, FISCHER P, SPRINGENBERG J T, et al. Discriminative unsupervised feature learning with exemplar convolutional neural networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(9): 1734–1747.
[23] WU Zhirong, XIONG Yuanjun, YU S X, et al. Unsupervised feature learning via non-parametric instance discrimination[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3733?3742.
[24] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770?778.
[25] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[EB/OL]. (2015?03?09)[2021?11?29].https://arxiv.org/abs/1503.02531.
[26] VAN der MAATEN, LAURENS, HINTON Geoffrey. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(11).
[27] SHIN S Y, LEE S, YUN I D, et al. Joint weakly and semi-supervised deep learning for localization and classification of masses in breast ultrasound images[J]. IEEE transactions on medical imaging, 2019, 38(3): 762–774.
[28] YAP M H, GOYAL M, OSMAN F M, et al. Breast ultrasound lesions recognition: end-to-end deep learning approaches[J]. Journal of medical imaging, 2018, 6(1): 011007.
[29] YU Chaohui, WANG Jindong, CHEN Yiqiang, et al. Transfer learning with dynamic adversarial adaptation network[C]//2019 IEEE International Conference on Data Mining. Beijing: IEEE, 2019: 778?786.
[30] ZHU Yongchun, ZHUANG Fuzhen, WANG Jindong, et al. Deep subdomain adaptation network for image classification[J]. IEEE transactions on neural networks and learning systems, 2021, 32(4): 1713–1722.
[31] CHEN Chao, CHEN Zhihong, JIANG Boyuan, et al. Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation[J]. Proceedings of the AAAI conference on artificial intelligence, 2019, 33: 3296–3303.
[32] YE Mang, ZHANG Xu, YUEN P C, et al. Unsupervised embedding learning via invariant and spreading instance feature[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 6203?6212.
[33] HéNAFF O J, SRINIVAS A, FAUW J, et al. Data-efficient image recognition with contrastive predictive coding[C]// Proceedings of the 37th International Conference on Machine Learning. JMLR, 2020, 119: 4182?4192
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems