[1]LIU Xia,LYU Zhiwei,WANG Bo,et al.Multi-task method for segmentation and classification of thyroid nodules combined with ultrasound images[J].CAAI Transactions on Intelligent Systems,2023,18(4):764-774.[doi:10.11992/tis.202203063]
Copy
Multi-task method for segmentation and classification of thyroid nodules combined with ultrasound images
CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume:
18
Number of periods:
2023 4
Page number:
764-774
Column:
学术论文—机器感知与模式识别
Public date:
2023-07-15
- Title:
- Multi-task method for segmentation and classification of thyroid nodules combined with ultrasound images
- Keywords:
- deep learning; multi-task learning; ultrasound image of thyroid nodule; image segmentation; image classification; deep layer convolutional block; multiscale convolutional block attention module; residual structure
- CLC:
- TP391
- DOI:
- 10.11992/tis.202203063
- Abstract:
- Aiming at the problems of multi-scale thyroid nodules, blurred nodule edges, and unbalanced classification of benign and malignant thyroid nodules in ultrasound images, this paper proposes a multi-task method for segmentation and classification of thyroid nodules combined with ultrasound. The fully convolutional network is used as the backbone sharing network, and the extracted shallow features are shared to the multi-task branch network. In the branch segmentation networks, deep convolution blocks are added to obtain the deep features of the segmented branches, and then the deep features are up-sampled. An improved multi-scale convolutional attention module is proposed, which combines the up-sampling results with the feature tensor of each feature extraction stage of trunk sharing network after jumping connection with multi-scale convolution attention module, so as to reduce the fuzzy problem of nodule edge blurs and improve the segmentation performance. At the same time, a multi-scale convolutional attention module is integrated into the classification branch to optimize the classification performance. The experimental results show that the multi-task method proposed in this paper can effectively improve the accuracy of segmentation and classification, having better segmentation and classification performance than single-task deep learning network. It can effectively deal with the problem of multi-scale thyroid nodules and blurred nodule edges, and reduce the impact brought by unbalanced classification of benign and malignant.
- References:
-
[1] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[M]. Cham: Springer International Publishing, 2015: 234?241.
[2] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770?778.
[3] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132?7141.
[4] 吴俊霞, 强彦, 王梦南, 等. 基于条件分割对抗网络的超声甲状腺结节分割[J]. 太原理工大学学报, 2023,54(2):3920-398.
WU Junxia, QIANG Yan, WANG Mengnan, et al. Ultrasonic thyroid nodule segmentation based on segmentation adversarial network[J]. Journal of Taiyuan University of Technology, 2023,54(2):3920-398.
[5] 胡屹杉, 秦品乐, 曾建潮, 等. 基于特征融合和动态多尺度空洞卷积的超声甲状腺分割网络[J]. 计算机应用, 2021, 41(3): 891–897
HU Yishan, QIN Pinle, ZENG Jianchao, et al. Ultrasound thyroid segmentation network based on feature fusion and dynamic multi-scale dilated convolution[J]. Journal of computer applications, 2021, 41(3): 891–897
[6] 赵科甫, 张蕾. 基于卷积神经网络的甲状腺结节超声图像分割[J]. 现代计算机, 2021(15): 54–60
ZHAO Kefu, ZHANG Lei. Segmentation of thyroid nodules on ultrasound image using convolution neural network[J]. Modern computer, 2021(15): 54–60
[7] CHU Chen, ZHENG Jihui, ZHOU yong. Ultrasonic thyroid nodule detection method based on U-Net network[J]. Computer methods and programs in biomedicine, 2021, 199: 105906.
[8] OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-net: learning where to look for the pancreas[EB/OL]. (2018?04?11)[2022?03?31].https://arxiv.org/abs/1804.03999.
[9] ZHOU Zongwei, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: A nested U-net architecture for medical image segmentation[M]. Cham: Springer International Publishing, 2018: 3?11.
[10] CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Computer Vision - ECCV 2018: 15th European Conference. New York: ACM, 2018: 833-851.
[11] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(12): 2481–2495.
[12] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2261?2269.
[13] 迟剑宁, 于晓升, 张艺菲. 融合深度网络和浅层纹理特征的甲状腺结节癌变超声图像诊断[J]. 中国图象图形学报, 2018, 23(10): 1582–1593
CHI Jianning, YU Xiaosheng, ZHANG Yifei. Thyroid nodule malignantrisk detection in ultrasound image by fusing deep and texture features[J]. Journal of image and graphics, 2018, 23(10): 1582–1593
[14] 邹奕轩, 周蕾蕾, 赵紫婷, 等. 基于卷积神经网络的甲状腺结节超声图像良恶性分类研究[J]. 中国医学装备, 2020, 17(3): 9–13
ZOU Yixuan, ZHOU Leilei, ZHAO Ziting, et al. Study on the classification of benign and malignant thyroid nodule in ultrasound image on the basis of cnns[J]. China medical equipment, 2020, 17(3): 9–13
[15] MOUSSA O, KHACHNAOUI H, GUETARI R, et al. Thyroid nodules classification and diagnosis in ultrasound images using fine-tuning deep convolutional neural network[J]. International journal of imaging systems and technology, 2020, 30(1): 185–195.
[16] WEI Xi, GAO Ming, YU Ruiguo, et al. Ensemble deep learning model for multicenter classification of thyroid nodules on ultrasound images[J]. Med sci monit, 2020, 26: e926096.
[17] XIE Jiahao, GUO Lehang, ZHAO Chongke, et al. A hybrid deep learning and handcrafted features based approach for thyroid nodule classification in ultrasound images[J]. Journal of physics:conference series, 2020, 1693(1): 012160.
[18] SHI Guohua, WANG Jiawen, QIANG yan, et al. Knowledge-guided synthetic medical image adversarial augmentation for ultrasonography thyroid nodule classification[J]. Computer methods and programs in biomedicine, 2020, 196: 105611.
[19] SHEN Xueda, OUYANG Xi, LIU Tianjiao, et al. Cascaded networks for thyroid nodule diagnosis from ultrasound images[M]. Cham: Springer International Publishing, 2021: 145?154.
[20] AMYAR A, MODZELEWSKI R, LI H, et al. Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia: classification and segmentation[J]. Computers in biology and medicine, 2020, 126: 104037.
[21] KIM S J, KIM H S. Multi-tasking U-net based paprika disease diagnosis[J]. Korean institute of smart media, 2020, 9(1): 16–22.
[22] CIPOLLA R, GAL Y, KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7482?7491.
[23] WU Yuhuan, GAO Shanghua, MEI Jie, et al. JCS: an explainable COVID-19 diagnosis system by joint classification and segmentation[J]. IEEE transactions on image processing:a publication of the IEEE signal processing society, 2021, 30: 3113–3126.
[24] 叶剑锋, 徐轲, 熊峻峰, 等. 基于注意力机制和辅助任务的语义分割算法[J]. 计算机工程, 2021, 47(9): 203–209,216
YE Jianfeng, XU Ke, XIONG Junfeng, et al. Semantic segmentation algorithm based on attention mechanism and auxiliary task[J]. Computer engineering, 2021, 47(9): 203–209,216
[25] LYU Kejie, LI Yingming, ZHANG Zhongfei. Attention-aware multi-task convolutional neural networks[J]. IEEE transactions on image processing, 2020, 29: 1867–1878.
[26] MEHTA S, MERCAN E, BARTLETT J, et al. Y-net: joint segmentation and classification for diagnosis of breast biopsy images[M]. Cham: Springer International Publishing, 2018: 893?901.
- Similar References:
Memo
-
Last Update:
1900-01-01