[1]LIU Wanjun,JIANG Lan,QU Haicheng,et al.MRI brain tumor image segmentation by fusing CNN and Transformer[J].CAAI Transactions on Intelligent Systems,2024,19(4):1007-1015.[doi:10.11992/tis.202301016]
Copy

MRI brain tumor image segmentation by fusing CNN and Transformer

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 4 Page number: 1007-1015 Column: 学术论文—人工智能基础 Public date: 2024-07-05
Title:
MRI brain tumor image segmentation by fusing CNN and Transformer
Author(s):
LIU Wanjun;  JIANG Lan;  QU Haicheng;  WANG Xiaona;  CUI Heng
School of Software, Liaoning Technical University, Huludao 125105, China
Keywords:
medical image segmentation; brain tumor; cascaded neural network; convolutional neural networks; Transformer; feature fusion; multiple attention; residual learning
CLC:
TP391
DOI:
10.11992/tis.202301016
Abstract:
This study presents a cascaded neural network that learns both global semantic information and local spatial details for medical image segmentation of brain tumors, solving the problem that convolutional neural networks(CNN) are greatly restricted in learning global contextual information and edge details. First, the input voxels are fed into the CNN and Transformer branches separately. After the encoding phase, a two-branch fusion module is used to effectively combine the features learned in both branches to achieve the fusion of global and local information. The two-branch fusion module uses Hadamard products to model the fine-grained interactions between the two-branch features, while using multiple attention mechanisms to fully extract the feature map channels and spatial information and suppress the invalid noise information. The method of this paper has been evaluated on the BraTS competition website, with Dice scores of 77.92%, 89.20% and 81.20% for the enhanced tumor region, full tumor region and tumor core region on the BraTS2019 validation set, respectively. Compared with other advanced 3D medical image segmentation methods, this method shows better segmentation performance, which provides a reliable basis for clinicians to make accurate brain tumor cell assessment and treatment plans.
References:
[1] OSTROM Q T, BAUCHET L, DAVIS F G, et al. The epidemiology of glioma in adults: a “state of the science” review[J]. Neuro-oncology, 2014, 16(7): 896–913.
[2] MENZE B H, JAKAB A, BAUER S, et al. The multimodal brain tumor image segmentation benchmark (BRATS)[J]. IEEE transactions on medical imaging, 2014, 34(10): 1993–2024.
[3] HUANG Huimin, LIN Lanfen, TONG Ruofeng, et al. Unet 3+: a full-scale connected unet for medical image segmentation[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 1055-1059.
[4] NGUYEN T, HUA B S, LE N. 3D-UCaps: 3D capsules unet for volumetric image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021: 548-558.
[5] LOU A, GUAN S, LOEW M. DC-UNet: rethinking the U-Net architecture with dual channel efficient CNN for medical image segmentation[C]//Medical Imaging 2021: Image Processing. Virtual Online: SPIE, 2021: 758-768.
[6] ?I?EK ?, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention. Cham: Springer, 2016: 424-432.
[7] MYRONENKO A. 3D MRI brain tumor segmentation using autoencoder regularization[C]//International MICCAI Brainlesion Workshop. Cham: Springer, 2019: 311-320.
[8] CHEN Chen, LIU Xiaopeng, DING Meng, et al. 3D dilated multi-fiber network for real-time brain tumor segmentation in MRI[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2019: 184-192.
[9] 张睦卿, 韩雨童, 陈柏年, 等. 基于多尺度伪影生成对抗网络的磁共振成像脑肿瘤分割方法[J]. 光子学报, 2023, 52(8): 194–205
ZHANG Muqing, HAN Yutong, CHEN Bonian, et al. Magnetic resonance imaging brain tumor segmentation using multiscale ghost generative adversarial network[J]. Acta photonica sinica, 2023, 52(8): 194–205
[10] VALANARASU J M J, SINDAGI V A, HACIHALILOGLU I, et al. KiU-Net: overcomplete convolutional architectures for biomedical image and volumetric segmentation[J]. IEEE transactions on medical imaging, 2021, 41(4): 965–976.
[11] ISENSEE F, J?GER P F, FULL P M, et al. NNU-Net for brain tumor segmentation[C]//International MICCAI Brainlesion Workshop. Cham: Springer, 2021: 118-132.
[12] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale [EB/OL]. (2021-06-03)[2023-01-16]. https:// arxiv.org/pdf/2010.11929.pdf.
[13] HATAMIZADEH A, TANG Yuchen, NATH V, et al. Unetr: transformers for 3d medical image segmentation[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE. 2022: 574-584.
[14] VALANARASU J M J, OZA P, HACIHALILOGLU I, et al. Medical transformer: gated axial-attention for medical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021: 36-46.
[15] JIANG Yun, ZHANG Yuan, LIN Xin, et al. SwinBTS: a method for 3D multimodal brain tumor segmentation using swin transformer[J]. Brain sciences, 2022, 12(6): 797.
[16] CAO Hu, WANG Yueyue, CHEN J, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation[EB/OL]. (2021-05-12)[2023-01-16]. https://arxiv.org/pdf/2105.05537.pdf.
[17] LIN Ailiang, CHEN Bingzhi, XU Jiayu, et al. Ds-transunet: dual swin transformer u-net for medical image segmentation[J]. IEEE transactions on instrumentation and measurement, 2022, 71: 1–15.
[18] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The journal of machine learning research, 2014, 15(1): 1929–1958.
[19] HENDRYCKS D, GIMPEL K. Gaussian error linear units (GELUs)[EB/OL]. (2020-07-08)[2013-01-16]. https://arxiv.org/pdf/1606.08415v4.pdf.
[20] WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11534-11542.
[21] MILLETARI F, NAVAB N, AHMADI S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]//2016 Fourth International Conference on 3D Vision. Stanford: IEEE, 2016: 565-571.
[22] WANG Guotai, LI Wenqi, OURSELIN S, et al. Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation[C]//International MICCAI Brainlesion Workshop. Cham: Springer, 2019: 61-72.
[23] OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention u-net: Learning where to look for the pancreas[EB/OL]. (2018-05-20)[2023-01-16]. https://arxiv.org/pdf/1804.03999.pdf
[24] WANG Wenxuan, CHEN Chen, DING Meng, et al. Transbts: multimodal brain tumor segmentation using transformer[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021: 109-119.
[25] HUANG He, YANG Guang, ZHANG Wenbo, et al. A deep multi-task learning framework for brain tumor segmentation[J]. Frontiers in oncology, 2021, 11: 690244.
[26] VADACCHINO S, MEHTA R, SEPAHVAND N M, et al. HAD-Net: a hierarchical adversarial knowledge distillation network for improved enhanced tumour segmentation without post-contrast images[EB/OL]. (2021-03-30)[2023-01-16]. https://arxiv.org/abs/2103.16617.
[27] LI Sahohua, SUI Xiuchao, LUO Xiangde, et al. Medical image segmentation using squeeze-and-expansion transformers[C]//International Joint Conference on Artificial Intelligence. Montreal: IJCAI, 2019: 61-72.
[28] XU Weijin, YANG Huihua, ZHANG Mingying, et al. Brain tumor segmentation with corner attention and high-dimensional perceptual loss[J]. Biomedical signal processing and control, 2022, 73: 103438.
[29] AKBAR A S, FATICHAH C, SUCIATI N. Single level UNet3D with multipath residual attention block for brain tumor segmentation[J]. Journal of King Saud University-computer and information sciences, 2022: 3247-3258.
[30] LIANG Junjie, YANG Cihui, ZHONG Jingting, et al. BTSwin-Unet: 3D U-shaped symmetrical swin transformer-based network for brain tumor segmentation with self-supervised pre-training[J]. Neural processing letters, 2022, 55(4): 3695–3713.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems