[1]刘万军,姜岚,曲海成,等.融合CNN与Transformer的MRI脑肿瘤图像分割[J].智能系统学报,2024,19(4):1007-1015.[doi:10.11992/tis.202301016]
LIU Wanjun,JIANG Lan,QU Haicheng,et al.MRI brain tumor image segmentation by fusing CNN and Transformer[J].CAAI Transactions on Intelligent Systems,2024,19(4):1007-1015.[doi:10.11992/tis.202301016]
点击复制
LIU Wanjun,JIANG Lan,QU Haicheng,et al.MRI brain tumor image segmentation by fusing CNN and Transformer[J].CAAI Transactions on Intelligent Systems,2024,19(4):1007-1015.[doi:10.11992/tis.202301016]
融合CNN与Transformer的MRI脑肿瘤图像分割
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第4期
页码:
1007-1015
栏目:
学术论文—人工智能基础
出版日期:
2024-07-05
- Title:
- MRI brain tumor image segmentation by fusing CNN and Transformer
- Keywords:
- medical image segmentation; brain tumor; cascaded neural network; convolutional neural networks; Transformer; feature fusion; multiple attention; residual learning
- 分类号:
- TP391
- 摘要:
- 为解决卷积神经网络(convolutional neural network, CNN)在学习全局上下文信息和边缘细节方面受到很大限制的问题，提出一种同时学习局语义信息和局部空间细节的级联神经网络用于脑肿瘤医学图像分割。首先将输入体素分别送入CNN和Transformer分支，在编码阶段结束后，采用一种双分支融合模块将2个分支学习到的特征有效地结合起来以实现全局信息与局部信息的融合。双分支融合模块利用哈达玛积对双分支特征之间的细粒度交互进行建模，同时使用多重注意力机制充分提取特征图通道和空间信息并抑制无效的噪声信息。在BraTS竞赛官网评估了本文方法，在BraTS2019验证集上增强型肿瘤区、全肿瘤区和肿瘤核心区的Dice分数分别为77.92%，89.20%和81.20%。相较于其他先进的三维医学图像分割方法，本文方法表现出了更好的分割性能，为临床医生做出准确的脑肿瘤细胞评估和治疗方案提供了可靠依据。
- Abstract:
- This study presents a cascaded neural network that learns both global semantic information and local spatial details for medical image segmentation of brain tumors, solving the problem that convolutional neural networks(CNN) are greatly restricted in learning global contextual information and edge details. First, the input voxels are fed into the CNN and Transformer branches separately. After the encoding phase, a two-branch fusion module is used to effectively combine the features learned in both branches to achieve the fusion of global and local information. The two-branch fusion module uses Hadamard products to model the fine-grained interactions between the two-branch features, while using multiple attention mechanisms to fully extract the feature map channels and spatial information and suppress the invalid noise information. The method of this paper has been evaluated on the BraTS competition website, with Dice scores of 77.92%, 89.20% and 81.20% for the enhanced tumor region, full tumor region and tumor core region on the BraTS2019 validation set, respectively. Compared with other advanced 3D medical image segmentation methods, this method shows better segmentation performance, which provides a reliable basis for clinicians to make accurate brain tumor cell assessment and treatment plans.
- 参考文献/References:
-
[1] OSTROM Q T, BAUCHET L, DAVIS F G, et al. The epidemiology of glioma in adults: a “state of the science” review[J]. Neuro-oncology, 2014, 16(7): 896–913.
[2] MENZE B H, JAKAB A, BAUER S, et al. The multimodal brain tumor image segmentation benchmark (BRATS)[J]. IEEE transactions on medical imaging, 2014, 34(10): 1993–2024.
[3] HUANG Huimin, LIN Lanfen, TONG Ruofeng, et al. Unet 3+: a full-scale connected unet for medical image segmentation[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 1055-1059.
[4] NGUYEN T, HUA B S, LE N. 3D-UCaps: 3D capsules unet for volumetric image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021: 548-558.
[5] LOU A, GUAN S, LOEW M. DC-UNet: rethinking the U-Net architecture with dual channel efficient CNN for medical image segmentation[C]//Medical Imaging 2021: Image Processing. Virtual Online: SPIE, 2021: 758-768.
[6] ?I?EK ?, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention. Cham: Springer, 2016: 424-432.
[7] MYRONENKO A. 3D MRI brain tumor segmentation using autoencoder regularization[C]//International MICCAI Brainlesion Workshop. Cham: Springer, 2019: 311-320.
[8] CHEN Chen, LIU Xiaopeng, DING Meng, et al. 3D dilated multi-fiber network for real-time brain tumor segmentation in MRI[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2019: 184-192.
[9] 张睦卿, 韩雨童, 陈柏年, 等. 基于多尺度伪影生成对抗网络的磁共振成像脑肿瘤分割方法[J]. 光子学报, 2023, 52(8): 194–205
ZHANG Muqing, HAN Yutong, CHEN Bonian, et al. Magnetic resonance imaging brain tumor segmentation using multiscale ghost generative adversarial network[J]. Acta photonica sinica, 2023, 52(8): 194–205
[10] VALANARASU J M J, SINDAGI V A, HACIHALILOGLU I, et al. KiU-Net: overcomplete convolutional architectures for biomedical image and volumetric segmentation[J]. IEEE transactions on medical imaging, 2021, 41(4): 965–976.
[11] ISENSEE F, J?GER P F, FULL P M, et al. NNU-Net for brain tumor segmentation[C]//International MICCAI Brainlesion Workshop. Cham: Springer, 2021: 118-132.
[12] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale [EB/OL]. (2021-06-03)[2023-01-16]. https:// arxiv.org/pdf/2010.11929.pdf.
[13] HATAMIZADEH A, TANG Yuchen, NATH V, et al. Unetr: transformers for 3d medical image segmentation[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE. 2022: 574-584.
[14] VALANARASU J M J, OZA P, HACIHALILOGLU I, et al. Medical transformer: gated axial-attention for medical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021: 36-46.
[15] JIANG Yun, ZHANG Yuan, LIN Xin, et al. SwinBTS: a method for 3D multimodal brain tumor segmentation using swin transformer[J]. Brain sciences, 2022, 12(6): 797.
[16] CAO Hu, WANG Yueyue, CHEN J, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation[EB/OL]. (2021-05-12)[2023-01-16]. https://arxiv.org/pdf/2105.05537.pdf.
[17] LIN Ailiang, CHEN Bingzhi, XU Jiayu, et al. Ds-transunet: dual swin transformer u-net for medical image segmentation[J]. IEEE transactions on instrumentation and measurement, 2022, 71: 1–15.
[18] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The journal of machine learning research, 2014, 15(1): 1929–1958.
[19] HENDRYCKS D, GIMPEL K. Gaussian error linear units (GELUs)[EB/OL]. (2020-07-08)[2013-01-16]. https://arxiv.org/pdf/1606.08415v4.pdf.
[20] WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11534-11542.
[21] MILLETARI F, NAVAB N, AHMADI S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]//2016 Fourth International Conference on 3D Vision. Stanford: IEEE, 2016: 565-571.
[22] WANG Guotai, LI Wenqi, OURSELIN S, et al. Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation[C]//International MICCAI Brainlesion Workshop. Cham: Springer, 2019: 61-72.
[23] OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention u-net: Learning where to look for the pancreas[EB/OL]. (2018-05-20)[2023-01-16]. https://arxiv.org/pdf/1804.03999.pdf
[24] WANG Wenxuan, CHEN Chen, DING Meng, et al. Transbts: multimodal brain tumor segmentation using transformer[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021: 109-119.
[25] HUANG He, YANG Guang, ZHANG Wenbo, et al. A deep multi-task learning framework for brain tumor segmentation[J]. Frontiers in oncology, 2021, 11: 690244.
[26] VADACCHINO S, MEHTA R, SEPAHVAND N M, et al. HAD-Net: a hierarchical adversarial knowledge distillation network for improved enhanced tumour segmentation without post-contrast images[EB/OL]. (2021-03-30)[2023-01-16]. https://arxiv.org/abs/2103.16617.
[27] LI Sahohua, SUI Xiuchao, LUO Xiangde, et al. Medical image segmentation using squeeze-and-expansion transformers[C]//International Joint Conference on Artificial Intelligence. Montreal: IJCAI, 2019: 61-72.
[28] XU Weijin, YANG Huihua, ZHANG Mingying, et al. Brain tumor segmentation with corner attention and high-dimensional perceptual loss[J]. Biomedical signal processing and control, 2022, 73: 103438.
[29] AKBAR A S, FATICHAH C, SUCIATI N. Single level UNet3D with multipath residual attention block for brain tumor segmentation[J]. Journal of King Saud University-computer and information sciences, 2022: 3247-3258.
[30] LIANG Junjie, YANG Cihui, ZHONG Jingting, et al. BTSwin-Unet: 3D U-shaped symmetrical swin transformer-based network for brain tumor segmentation with self-supervised pre-training[J]. Neural processing letters, 2022, 55(4): 3695–3713.
- 相似文献/References:
-
[1]程显毅,巩向普.改进的模糊C-均值算法在医学图像分割中的应用[J].智能系统学报,2010,5(1):80.
CHENG Xian-yi,GONG Xiang-pu.An improved fuzzy Cmeans algorithm for segmentation of medical images[J].CAAI Transactions on Intelligent Systems,2010,5():80.
备注/Memo
收稿日期:2023-01-16。
基金项目:辽宁省高等学校基本科研项目(LJKMZ20220699)；辽宁工程技术大学学科创新团队项目(LNTU20T-D-23)
作者简介:刘万军，教授，博士生导师，主要研究方向为模式识别与人工智能、计算机视觉与图像处理。主持国家自然科学基金面上项目等各类科研项目20余项。发表学术论文200余篇。E-mail：liuwanjun@lntu.edu.cn;姜岚，硕士研究生，主要研究方向为计算机视觉与图像处理。E-mail：13562859231@163.com;曲海成，副教授，博士，主要研究方向为遥感大数据智能处理和目标识别与跟踪。主持辽宁省自然科学基金1项、省教育厅面上项目2项。发表学术论文60余篇。E-mail：quhaicheng@lntu.edu.cn
通讯作者:刘万军. E-mail：liuwanjun@lntu.edu.cn
更新日期/Last Update:
1900-01-01