[1]郭磊,王骏,丁维昌,等.4D卷积神经网络的自闭症功能磁共振图像分类[J].智能系统学报,2021,16(6):1021-1029.[doi:10.11992/tis.202009022]
GUO Lei,WANG Jun,DING Weichang,et al.Classification of the functional magnetic resonance image of autism based on 4D convolutional neural network[J].CAAI Transactions on Intelligent Systems,2021,16(6):1021-1029.[doi:10.11992/tis.202009022]
点击复制
GUO Lei,WANG Jun,DING Weichang,et al.Classification of the functional magnetic resonance image of autism based on 4D convolutional neural network[J].CAAI Transactions on Intelligent Systems,2021,16(6):1021-1029.[doi:10.11992/tis.202009022]
4D卷积神经网络的自闭症功能磁共振图像分类
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第6期
页码:
1021-1029
栏目:
学术论文—机器学习
出版日期:
2021-11-05
- Title:
- Classification of the functional magnetic resonance image of autism based on 4D convolutional neural network
- Keywords:
- deep learning; convolutional neural network; autism; 4D convolution; functional magnetic resonance imaging; feature extraction; feature fusion; image classification
- 分类号:
- TP391
- 摘要:
- 静息态功能磁共振图像是随着时间变化的一系列三维图像。已有的3D卷积过程本质上是对三维图像数据或二维图像+时间维数据进行处理，无法有效地融合静息态功能磁共振图像的时间轴信息。为此，本文提出了新型的4D卷积神经网络识别模型。具体而言，通过对输入的fMRI使用四维卷积核执行四维卷积，在自闭症患者的功能磁共振图像中，从空间和时间上提取特征，从而捕获图像在时间序列上的变化信息。所开发的模型从输入图像中生成多个信息通道，最终的特征表示结合了所有通道的信息。实验结果表明，在保证模型泛化性能的前提下，该方法融合了功能像的全局信息，并且采集了功能像随时间变化的趋势信息，进而解决了用卷积神经网络处理三维图像随时间变化的分类问题。
- Abstract:
- Resting-state functional magnetic resonance images are a series of three-dimensional (3D) images that change over time. The existing 3D convolution processes 3D image data or two-dimensional image and time-dimensional data, but it cannot effectively fuse the time axis information of a resting-state functional magnetic resonance image. To resolve this, a new four-dimensional (4D) convolutional neural network (CNN) recognition model is proposed in this paper. Specifically, by performing a 4D convolution using a 4D convolution kernel on the input functional magnetic resonance imaging, features are spatially and temporally extracted from the functional magnetic resonance image of a patient with autism, thereby capturing information about the changes in the image’s time series. The developed model generates multiple information channels from the input image, and the final feature representation combines information from all channels. The experimental results show that to ensure the generalization performance of the model, the method fuses the global information of the functional image and collects its trend information over time, consequently solving the classification problem of 3D image changes with time using a CNN.
- 参考文献/References:
-
[1] 张芬, 王穗苹, 杨娟华, 等. 自闭症谱系障碍者异常的大脑功能连接[J]. 心理科学进展, 2015, 23(7): 1196-1204
ZHANG Fen, WANG Suiping, YANG Juanhua, et al. Atypical brain functional connectivity in autism spectrum disorders[J]. Advances in psychological science, 2015, 23(7): 1196-1204
[2] CABALLERO-GAUDES C, REYNOLDS R C. Methods for cleaning the BOLD fMRI signal[J]. Neuroimage, 2017, 154: 128-149.
[3] MOELLER S, YACOUB E, OLMAN C A, et al. Multiband multislice GE-EPI at 7 tesla, with 16-fold acceleration using partial parallel imaging with application to high spatial and temporal whole-brain FMRI[J]. Magnetic resonance in medicine, 2010, 63(5): 1144-1153.
[4] 常春云. 基于静息态功能磁共振成像的自闭症预测研究[D]. 北京: 北京交通大学, 2017.
CHANG Chunyun. Research on prediction of autism spectrum disorder based on resting-state fMRI[D]. Beijing: Beijing Jiaotong University, 2017.
[5] DVORNEK N C, VENTOLA P, PELPHREY K A, et al. Identifying autism from resting-state fMRI using long short-term memory networks[M]//WANG Qian, SHI Yinghuan, SUK H I, et al. Machine Learning in Medical Imaging. Cham: Springer, 2017: 362-370.
[6] ZHAO Yu, DONG Qinglin, ZHANG Shu, et al. Automatic recognition of fMRI-derived functional networks using 3-D convolutional neural networks[J]. IEEE transactions on biomedical engineering, 2018, 65(9): 1975-1984.
[7] LI Xiaoxiao, DVORNEK N C, PAPADEMETRIS X, et al. 2-Channel convolutional 3D deep neural network (2CC3D) for fMRI analysis: ASD classification and feature learning[C]//2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). Washington, USA, 2018: 1252-1255.
[8] KHOSLA M, JAMISON K, KUCEYESKI A, et al. Ensemble learning with 3D convolutional neural networks for functional connectome-based prediction[J]. NeuroImage, 2019, 199: 651-662.
[9] LU Zhou, PU Hongming, WANG Feicheng, et al. The expressive power of neural networks: a view from the width[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA, 2017: 6232-6240.
[10] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, 2012: 1097-1105.
[11] JI Shuiwang, XU Wei, YANG Ming, et al. 3D Convolutional neural networks for human action recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 35(1): 221-231.
[12] DIBA A, FAYYAZ M, SHARMA V, et al. Temporal 3D convnets: new architecture and transfer learning for video classification [EB/OL].(2017-11-22)[2020-9-20]https:// arxiv. org/ abs/1711.08200.
[13] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[14] HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors[J]. Computer science, 2012, 3(4): 212-223.
[15] SIMONYAN K, ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[EB/OL]// (2018-8-16)[2019-4-26]https://arxiv.org/abs/1808.05377v1.
[16] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA, 2014: 1-9.
[17] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning. Lille, France, 2015: 448-456.
[18] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA, 2016: 2818-2826.
[19] SZEGEDY C, IOFFE S, VANHOUCKE V, et al. Inception-v4, inception-ResNet and the impact of residual connections on learning[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. San Francisco, USA, 2017: 4278-4284.
[20] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA, 2016: 770-778.
[21] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA, 2017: 2261-2269.
[22] EL GAZZAR A, CERLIANI L, VAN WINGEN G, et al. Simple 1-D convolutional networks for resting-state fMRI based classification in autism[C]//2019 International Joint Conference on Neural Networks (IJCNN). Budapest, Hungary, 2019: 1-6.
[23] YAN Chaogan, WANG Xindi, ZUO Xinian, et al. DPABI: data processing & analysis for (resting-state) brain imaging[J]. Neuroinformatics, 2016, 14(3): 339-351.
[24] HECHT-NIELSEN. Theory of the backpropagation neural network[C]//International 1989 Joint Conference on Neural Networks. Washington, USA, 1989: 593-605.
[25] ELSKEN T, METZEN J H, HUTTER F . Neural architecture search: A survey[EB/OL]. (2018-8-16)[2020-9-20]https://arxiv.org/abs/1808.05377v1.
[26] SALVADOR R, SUCKLING J, COLEMAN M R, et al. Neurophysiological architecture of functional magnetic resonance images of human brain[J]. Cerebral cortex, 2005, 15(9): 1332-1342.
- 相似文献/References:
-
[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]殷瑞,苏松志,李绍滋.一种卷积神经网络的图像矩正则化策略[J].智能系统学报,2016,11(1):43.[doi:10.11992/tis.201509018]
YIN Rui,SU Songzhi,LI Shaozi.Convolutional neural network’s image moment regularizing strategy[J].CAAI Transactions on Intelligent Systems,2016,11():43.[doi:10.11992/tis.201509018]
[4]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[5]龚震霆,陈光喜,任夏荔,等.基于卷积神经网络和哈希编码的图像检索方法[J].智能系统学报,2016,11(3):391.[doi:10.11992/tis.201603028]
GONG Zhenting,CHEN Guangxi,REN Xiali,et al.An image retrieval method based on a convolutional neural network and hash coding[J].CAAI Transactions on Intelligent Systems,2016,11():391.[doi:10.11992/tis.201603028]
[6]师亚亭,李卫军,宁欣,等.基于嘴巴状态约束的人脸特征点定位算法[J].智能系统学报,2016,11(5):578.[doi:10.11992/tis.201602006]
SHI Yating,LI Weijun,NING Xin,et al.A facial feature point locating algorithmbased on mouth-state constraints[J].CAAI Transactions on Intelligent Systems,2016,11():578.[doi:10.11992/tis.201602006]
[7]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[8]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[9]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
[10]杨晓兰,强彦,赵涓涓,等.基于医学征象和卷积神经网络的肺结节CT图像哈希检索[J].智能系统学报,2017,12(6):857.[doi:10.11992/tis.201706035]
YANG Xiaolan,QIANG Yan,ZHAO Juanjuan,et al.Hashing retrieval for CT images of pulmonary nodules based on medical signs and convolutional neural networks[J].CAAI Transactions on Intelligent Systems,2017,12():857.[doi:10.11992/tis.201706035]
[11]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[12]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[13]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
[14]莫凌飞,蒋红亮,李煊鹏.基于深度学习的视频预测研究综述[J].智能系统学报,2018,13(1):85.[doi:10.11992/tis.201707032]
MO Lingfei,JIANG Hongliang,LI Xuanpeng.Review of deep learning-based video prediction[J].CAAI Transactions on Intelligent Systems,2018,13():85.[doi:10.11992/tis.201707032]
[15]葛园园,许有疆,赵帅,等.自动驾驶场景下小且密集的交通标志检测[J].智能系统学报,2018,13(3):366.[doi:10.11992/tis.201706040]
GE Yuanyuan,XU Youjiang,ZHAO Shuai,et al.Detection of small and dense traffic signs in self-driving scenarios[J].CAAI Transactions on Intelligent Systems,2018,13():366.[doi:10.11992/tis.201706040]
[16]汪鸿翔,柳培忠,骆炎民,等.高斯核函数卷积神经网络跟踪算法[J].智能系统学报,2018,13(3):388.[doi:10.11992/tis.201612040]
WANG Hongxiang,LIU Peizhong,LUO Yanmin,et al.Convolutional neutral network tracking algorithm accelerated by Gaussian kernel function[J].CAAI Transactions on Intelligent Systems,2018,13():388.[doi:10.11992/tis.201612040]
[17]冯小荣,惠康华,柳振东.基于卷积特征和贝叶斯分类器的人脸识别[J].智能系统学报,2018,13(5):769.[doi:10.11992/tis.201706052]
FENG Xiaorong,HUI Kanghua,LIU Zhendong.Face recognition based on convolution feature and Bayes classifier[J].CAAI Transactions on Intelligent Systems,2018,13():769.[doi:10.11992/tis.201706052]
[18]莫宏伟,汪海波.基于Faster R-CNN的人体行为检测研究[J].智能系统学报,2018,13(6):967.[doi:10.11992/tis.201801025]
MO Hongwei,WANG Haibo.Research on human behavior detection based on Faster R-CNN[J].CAAI Transactions on Intelligent Systems,2018,13():967.[doi:10.11992/tis.201801025]
[19]胡越,罗东阳,花奎,等.关于深度学习的综述与讨论[J].智能系统学报,2019,14(1):1.[doi:10.11992/tis.201808019]
HU Yue,LUO Dongyang,HUA Kui,et al.Overview on deep learning[J].CAAI Transactions on Intelligent Systems,2019,14():1.[doi:10.11992/tis.201808019]
[20]孙必慎,石武祯,姜峰.计算视觉核心问题：自然图像先验建模研究综述[J].智能系统学报,2019,14(1):71.[doi:10.11992/tis.201804019]
SUN Bishen,SHI Wuzhen,JIANG Feng.Core problem in computer vision: survey of natural image prior models[J].CAAI Transactions on Intelligent Systems,2019,14():71.[doi:10.11992/tis.201804019]
备注/Memo
收稿日期:2020-09-16。
基金项目:江苏省自然科学基金项目(BK20181339)
作者简介:郭磊，硕士研究生，主要研究方向为深度习与医学图像处理;王骏，副教授，博士，主要研究方向为机器学习、模糊系统、医学影像分析。主持国家自然科学基金项目1项，江苏省自然科学基金项目1项。2016年获江苏省高校科研成果自然科学一等奖。获得国家发明专利5项，发表学术论文50余篇;王士同，教授，博士，主要研究方向为模式识别、人工智能。曾获教育部、中船总公司、湖南省等省部级科技进步奖10项。获国务院政府特殊津贴，省部级有突出贡献的中青年专家。发表学术论文百余篇.
通讯作者:王骏.E-mail:wangjun_shu@shu.edu.cn
更新日期/Last Update:
2021-12-25