[1]顿家乐,王骏,彭汉琛,等.面向自闭症辅助诊断的知识蒸馏混合域适应方法[J].智能系统学报,2025,20(1):81-90.[doi:10.11992/tis.202403030]
DUN Jiale,WANG Jun,PENG Hanchen,et al.Blended domain adaptation for computer-aided diagnosis of autism through knowledge distillation[J].CAAI Transactions on Intelligent Systems,2025,20(1):81-90.[doi:10.11992/tis.202403030]
点击复制
DUN Jiale,WANG Jun,PENG Hanchen,et al.Blended domain adaptation for computer-aided diagnosis of autism through knowledge distillation[J].CAAI Transactions on Intelligent Systems,2025,20(1):81-90.[doi:10.11992/tis.202403030]
面向自闭症辅助诊断的知识蒸馏混合域适应方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
20
期数:
2025年第1期
页码:
81-90
栏目:
学术论文—机器学习
出版日期:
2025-01-05
- Title:
- Blended domain adaptation for computer-aided diagnosis of autism through knowledge distillation
- Keywords:
- autism spectrum disorder; domain adaptation; blended target domain; knowledge distillation; graph convolutional network; teacher network; student network; adversarial learning
- 分类号:
- TP391
- 摘要:
- 使用领域自适应方法构建自闭症辅助诊断模型时，通常会面临目标域中混合了来自多个影像中心的样本的情况（即混合目标域），这使得目标域中包含了多个分布。传统领域自适应方法只能处理目标域包含单一分布的情况，而无法直接处理混合目标域的情况。为此，本文提出了一种基于知识蒸馏的混合目标领域自适应模型。具体地，将图卷积网络（graph convolutional network，GCN）作为教师模型，多层感知机（multilayer perceptron，MLP）作为学生模型。针对混合目标域数据分布的多样性，提出了一种新型的对抗知识蒸馏机制，通过对抗训练特征提取器和域鉴别器来减少源域和目标域之间的分布差异；与此同时，使用知识蒸馏，使教师模型在领域自适应的同时将知识传递给学生模型。在ABIDE数据集上验证了算法的有效性，本文方法一方面有效降低了网络的复杂度，另一方面，在混合目标域的分类准确率达到69.17%，与其他领域自适应方法相比效果更好。
- Abstract:
- In the modeling of computer-aided diagnosis for autism spectrum disorder (ASD) across multiple centers with domain adaptation methods, unlabeled samples from multiple imaging centers are blended together in the target domain. Traditional domain adaptation methods lack the capability to address the clinical scenario of identifying ASD in blended-target domains. To this end, we propose a knowledge distillation blended-target domain adaptation model. Specifically, the graph convolutional network (GCN) is used as the teacher model and the multilayer perceptron (MLP) is used as the student model. To address distribution differences between source and target domains, a novel adversarial knowledge distillation mechanism is proposed to reduce the distribution difference by adversarially training feature extractors and domain discriminators. At the same time, knowledge distillation is used to enable the teacher model to transfer knowledge to the student model while achieving domain adaptation. The ABIDE dataset is employed to validate the effectiveness of the model. Our method not only reduces the complexity of the network but also achieves a classification accuracy of 69.17% in the blended target domains, surpassing other domain adaptation methods.
- 参考文献/References:
-
[1] LORD C, ELSABBAGH M, BAIRD G, et al. Autism spectrum disorder[J]. Lancet, 2018, 392(10146): 508-520.
[2] HEINSFELD A S, FRANCO A R, CRADDOCK R C, et al. Identification of autism spectrum disorder using deep learning and the ABIDE dataset[J]. NeuroImage: clinical, 2018, 17: 16-23.
[3] LIU Xingdan, WU Jiacheng, LI Wenqi, et al. Domain adaptation via low rank and class discriminative representation for autism spectrum disorder identification: a multi-site fMRI study[J]. IEEE transactions on neural systems and rehabilitation engineering, 2023: 3233656.
[4] WANG Mingliang, ZHANG Daoqiang, HUANG Jiashuang, et al. Identifying autism spectrum disorder with multi-site fMRI via low-rank domain adaptation[J]. IEEE transactions on medical imaging, 2020, 39(3): 644-655.
[5] ABRAHAM A, MILHAM M P, DI MARTINO A, et al. Deriving reproducible biomarkers from multi-site resting-state data: an autism-based example[J]. NeuroImage, 2017, 147: 736-745.
[6] 蔡鸿顺, 张琼敏, 龙颖. 面向阿尔茨海默症辅助诊断的多尺度域适应网络[J]. 智能系统学报, 2023, 18(5): 1090-1098.
CAI Hongshun, ZHANG Qiongmin, LONG Ying. Multiscale domain adaptation network for the auxiliary diagnosis of Alzheimer’s disease[J]. CAAI transactions on intelligent systems, 2023, 18(5): 1090-1098.
[7] 梁艳, 温兴, 潘家辉. 融合全局与局部特征的跨数据集表情识别方法[J]. 智能系统学报, 2023, 18(6): 1205-1212.
LIANG Yan, WEN Xing, PAN Jiahui. Cross-dataset facial expression recognition method fusing global and local features[J]. CAAI transactions on intelligent systems, 2023, 18(6): 1205-1212.
[8] MANSOUR Y, MOHRI M, ROSTAMIZADEH A. Domain adaptation: learning bounds and algorithms[EB/OL]. (2023-11-30)[2024-03-13]. https://arxiv.org/abs/0902.3430v3.
[9] BEN-DAVID S, BLITZER J, CRAMMER K, et al. A theory of learning from different domains[J]. Machine learning, 2010, 79(1): 151-175.
[10] LONG Mingsheng, CAO Yue, WANG Jianmin, et al. Learning transferable features with deep adaptation networks[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille: ACM, 2015: 97-105.
[11] DING Jie, WANG Li, YU Lei, et al. Low-rank domain adaptive method with inter-class difference constraint for multi-site autism spectrum disorder identification[C]//2022 7th International Conference on Computational Intelligence and Applications. Nanjing: IEEE, 2022: 237-242.
[12] CHEN Ziliang, ZHUANG Jingyu, LIANG Xiaodan, et al. Blending-target domain adaptation by adversarial meta-adaptation networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 2243-2252.
[13] LIU Ziwei, MIAO Zhongqi, PAN Xingang, et al. Open compound domain adaptation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12406-12415.
[14] ZHENG J, WU Wenzhao, FU Haohuan, et al. Unsupervised mixed multi-target domain adaptation for remote sensing images classification[C]//2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa: IEEE, 2020: 1381-1384.
[15] XU Pengcheng, WANG Boyu, LING C. Class overwhelms: mutual conditional blended-target domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2023, 37(3): 3036-3044.
[16] ROY S, KRIVOSHEEV E, ZHONG Zhun, et al. Curriculum graph co-teaching for multi-target domain adaptation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 5347-5356.
[17] PARISOT S, KTENA S I, FERRANTE E, et al. Disease prediction using graph convolutional networks: application to autism spectrum disorder and alzheimer’s disease[J]. Medical image analysis, 2018, 48: 117-130.
[18] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[EB/OL]. (2015-05-09)[2024-03-13]. https://arxiv.org/abs/1503.02531v1.
[19] 黄震华, 杨顺志, 林威, 等. 知识蒸馏研究综述[J]. 计算机学报, 2022, 45(3): 624-653.
HUANG Zhenhua, YANG Shunzhi, LIN Wei, et al. Knowledge distillation: a survey[J]. Chinese journal of computers, 2022, 45(3): 624-653.
[20] CHEN Defang, MEI Jianping, WANG Can, et al. Online knowledge distillation with diverse peers[C]//Proceedings of the AAAI conference on artificial intelligence. New York: AAAI, 2020, 34(4): 3430-3437.
[21] WILSON G, COOK D J. A survey of unsupervised deep domain adaptation[J]. ACM transactions on intelligent systems and technology, 2020, 11(5): 1-46.
[22] LUO Yadan, WANG Zijian, HUANG Zi, et al. Progressive graph learning for open-set domain adaptation[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ACM, 2020: 6468-6478.
[23] LONG Mingsheng, CAO Zhangjie, WANG Jianmin, et al. Conditional adversarial domain adaptation[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal: ACM, 2018: 1647-1657.
[24] DI MARTINO A, YAN C G, LI Q, et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism[J]. Molecular psychiatry, 2014, 19(6): 659-667.
[25] CAMERON C, SHARAD S, BRIAN C, et al. Towards automated analysis of connectomes: the configurable pipeline for the analysis of connectomes (C-PAC)[J]. Frontiers in neuroinformatics, 2013, 7: 42.
[26] RYU M, LEE G, LEE K. Knowledge distillation for BERT unsupervised domain adaptation[J]. Knowledge and information systems, 2022, 64(11): 3113-3128.
[27] MURPHY K, SCH?LKOPF B, GANIN Y, et al. Domain-adversarial training of neural networks[J]. Journal of machine learning research, 2016, 17(1): 2096-2030.
[28] LONG Mingsheng, ZHU Han, WANG Jianmin, et al. Deep transfer learning with joint adaptation networks[C]//Proceedings of the 34th International Conference on Machine Learning. Sydney: ACM, 2017: 2208-2217.
[29] ZELLINGER W, MOSER B A, GRUBINGER T, et al. Robust unsupervised domain adaptation for neural networks via moment alignment[J]. Information sciences, 2019, 483: 174-191.
[30] NGUYEN-MEIDINE L T, BELAL A, KIRAN M, et al. Unsupervised multi-target domain adaptation through knowledge distillation[C]//2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2021: 1338-1346.
- 相似文献/References:
-
[1]许敏,俞林.一种新颖的领域自适应概率密度估计器[J].智能系统学报,2015,10(2):221.[doi:10.3969/j.issn.1673-4785.201312041]
XU Min,YU Lin.A probability density estimator for domain adaptation[J].CAAI Transactions on Intelligent Systems,2015,10():221.[doi:10.3969/j.issn.1673-4785.201312041]
[2]梁艳,温兴,潘家辉.融合全局与局部特征的跨数据集表情识别方法[J].智能系统学报,2023,18(6):1205.[doi:10.11992/tis.202212030]
LIANG Yan,WEN Xing,PAN Jiahui.Cross-dataset facial expression recognition method fusing global and local features[J].CAAI Transactions on Intelligent Systems,2023,18():1205.[doi:10.11992/tis.202212030]
备注/Memo
收稿日期:2024-3-18。
基金项目:国家自然科学基金项目（62272289）.
作者简介:顿家乐，硕士研究生，主要研究方向为深度学习、计算机视觉、迁移学习。E-mail: dunjiale1997@163.com。;王骏，副教授，博士。中国计算机学会高级会员，IEEE高级会员，中国人工智能学会粒计算与知识发现专业委员会委员、机器学习专业委员会委员，MICS online委员。主要研究方向为机器学习、医学影像智能计算。发表学术论文70余篇。E-mail：wangjun_ shu@shu.edu.cn。;彭汉琛，硕士研究生，主要研究方向为深度学习、迁移学习、图像处理。E-mail: phcking0219@163.com。
通讯作者:王骏. E-mail：wangjun_shu@shu.edu.cn
更新日期/Last Update:
2025-01-05