[1]DUN Jiale,WANG Jun,PENG Hanchen,et al.Blended domain adaptation for computer-aided diagnosis of autism through knowledge distillation[J].CAAI Transactions on Intelligent Systems,2025,20(1):81-90.[doi:10.11992/tis.202403030]
Copy

Blended domain adaptation for computer-aided diagnosis of autism through knowledge distillation

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 1 Page number: 81-90 Column: 学术论文—机器学习 Public date: 2025-01-05
Title:
Blended domain adaptation for computer-aided diagnosis of autism through knowledge distillation
Author(s):
DUN Jiale;  WANG Jun;  PENG Hanchen;  LI Juncheng;  SHI Jun
School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China
Keywords:
autism spectrum disorder; domain adaptation; blended target domain; knowledge distillation; graph convolutional network; teacher network; student network; adversarial learning
CLC:
TP391
DOI:
10.11992/tis.202403030
Abstract:
In the modeling of computer-aided diagnosis for autism spectrum disorder (ASD) across multiple centers with domain adaptation methods, unlabeled samples from multiple imaging centers are blended together in the target domain. Traditional domain adaptation methods lack the capability to address the clinical scenario of identifying ASD in blended-target domains. To this end, we propose a knowledge distillation blended-target domain adaptation model. Specifically, the graph convolutional network (GCN) is used as the teacher model and the multilayer perceptron (MLP) is used as the student model. To address distribution differences between source and target domains, a novel adversarial knowledge distillation mechanism is proposed to reduce the distribution difference by adversarially training feature extractors and domain discriminators. At the same time, knowledge distillation is used to enable the teacher model to transfer knowledge to the student model while achieving domain adaptation. The ABIDE dataset is employed to validate the effectiveness of the model. Our method not only reduces the complexity of the network but also achieves a classification accuracy of 69.17% in the blended target domains, surpassing other domain adaptation methods.
References:
[1] LORD C, ELSABBAGH M, BAIRD G, et al. Autism spectrum disorder[J]. Lancet, 2018, 392(10146): 508-520.
[2] HEINSFELD A S, FRANCO A R, CRADDOCK R C, et al. Identification of autism spectrum disorder using deep learning and the ABIDE dataset[J]. NeuroImage: clinical, 2018, 17: 16-23.
[3] LIU Xingdan, WU Jiacheng, LI Wenqi, et al. Domain adaptation via low rank and class discriminative representation for autism spectrum disorder identification: a multi-site fMRI study[J]. IEEE transactions on neural systems and rehabilitation engineering, 2023: 3233656.
[4] WANG Mingliang, ZHANG Daoqiang, HUANG Jiashuang, et al. Identifying autism spectrum disorder with multi-site fMRI via low-rank domain adaptation[J]. IEEE transactions on medical imaging, 2020, 39(3): 644-655.
[5] ABRAHAM A, MILHAM M P, DI MARTINO A, et al. Deriving reproducible biomarkers from multi-site resting-state data: an autism-based example[J]. NeuroImage, 2017, 147: 736-745.
[6] 蔡鸿顺, 张琼敏, 龙颖. 面向阿尔茨海默症辅助诊断的多尺度域适应网络[J]. 智能系统学报, 2023, 18(5): 1090-1098.
CAI Hongshun, ZHANG Qiongmin, LONG Ying. Multiscale domain adaptation network for the auxiliary diagnosis of Alzheimer’s disease[J]. CAAI transactions on intelligent systems, 2023, 18(5): 1090-1098.
[7] 梁艳, 温兴, 潘家辉. 融合全局与局部特征的跨数据集表情识别方法[J]. 智能系统学报, 2023, 18(6): 1205-1212.
LIANG Yan, WEN Xing, PAN Jiahui. Cross-dataset facial expression recognition method fusing global and local features[J]. CAAI transactions on intelligent systems, 2023, 18(6): 1205-1212.
[8] MANSOUR Y, MOHRI M, ROSTAMIZADEH A. Domain adaptation: learning bounds and algorithms[EB/OL]. (2023-11-30)[2024-03-13]. https://arxiv.org/abs/0902.3430v3.
[9] BEN-DAVID S, BLITZER J, CRAMMER K, et al. A theory of learning from different domains[J]. Machine learning, 2010, 79(1): 151-175.
[10] LONG Mingsheng, CAO Yue, WANG Jianmin, et al. Learning transferable features with deep adaptation networks[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille: ACM, 2015: 97-105.
[11] DING Jie, WANG Li, YU Lei, et al. Low-rank domain adaptive method with inter-class difference constraint for multi-site autism spectrum disorder identification[C]//2022 7th International Conference on Computational Intelligence and Applications. Nanjing: IEEE, 2022: 237-242.
[12] CHEN Ziliang, ZHUANG Jingyu, LIANG Xiaodan, et al. Blending-target domain adaptation by adversarial meta-adaptation networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 2243-2252.
[13] LIU Ziwei, MIAO Zhongqi, PAN Xingang, et al. Open compound domain adaptation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12406-12415.
[14] ZHENG J, WU Wenzhao, FU Haohuan, et al. Unsupervised mixed multi-target domain adaptation for remote sensing images classification[C]//2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa: IEEE, 2020: 1381-1384.
[15] XU Pengcheng, WANG Boyu, LING C. Class overwhelms: mutual conditional blended-target domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2023, 37(3): 3036-3044.
[16] ROY S, KRIVOSHEEV E, ZHONG Zhun, et al. Curriculum graph co-teaching for multi-target domain adaptation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 5347-5356.
[17] PARISOT S, KTENA S I, FERRANTE E, et al. Disease prediction using graph convolutional networks: application to autism spectrum disorder and alzheimer’s disease[J]. Medical image analysis, 2018, 48: 117-130.
[18] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[EB/OL]. (2015-05-09)[2024-03-13]. https://arxiv.org/abs/1503.02531v1.
[19] 黄震华, 杨顺志, 林威, 等. 知识蒸馏研究综述[J]. 计算机学报, 2022, 45(3): 624-653.
HUANG Zhenhua, YANG Shunzhi, LIN Wei, et al. Knowledge distillation: a survey[J]. Chinese journal of computers, 2022, 45(3): 624-653.
[20] CHEN Defang, MEI Jianping, WANG Can, et al. Online knowledge distillation with diverse peers[C]//Proceedings of the AAAI conference on artificial intelligence. New York: AAAI, 2020, 34(4): 3430-3437.
[21] WILSON G, COOK D J. A survey of unsupervised deep domain adaptation[J]. ACM transactions on intelligent systems and technology, 2020, 11(5): 1-46.
[22] LUO Yadan, WANG Zijian, HUANG Zi, et al. Progressive graph learning for open-set domain adaptation[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ACM, 2020: 6468-6478.
[23] LONG Mingsheng, CAO Zhangjie, WANG Jianmin, et al. Conditional adversarial domain adaptation[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal: ACM, 2018: 1647-1657.
[24] DI MARTINO A, YAN C G, LI Q, et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism[J]. Molecular psychiatry, 2014, 19(6): 659-667.
[25] CAMERON C, SHARAD S, BRIAN C, et al. Towards automated analysis of connectomes: the configurable pipeline for the analysis of connectomes (C-PAC)[J]. Frontiers in neuroinformatics, 2013, 7: 42.
[26] RYU M, LEE G, LEE K. Knowledge distillation for BERT unsupervised domain adaptation[J]. Knowledge and information systems, 2022, 64(11): 3113-3128.
[27] MURPHY K, SCH?LKOPF B, GANIN Y, et al. Domain-adversarial training of neural networks[J]. Journal of machine learning research, 2016, 17(1): 2096-2030.
[28] LONG Mingsheng, ZHU Han, WANG Jianmin, et al. Deep transfer learning with joint adaptation networks[C]//Proceedings of the 34th International Conference on Machine Learning. Sydney: ACM, 2017: 2208-2217.
[29] ZELLINGER W, MOSER B A, GRUBINGER T, et al. Robust unsupervised domain adaptation for neural networks via moment alignment[J]. Information sciences, 2019, 483: 174-191.
[30] NGUYEN-MEIDINE L T, BELAL A, KIRAN M, et al. Unsupervised multi-target domain adaptation through knowledge distillation[C]//2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2021: 1338-1346.
Similar References:

Memo

-

Last Update: 2025-01-05

Copyright © CAAI Transactions on Intelligent Systems