[1]LI Shanshan,ZHAO Qingjie,ZHU Wenlong,et al.Cross-domain knowledge generalization method introducing causal discovery learning[J].CAAI Transactions on Intelligent Systems,2025,20(4):1033-1045.[doi:10.11992/tis.202501005]
Copy

Cross-domain knowledge generalization method introducing causal discovery learning

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 4 Page number: 1033-1045 Column: 学术论文—机器学习 Public date: 2025-08-05
Title:
Cross-domain knowledge generalization method introducing causal discovery learning
Author(s):
LI Shanshan1; 2;  ZHAO Qingjie2;  ZHU Wenlong1;  RUAN Jinjia3;  YU Tiejun1;  MA Shaohui1;  SUN Baosheng1
1. Beijing Jinghang Research Institute of Computing and Communication, Beijing 100074, China;
2. School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China;
3. China Waterborne Transport Research Institute, Beijing 100088, China
Keywords:
transfer learning; domain generalization; image classification; causality; causal representation learning; variational inference; causal discovery; counterfactual contrastive
CLC:
TP391
DOI:
10.11992/tis.202501005
Abstract:
Domain generalization aims to generalize knowledge from multiple known domains to unknown target domains. However, existing models are easily affected by high-dimensional noise when extracting image features, which causes the unstable relationship between the extracted image features and labels. Thus, inspired by the cross-domain invariant causal mechanism, we propose a cross-domain knowledge generalization method introducing causal discovery learning. Specifically, we extract the low-dimensional latent features of the image to retain the basic information of the image. Meanwhile, we perform variational inference on the low-dimensional latent features to achieve mutual independence of latent feature variables. We reconstruct the causal directed acyclic graphs (DAG) between latent feature variables and category labels to discover the latent feature variables that have stable causal structures with category labels. We introduce a counterfactual contrastive regularization term, which exploits counterfactual variance and invariance during data generation to make causal inference and generate causal invariant representations. To verify the proposed method, we conducted tests on five datasets under the DomainBed framework and four datasets under the SWAD framework. Experiments show that compared with existing methods, our domain generalization model has greater improvements in performance and adaptability.
References:
[1] 张凯, 杨朋澄, 彭开香, 等. 基于深度置信网络的多模态过程故障评估方法及应用[J]. 自动化学报, 2024, 50(1): 89-102.
ZHANG Kai, YANG Pengcheng, PENG Kaixiang, et al. A deep belief network-based fault evaluation method for multimode processes and its applications[J]. Acta automatica sinica, 2024, 50(1): 89-102.
[2] 张汝波, 蔺庆龙, 张天一. 基于深度学习的图像篡改检测方法综述[J]. 智能系统学报, 2025, 20(2): 283-304.
ZHANG Rubo, LIN Qinglong, ZHANG Tianyi. A review of image tampering detection methods based on deep learning[J]. CAAI transactions on intelligent systems, 2025, 20(2): 283-304.
[3] 丁贵广, 陈辉, 王澳, 等. 视觉深度学习模型压缩加速综述[J]. 智能系统学报, 2024, 19(5): 1072-1081.
DING Guiguang, CHEN Hui, WANG Ao, et al. Review of model compression and acceleration for visual deep learning[J]. CAAI transactions on intelligent systems, 2024, 19(5): 1072-1081.
[4] LI Shanshan, ZHAO Qingjie, ZHANG Changchun, et al. Deep discriminative causal domain generalization[J]. Information sciences, 2023, 645: 119335.
[5] 邵海东, 肖一鸣, 颜深, 等. 域泛化机械故障诊断研究进程与展望[J]. 中国科学: 技术科学, 2025, 55(1): 14-32.
SHAO Haidong, XIAO Yiming, YAN Shen, et al. Progress and prospects of domain generalization mechanical fault diagnosis research[J]. Scientia sinica (technologica), 2025, 55(1): 14-32.
[6] 崔腾, 张海军, 代伟. 基于分布共识的联邦增量迁移学习[J]. 计算机学报, 2024, 47(4): 821-841.
CUI Teng, ZHANG Haijun, DAI Wei. Federated incremental transfer learning based on distributed consensus[J]. Chinese journal of computers, 2024, 47(4): 821-841.
[7] 李鑫尧, 李晶晶, 朱磊, 等. 资源受限的大模型高效迁移学习算法研究综述[J]. 计算机学报, 2024, 47(11): 2491-2521.
LI Xinyao, LI Jingjing, ZHU Lei, et al. Efficient transfer learning of large models with limited resources: a survey[J]. Chinese journal of computers, 2024, 47(11): 2491-2521.
[8] MAHAJAN D, TOPLE S, SHARMA A. Domain generalization using causal matching[C]//Proceedings of International Conference on Machine Learning. [S.l.]: OpenReview.net, 2021: 7313-7324.
[9] MAO Haiyi, LIU Hongfu, DOU J, et al. Towards cross-modal causal structure and representation learning[C]//Machine Learning for Health. New Orleans: PMLR, 2022: 120-140.
[10] ZHAO Shengjia, SONG Jiaming, ERMON S. Learning hierarchical features from deep generative models[C]//Proceedings of the International Conference on Machine Learning. Sydney: OpenReview.net, 2017: 4091-4099.
[11] ZHENG Xun, DAN Chen, ARAGAM B, et al. Learning sparse nonparametric dags[C]//Proceedings of the International Conference on Artificial Intelligence and Statistics. [S.l.]: PMLR, 2020: 3414-3425.
[12] GULRAJANI I, LOPEZ-PAZ D. In search of lost domain generalization[C]//Proceedings of International Conference on Learning Representations. [S.l.]: PMLR, 2020: 1-29.
[13] CHA J, CHUN S, LEE K, et al. SWAD: domain generalization by seeking flat minima[EB/OL]. (2021-02-17) [2025-01-07]. https://arxiv.org/abs/2102.08604.
[14] GHIFARY M, KLEIJN W B, ZHANG Mengjie, et al. Domain generalization for object recognition with multi-task autoencoders[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 2551-2559.
[15] FANG Chen, XU Ye, ROCKMORE D N. Unbiased metric learning: on the utilization of multiple datasets and web images for softening bias[C]//2013 IEEE International Conference on Computer Vision. Sydney: IEEE, 2013: 1657-1664.
[16] LI Da, YANG Yongxin, SONG Yizhe, et al. Deeper, broader and artier domain generalization[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 5543-5551.
[17] BEERY S, VAN HORN G, PERONA P. Recognition in terra incognita[C]//Computer Vision–ECCV 2018. Cham: Springer International Publishing, 2018: 472-489.
[18] PENG Xingchao, BAI Qinxun, XIA Xide, et al. Moment matching for multi-source domain adaptation[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 1406-1415.
[19] ARJOVSKY M, BOTTOU L, GULRAJANI I, et al. Invariant risk minimization[EB/OL]. (2019-07-05) [2025-01-07]. https://arxiv.org/abs/1907.02893.
[20] VAPNIK V N. An overview of statistical learning theory[J]. IEEE transactions on neural networks, 1999, 10(5): 988-999.
[21] HUANG Zeyi, WANG Haohan, XING E P, et al. Self-challenging improves cross-domain generalization[C]//Computer Vision–ECCV 2020. Cham: Springer International Publishing, 2020: 124-140.
[22] NAM H, LEE H, PARK J, et al. Reducing domain gap by reducing style bias[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 8686-8695.
[23] PARASCANDOLO G, NEITZ A, ORVIETO A, et al. Learning explanations that are hard to vary[C]//Proceedings of International Conference on Learning Representations. [S.l.]: PMLR, 2021: 1-24.
[24] SHAHTALEBI S, GAGNON-AUDET J C, LALEH T, et al. SAND-mask: an enhanced gradient masking strategy for the discovery of invariances in domain generalization[EB/OL]. (2021-06-04) [2025-01-07]. https://arxiv.org/pdf/2106.02266.
[25] SHI Yuge, SEELY J, TORR P, et al. Gradient matching for domain generalization[C]//Proceedings of International Conference on Learning Representations. [S.l.]: OpenReview.net, 2022: 1-28.
[26] RAME A, DANCETTE C, CORD M. Fishr: invariant gradient variances for out-of-distribution generalization[C]//Proceedings of International Conference on Machine Learning. Baltimore: OpenReview.net, 2022: 18347-18377.
[27] CHEN Liang, ZHANG Yong, SONG Yibing, et al. Domain generalization via rationale invariance[C]//2023 IEEE/CVF International Conference on Computer Vision. Paris: IEEE, 2023: 1751-1760.
[28] WANG Pengfei, ZHANG Zhaoxiang, LEI Zhen, et al. Sharpness-aware gradient matching for domain generalization[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 3769-3778.
[29] AMINBEIDOKHTI M, PE?A F A G, MEDEIROS H R, et al. Domain generalization by rejecting extreme augmentations[C]//2024 IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2024: 2204-2214.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems