[1]LI Changyuan,CHENG Kai,HAO Wenning,et al.Comprehensive survey on compositional generalization methodologies in artificial intelligence[J].CAAI Transactions on Intelligent Systems,2026,21(4):815-833.[doi:10.11992/tis.202507001]
Copy

Comprehensive survey on compositional generalization methodologies in artificial intelligence

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 21 Number of periods: 2026 4 Page number: 815-833 Column: 综述 Public date: 2026-07-05
Title:
Comprehensive survey on compositional generalization methodologies in artificial intelligence
Author(s):
LI Changyuan;  CHENG Kai;  HAO Wenning;  SHAO Tianhao;  ZHANG Jinpeng
College of Command and Control Engineering, Army Engineering University of PLA, Nanjing 210007, China
Keywords:
compositionality; compositional generalization; artificial intelligence; foundational representation architecture; relational modeling architecture; learning mechanism architecture; machine learning; meta-learning
CLC:
TP183; TP181; TP391.1
DOI:
10.11992/tis.202507001
Abstract:
Compositionality, a critical aspect of human intelligence, is often lacking in modern artificial intelligence systems. Its integration can considerably enhance a system’s generalization capabilities and its ability to solve novel and complex tasks. This study addresses the current challenges of a vague understanding of compositional generalization, fragmented method classification, and superficial explanations of underlying mechanisms. The study begins by clarifying the fundamental concept of compositional generalization, analyzing its characteristics across four dimensions: systematicity, compositionality, interpretability, and dynamic adaptability. A classification system for compositional generalization methods based on architectural evolution is then proposed, in which existing compositional methods are categorized according to their foundational representation architecture, relational modeling architecture, and learning mechanism architecture. The mechanism linking each method’s model structure to compositional generalization is elaborated in detail. The advantages and disadvantages, as well as the scene applicability of different methods, are further compared across various scenarios. Finally, feasible solutions are proposed to address persistent challenges in compositional generalization, such as dynamics, knowledge dependence, computational complexity, and heterogeneous relationships, thereby providing guidance for in-depth research in this field.
References:
[1] MITROU I N, STAMATOPOULOS P. Generative artificial intelligence: models, benefits, dangers and detection of AI-generated text on specialized domains[D]. Athens: National and Kapodistrian University of Athens, 2024: 16.
[2] STIENNON N, OUYANG L, WU J, et al. Learning to summarize with human feedback[C]//Advances in Neural Information Processing Systems 33. Vancouver: Curran Associates, Inc. , 2020: 3008-3021.
[3] BUBECK S, CHANDRASEKARAN V, ELDAN R, et al. Sparks of artificial general intelligence: early experiments with GPT-4[EB/OL]. (2023-03-22)[2025-01-01]. https://arxiv.org/abs/2303.12712.
[4] VIJAYARAGHAVAN P, QUEI?ER J F, FLORES S V, et al. Development of compositionality through interactive learning of language and action of robots[J]. Science robotics, 2025, 10(98): eadp0751
[5] MORRIS M R, SOHL-DICKSTEIN J N, FIEDEL N, et al. Position: levels of AGI for operationalizing progress on the path to AGI[C]//International Conference on Machine Learning. Vienna: PMLR, 2024: 36308-36321.
[6] FODOR J A, PYLYSHYN Z W. Connectionism and cognitive architecture: a critical analysis[J]. Cognition, 1988, 28(1/2): 3-71
[7] 柯琳, 杨笑笑, 陈智斌. 一种带泛化性能的动态混合模型求解大范围TSP问题[J]. 系统科学与数学, 2024, 44(1): 31-44 KE Lin, YANG Xiaoxiao, CHEN Zhibin. A dynamic hybrid model with generalization performance to solve large-scale TSP[J]. Journal of systems science and mathematical sciences, 2024, 44(1): 31-44
[8] 吴国栋, 秦辉, 胡全兴, 等. 大语言模型及其个性化推荐研究[J]. 智能系统学报, 2024, 19(6): 1351-1365 WU Guodong, QIN Hui, HU Quanxing, et al. Research on large language models and personalized recommendation[J]. CAAI transactions on intelligent systems, 2024, 19(6): 1351-1365
[9] 潘登, 毕晓君. 基于Transformer模型的自闭症功能磁共振图像分类[J]. 智能系统学报, 2025, 20(2): 400-406 PAN Deng, BI Xiaojun. Classification of functional magnetic resonance images for autism based on Transformer model[J]. CAAI transactions on intelligent systems, 2025, 20(2): 400-406
[10] YAGCIOGLU S, ?NCE O B, ERDEM A, et al. Sequential compositional generalization in multimodal models[C]//Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Kerrville: Association for Computational Linguistics, 2024: 5591-5611.
[11] KOH P W, SAGAWA S, MARKLUND H, et al. WILDS: a benchmark of in-the-wild distribution shifts[EB/OL]. (2020-12-14)[2025-01-01]. https://arxiv.org/abs/2012.07421.
[12] LAKE B M, BARONI M. Human-like systematic generalization through a meta-learning neural network[J]. Nature, 2023, 623(7985): 115-121
[13] SINHA S, PREMSRI T, KORDJAMSHIDI P. A survey on compositional learning of AI models: theoretical and experimental practices[EB/OL]. (2024-06-13)[2025-01-01]. https://arxiv.org/abs/2406.08787.
[14] 汤欧, 郑琪. 论符号主义与联结主义人工智能的发展[J]. 哈尔滨学院学报, 2024, 45(7): 20-24 TANG Ou, ZHENG Qi. On the development of symbolism and connectionism artificial intelligence[J]. Journal of Harbin University, 2024, 45(7): 20-24
[15] 丁林. 人工智能如何走到今天?[N]. 北京科技报, 2024-10-14(006). DOI:10.28030/n.cnki.nbjkj.2024.000087.
[16] 刘超, 梁安婷, 刘小洋, 等. 融合多角度信息和图卷积网络的社交网络节点分类模型[J]. 重庆理工大学学报(自然科学), 2022, 36(5): 147-160 LIU Chao, LIANG Anting, LIU Xiaoyang, et al. Social networks node classification model based on multi-angle information fusion and graph convolutional networks[J]. Journal of Chongqing University of Technology (natural science), 2022, 36(5): 147-160
[17] 李玲, 雷宏友. 结合MoE与Transformer的生态翻译模型优化研究[J]. 自动化与仪器仪表, 2025(4): 178-181,186 LI Ling, LEI Hongyou. Research on ecological translation model optimization combining MoE and Transformer[J]. Automation & instrumentation, 2025(4): 178-181,186
[18] SHIN R, LIN C, THOMSON S, et al. Constrained language models yield few-shot semantic parsers[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Kerrville: Association for Computational Linguistics, 2021: 7699-7715.
[19] KIPF T, ELSAYED G F, MAHENDRAN A, et al. Conditional object-centric learning from video[EB/OL]. (2021-11-24)[2025-01-01]. https://arxiv.org/abs/2111.12594.
[20] BAHDANAU D, MURTY S, NOUKHOVITCH M, et al. Systematic generalization: what is required and can it be learned? [EB/OL]. (2018-11-30)[2025-01-01]. https://arxiv.org/abs/1811.12889.
[21] 王雨嫣, 廖柏林, 彭晨, 等. 递归神经网络研究综述[J]. 吉首大学学报(自然科学版), 2021, 42(1): 41-48 WANG Yuyan, LIAO B L, PENG Chen, et al. Research review of recurrent neural networks[J]. Journal of Jishou University (natural science edition), 2021, 42(1): 41-48
[22] ZHANG C, PENG Benji, SUN Xintian, et al. From word vectors to multimodal embeddings: techniques, applications, and future directions for large language models[EB/OL]. (2024-11-06)[2025-01-01]. https://arxiv.org/abs/2411.05036.
[23] TU Z, HE F, TAO D. Understanding generalization in recurrent neural networks[C]//International Conference on Learning Representations. Addis Ababa: International Conference on Learning Representations, 2020: 1-17.
[24] ZHU Jiancong. Comparative study of sequence-to-sequence models: from RNNs to transformers[J]. Applied and computational engineering, 2024, 42(1): 67-75
[25] CABRERA-LE?N Y, B?EZ P G, FERN?NDEZ-L?PEZ P, et al. Neural computation-based methods for the early diagnosis and prognosis of Alzheimer’s disease not using neuroimaging biomarkers: a systematic review[J]. Journal of Alzheimer’s disease, 2024, 98(3): 793-823
[26] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2. New York: ACM, 2014: 3104-3112.
[27] GREFF K, SRIVASTAVA R K, KOUTN?K J, et al. LSTM: a search space odyssey[J]. IEEE transactions on neural networks and learning systems, 2017, 28(10): 2222-2232
[28] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30. Long Beach: Curran Associates, Inc. , 2017: 5998-6008.
[29] RUSSIN J, JO J, O’REILLY R, et al. Compositional generalization by factorizing alignment and translation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. Online: ACL, 2020: 313-327.
[30] BHARADHWAJ H, VAKIL J, SHARMA M, et al. RoboAgent: generalization and efficiency in robot manipulation via semantic augmentations and action chunking[C]//2024 IEEE International Conference on Robotics and Automation. Yokohama: IEEE, 2024: 4788-4795.
[31] 陈金磊. 基于聚类分析的模块化神经网络研究与应用[D]. 郑州: 郑州大学, 2019: 1-54. CHEN Jinlei. Research and application of modular neural network based on cluster analysis[D]. Zhengzhou: Zhengzhou University, 2019: 1-54.
[32] WANG Liyuan, ZHANG Xingxing, LI Qian, et al. Incorporating neuro-inspired adaptability for continual learning in artificial intelligence[J]. Nature machine intelligence, 2023, 5(12): 1356-1368
[33] LIU Ziming, KHONA M, FIETE I R, et al. Growing brains: co-emergence of anatomical and functional modularity in recurrent neural networks[EB/OL]. (2023-10-11)[2025-01-01]. https://arxiv.org/abs/2310.07711.
[34] FEDUS W, ZOPH B, SHAZEER N. Switch transformers: scaling to trillion parameter models with simple and efficient sparsity[J]. Journal of machine learning research, 2022, 23(1): 5232-5270
[35] ANDREAS J, ROHRBACH M, DARRELL T, et al. Neural module networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 39-48.
[36] ZHENG Yihan, WEN Zhiquan, TAN Mingkui, et al. Modular graph attention network for complex visual relational reasoning[M]//Computer Vision. Cham: Springer International Publishing, 2021: 137-153.
[37] WANG Xin, CHEN Hong, TANG Si’ao, et al. Disentangled representation learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2024, 46(12): 9677-9696
[38] ZHENG Hao, LAPATA M. Disentangled sequence to sequence learning for compositional generalization[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Kerrville: Association for Computational Linguistics, 2022: 4256-4268.
[39] WANG Jindong, LAN Cuiling, LIU Chang, et al. Generalizing to unseen domains: a survey on domain generalization[J]. IEEE transactions on knowledge and data engineering, 2023, 35(8): 8052-8072
[40] 谢娟英, 张建宇. 图卷积神经网络综述[J]. 陕西师范大学学报(自然科学版), 2024, 52(2): 89-101 XIE Juanying, ZHANG Jianyu. The review of the graph convolutional neural networks[J]. Journal of Shaanxi Normal University (natural science edition), 2024, 52(2): 89-101
[41] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. (2016-09-09)[2025-01-01]. https://arxiv.org/abs/1609.02907.
[42] HAMILTON W L, YING R, LESKOVEC J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 1025-1035.
[43] PAN H, FANG Y, HUANG C, et al. GCNXSS: an attack detection approach for cross-site scripting based on graph convolutional networks[J]. KSII transactions on internet and information systems, 2022, 16(12): 4008-4023
[44] 吴琳, 许茹玉, 粟兴旺, 等. 基于结构误差的图卷积网络[J]. 计算机应用研究, 2023, 40(1): 155-159 WU Lin, XU Ruyu, SU Xingwang, et al. Graph convolutional networks based on structural errors[J]. Application research of computers, 2023, 40(1): 155-159
[45] PASA L, NAVARIN N, SPERDUTI A. Polynomial-based graph convolutional neural networks for graph classification[J]. Machine learning, 2022, 111(4): 1205-1237
[46] 吴誉兰, 舒建文. 基于自适应差异化图卷积的图注意力网络表示学习算法[J]. 现代电子技术, 2025, 48(2): 51-54 WU Yulan, SHU Jianwen. Graph attention network representation learning algorithm based on adaptive differentiation graph convolution[J]. Modern electronics technique, 2025, 48(2): 51-54
[47] ZHOU Zhiyuan, YIN Yueming, HAN Hao, et al. ProAffinity-GNN: a novel approach to structure-based protein-protein binding affinity prediction via a curated data set and graph neural networks[J]. Journal of chemical information and modeling, 2024, 64(23): 8796-8808
[48] POURSAEED O, JIANG Tianxing, YANG H, et al. Robustness and generalization via generative adversarial training[C]//2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 15691-15700.
[49] ZHANG Zhi, LI Weijian, LIU Han. Multivariate time series forecasting by graph attention networks with theoretical guarantees[C]//International Conference on Artificial Intelligence and Statistics. Valencia: PMLR, 2024.
[50] BRODY S, ALON U, YAHAV E. How attentive are graph attention networks? [EB/OL]. (2021-05-30)[2025-01-01]. https://arxiv.org/abs/2105.14491.
[51] YU Dongran, YANG Bo, LIU Dayou, et al. A survey on neural-symbolic learning systems[J]. Neural networks, 2023, 166: 105-126
[52] LI Qing, ZHU Yixin, LIANG Yitao, et al. Neural-symbolic recursive machine for systematic generalization[EB/OL]. (2022-10-04)[2025-01-01]. https://arxiv.org/abs/2210.01603.
[53] GUPTA T, KEMBHAVI A. Visual programming: compositional visual reasoning without training[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 14953-14962.
[54] SATO S, SATO I. Can test-time computation mitigate reproduction bias in neural symbolic regression? [EB/OL]. (2025-05-28)[2025-06-01]. https://arxiv.org/abs/2505.22081.
[55] HOSPEDALES T M, ANTONIOU A, MICAELLI P, et al. Meta-learning in neural networks: a survey[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 44(9): 5149-5169
[56] FALLAH A, MOKHTARI A, OZDAGLAR A. Generalization of model-agnostic meta-learning algorithms: recurring and unseen tasks[EB/OL]. (2021-02-07)[2025-01-01]. https://arxiv.org/abs/2102.03832.
[57] DEB B, AWADALLAH A H, ZHENG Guoqing. Boosting natural language generation from instructions with meta-learning[C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2022: 6792-6808.
[58] VETTORUZZO A, BOUGUELIA M R, VANSCHOREN J, et al. Advances and challenges in meta-learning: a technical review[J]. IEEE transactions on pattern analysis and machine intelligence, 2024, 46(7): 4763-4779
[59] FINN C, ABBEEL P, LEVINE S. Model-agnostic meta-learning for fast adaptation of deep networks[C]//Proceedings of the 34th International Conference on Machine Learning. New York: ACM, 2017: 1126-1135.
[60] WANG Zhenhailong, YU Hang, LI Manling, et al. Rethinking task sampling for few-shot vision-language transfer learning[C]//Proceedings of the First Workshop on Performance and Interpretability Evaluations of Multimodal, Multipurpose, Massive-Scale Models. Online: ACL, 2022: 7-14.
[61] NICHOL A, ACHIAM J, SCHULMAN J. On first-order meta-learning algorithms[EB/OL]. (2018-03-04)[2025-01-01]. https://arxiv.org/abs/1803.02999.
[62] 马怡琳. 基于改进ProtoNet的小样本关系抽取方法研究[D]. 天津: 河北工业大学, 2021: 1-67. MA Yilin. Research on few-shot relationextraction method base on improved ProtoNet[D]. Tianjin: Hebei University of Technology, 2021: 1-67.
[63] CHEN CHAOFAN, LI OSCAR, TAO CHAOFAN, et al. This looks like that: deep learning for interpretable image recognition[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc. , 2019: 8930–8941.
[64] SNELL J, SWERSKY K, ZEMEL R S. Prototypical networks for few-shot learning[EB/OL]. (2017-03-15)[2025-01-01]. https://arxiv.org/abs/1703.05175.
[65] MANNEKOTE A. Towards a neural era in dialogue management for collaboration: a literature survey[EB/OL]. (2023-07-18)[2025-01-01]. https://arxiv.org/abs/2307.09021.
[66] G?LC? A, ALKAN M. Az ?rnekle ??renme problemleri i?in MAML ve ProtoNet algoritmalar?n?n ?ncelenmesi[J]. European journal of science and technology, 2021(21): 113-121
[67] CHANG H, PARK J, CHO H, et al. The coverage principle: a framework for understanding compositional generalization[EB/OL]. (2025-05-26)[2025-06-01]. https://arxiv.org/abs/2505.20278.
[68] ZHANG Baoquan, LI Xutao, YE Yunming, et al. Prototype completion with primitive knowledge for few-shot learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 3753-3761.
[69] CHOI J, KIM Y. Colorful cutout: enhancing image data augmentation with curriculum learning[EB/OL]. (2024-03-29)[2025-06-01]. https://arxiv.org/abs/2403.20012.
[70] CHEN P, LIU S, ZHAO H, et al. Gridmask data augmentation[EB/OL]. (2024-03-26)[2025-06-01]. https://arxiv.org/abs/2001.04086.
[71] AKY?REK E, AKY?REK A F, ANDREAS J. Learning to recombine and resample data for compositional generalization[EB/OL]. (2020-10-08)[2025-06-01]. https://arxiv.org/abs/2010.03706.
[72] QIU Linlu, SHAW P, PASUPAT P, et al. Improving compositional generalization with latent structure and data augmentation[C]//Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Kerrville: Association for Computational Linguistics, 2022: 4341-4362.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems