[1]SHEN Xueli,YU Jiahao.Multimodal graph contrastive learning recommendation based on fusion diffusion models[J].CAAI Transactions on Intelligent Systems,2026,21(4):1044-1054.[doi:10.11992/tis.202507015]
Copy

Multimodal graph contrastive learning recommendation based on fusion diffusion models

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 21 Number of periods: 2026 4 Page number: 1044-1054 Column: 人工智能院长论坛 Public date: 2026-07-05
Title:
Multimodal graph contrastive learning recommendation based on fusion diffusion models
Author(s):
SHEN Xueli;  YU Jiahao
School of Software, Liaoning Technical University, Huludao 125105, China
Keywords:
recommender system; diffusion model; multimodal; graph contrastive learning; self-supervised learning; multitask learning; data sparsity; user interest
CLC:
TP311
DOI:
10.11992/tis.202507015
Abstract:
To address the inaccurate modeling of user interest preferences caused by noise interference, data sparsity, and cross-modal semantic discrepancies in multimodal data, this paper proposes a diffusion-enhanced exploration framework for multimodal graph contrastive recommendation(DEMGR). First, a diffusion model is employed to generate structurally enhanced contrastive views, effectively suppressing modality-specific noise during self-supervised learning. Second, an isomorphic graph integrating users and items is constructed to strengthen graph connectivity and mitigate the effects of data sparsity on graph representation learning. Finally, a joint optimization framework is designed to combine multitask learning with diffusion enhancement, enabling the collaborative optimization of recommendation objectives and enhancement signals while learning semantically consistent and generalizable multimodal representations. Experiments conducted on three real-world datasets, Baby, Sports, and Clothing, demonstrate that DEMGR consistently outperforms existing mainstream models in terms of Recall and normalized discounted cumulative gain(NDCG), achieving maximum improvements of 5.45% and 5.01%, respectively. These results verify the effectiveness of the proposed method for multimodal recommendation and provide valuable insights for the optimization of multimodal recommender systems.
References:
[1] 姜信景, 齐小刚, 刘立芳. 个性化信息推荐方法研究[J]. 智能系统学报, 2018, 13(2): 189195 JIANG Xinjing, QI Xiaogang, LIU Lifang. Research on the recommendation method of personalized information[J]. CAAI transactions on intelligent systems, 2018, 13(2): 189195
[2] 束玮, 李翔, 孙纪舟, 等. 基于动态兴趣传播和知识图谱的推荐方法[J]. 智能系统学报, 2024, 19(4): 9971006 SHU Wei, LI Xiang, SUN Jizhou, et al. Recommendation method based on dynamic interest propagation and knowledge graph[J]. CAAI transactions on intelligent systems, 2024, 19(4): 9971006
[3] 姚暄, 高君宇, 徐常胜. 基于自监督图对比学习的视频问答方法[J]. 软件学报, 2023, 34(5): 20832100. YAO Xuan, GAO Junyu, XU Changsheng. Video question answering method based on self-supervised graph contrastive learning[J]. Journal of Software, 2023, 34(5): 20832100.
[4] HE Ruining, MCAULEY J. VBPR: visual Bayesian personalized ranking from implicit feedback[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Phoenix: AAAI, 2016.
[5] WANG Qifan, WEI Yinwei, YIN Jianhua, et al. DualGNN: dual graph neural network for multimedia recommendation[J]. IEEE transactions on multimedia, 2023, 25: 1074-1084
[6] WU Jiancan, WANG Xiang, FENG Fuli, et al. Self-supervised graph learning for recommendation[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. Online: ACM, 2021.
[7] XIA Lianghao, HUANG Chao, XU Yong, et al. Hypergraph contrastive collaborative filtering[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Madrid: ACM, 2022.
[8] WEI Wei, HUANG Chao, XIA Lianghao, et al. Multi-modal self-supervised learning for recommendation[C]//Proceedings of the ACM Web Conference 2023. Austin: ACM, 2023.
[9] RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: Bayesian personalized ranking from implicit feedback[EB/OL]. (2012-05-09)[2025-05-22]. https://arxiv.org/abs/1205.2618.
[10] 吴国栋, 查志康, 涂立静, 等. 图神经网络推荐研究进展[J]. 智能系统学报, 2020, 15(1): 1424 WU Guodong, ZHA Zhikang, TU Lijing, et al. Research advances in graph neural network recommendation[J]. CAAI transactions on intelligent systems, 2020, 15(1): 1424
[11] WANG Xiang, HE Xiangnan, WANG Meng, et al. Neural graph collaborative filtering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Paris: ACM, 2019.
[12] HE Xiangnan, DENG Kuan, WANG Xiang, et al. Lightgcn: simplifying and powering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. Xi’an: ACM, 2020.
[13] ZHANG Dan, ZHU Yifan, DONG Yuxiao, et al. ApeGNN: node-wise adaptive aggregation in GNNs for recommendation[C]//Proceedings of the ACM Web Conference 2023. Austin: ACM, 2023.
[14] GUAN Yuanshen, XU Ruikang, YAO Mingde, et al. Mutual-guided dynamic network for image fusion[C]//Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023.
[15] LIU Qiang, WU Shu, WANG Liang. DeepStyle: learning user preferences for visual recommendation[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. Shinjuku Tokyo: ACM, 2017.
[16] 田震, 潘腊梅, 尹朴, 等. 深度矩阵分解推荐算法[J]. 软件学报, 2021, 32(12): 39173928 TIAN Zhen, PAN Lamei, YIN Pu, et al. Deep matrix factorization recommendation algorithm[J]. Journal of software, 2021, 32(12): 39173928
[17] CHEN Xu, CHEN Hanxiong, XU Hongteng, et al. Personalized fashion recommendation with visual explanations based on multimodal attention network: towards visually explainable recommendation[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Paris: ACM, 2019.
[18] 范航舟, 梅红岩, 赵勤, 等. 融合双注意力机制的GNN 多维时间序列预测[J]. 智能系统学报, 2024, 19(5): 12771286 FAN Hangzhou, MEI Hongyan, ZHAO Qin, et al. Multivariate time series forecasting with a graph neural network and dual attention mechanism[J]. CAAI transactions on intelligent systems, 2024, 19(5): 12771286
[19] WEI Yinwei, WANG Xiang, NIE Liqiang, et al. MMGCN: multi-modal graph convolution network for personalized recommendation of mcro-video[C]//Proceedings of the 27th ACM International Conference on Multimedia. Nice: ACM, 2019.
[20] ZHANG Jinghao, ZHU Yanqiao, LIU Qiang, et al. Mining latent structures for multimedia recommendation[C]//Proceedings of the 29th ACM International Conference on Multimedia. Online: ACM, 2021.
[21] ZHOU Xin, SHEN Zhiqi. A tale of two graphs: freezing and denoising graph structures for multimodal recommendation[C]//Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023.
[22] 管凤旭, 张涵宇, 路斯棋, 等. 扩散模型在计算机视觉领域的研究现状[J]. 智能系统学报, 2025, 20(2): 265282 GUAN Fengxu, ZHANG Hanyu, LU Siqi, et al. Research status of diffusion models in computer vision[J]. CAAI transactions on intelligent systems, 2025, 20(2): 265282
[23] 蒋世杰, 夏秀山, 翟伟, 等. 基于ODE扩散模型的多类异常检测和定位[J]. 智能系统学报, 2025, 20(2): 376388 JIANG Shijie, XIA Xiushan, ZHAI Wei, et al. ODE diffusion model for multiclass anomaly detection and localization[J]. CAAI transactions on intelligent systems, 2025, 20(2): 376388
[24] MA Haokai, XIE Ruobing, MENG Lei, et al. Plug-In diffusion model for sequential recommendation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2024.
[25] YANG Zhengyi, WU Jiancan, WANG Zhicai, et al. Generate what you prefer: reshaping sequential recommendation via guided diffusion[C]//Advances in Neural Information Processing Systems 36. New Orleans: Neural Information Processing Systems Foundation, Inc., 2023.
[26] WANG Wenjie, XU Yiyan, FENG Fuli, et al. Diffusion recommender model[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. Taipei: ACM, 2023.
[27] JIANG Yangqin, YANG Yuhao, XIA Lianghao, et al. DiffKG: knowledge graph diffusion model for recommendation[C]//Proceedings of the 17th ACM International Conference on Web Search and Data Mining. Merida: ACM, 2024.
[28] YEH C H, HONG Chengyao, HSU Y C, et al. Decoupled contrastive learning[C]//Computer Vision–ECCV 2022. Cham: Springer, 2022.
[29] REIMERS N, GUREVYCH I. Sentence-BERT: sentence embeddings using siamese bert-networks[EB/OL]. (2019-08-27)[2025-05-22]. http://arxiv.org/abs/1908.10084, 2019.
[30] XU Jinfeng, CHEN Zheyu, LI Jinze, et al. MDVT: enhancing multimodal recommendation with model-agnostic multimodal-driven virtual triplets[C]//Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. Toronto: ACM, 2025.
[31] LIN Xixun, LIU Rui, CAO Yanan, et al. Contrastive modality-disentangled learning for multimodal recommendation[J]. ACM transactions on information systems, 2025, 43(3): 1-31
[32] ONG R K, KHONG A W H. Spectrum-based modality representation fusion graph convolutional network for multimodal recommendation[C]//Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining. Hannover: ACM, 2025.
[33] JIANG Yangqin, XIA Lianghao, WEI Wei, et al. DiffMM: multi-modal diffusion model for recommendation[C]//Proceedings of the 32nd ACM International Conference on Multimedia. Melbourne: ACM, 2024.
[34] KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. (2014-01-22)[2025-05-22]. http://arxiv.org/abs/1412.6980.
[35] VAN DER MAATEN L, HINTON G. Visualizing data using t-SNE[J]. Computo a journal of the French Statistical Society, 2008, 9(11): 1-25
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems