[1]仇佳庆,窦立云,王进.基于跨模态注意力的多阶段图像风格迁移[J].智能系统学报,2026,21(3):751-762.[doi:10.11992/tis.202508011]
QIU Jiaqing,DOU Liyun,WANG Jin.Multistage image style transfer based on cross-modal attention[J].CAAI Transactions on Intelligent Systems,2026,21(3):751-762.[doi:10.11992/tis.202508011]
点击复制
QIU Jiaqing,DOU Liyun,WANG Jin.Multistage image style transfer based on cross-modal attention[J].CAAI Transactions on Intelligent Systems,2026,21(3):751-762.[doi:10.11992/tis.202508011]
基于跨模态注意力的多阶段图像风格迁移
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第3期
页码:
751-762
栏目:
学术论文—智能系统
出版日期:
2026-05-05
- Title:
- Multistage image style transfer based on cross-modal attention
- Keywords:
- image style transfer; cross-modal attention; adaptive style modulation; latent diffusion model; image generation; text-guided image generation; multimodal generation; artistic style transfer
- 分类号:
- TP391；TH212
- 文献标志码:
- 2026-3-25
- 摘要:
- 图像风格迁移(image style transfer, IST)的核心是融合图像内容与目标艺术风格，生成兼具语义合理性与视觉表现力的图像，广泛应用于艺术创作、个性化图像编辑等领域。现有方法处理高分辨率图像、复杂纹理及跨模态引导任务时，存在计算效率低、风格可控性弱、细节丢失、风格与内容脱节等问题。为此，本文提出基于潜在扩散模型的多阶段风格迁移框架 CAST-Diff(cross-modal attention and style-adaptive diffusion framework)，采用解耦式协同设计，结合跨模态语义引导、自适应区域风格调节与潜空间扩散细化，通过跨模态注意力对齐图文特征，利用自适应模块控制区域风格强度，利用潜扩散模型完成去噪与细节重建。在 COCO、Flickr30k 数据集上与 StyleGAN-Diffusion、ControlNet 等主流方法对比表明，CAST-Diff 在风格一致性、细节保真度及视觉自然度上更优，能在复杂场景下保留图像结构与精细纹理，实现自然逼真的风格迁移，同时提升计算效率与泛化能力，为文本引导的高精度风格迁移提供可行方案。
- Abstract:
- Image style transfer (IST) aims to fuse image content with a target artistic style to generate semantically rational and visually expressive images, and it has been widely applied in art creation, personalized image editing, and other fields. Existing methods suffer from low computational efficiency, weak style controllability, loss of details, and disconnection between style and content when handling high-resolution images, complex textures, and cross-modal guided tasks. To address these issues, this paper proposes CAST-Diff, a multistage image style transfer framework built on latent diffusion models. CAST-Diff follows a decoupled collaborative design that combines cross-modal semantic guidance, adaptive regional style regulation, and latent space diffusion refinement so that text and image features can be aligned through cross-modal attention, regional style strength can be adjusted by the adaptive module, and denoising together with detail reconstruction can be completed by the latent diffusion model. Experimental comparisons with mainstream methods such as StyleGAN-Diffusion and ControlNet on the COCO and Flickr30k datasets show that CAST-Diff performs better in style consistency, detail fidelity, and visual naturalness while preserving image structure and fine textures in complex scenes, producing more natural and realistic style transfer results, and improving computational efficiency as well as generalization ability. These advantages make CAST-Diff a practical solution for text-guided high-precision style transfer.
- 参考文献/References:
-
[1] YOSHIOKA D, YASUDA Y, TODA T. Nonparallel spoken-text-style transfer for linguistic expression control in speech generation[J]. IEEE transactions on audio, speech and language processing, 2025, 33: 333-346
[2] LI Xiangtian, CAO Han, ZHANG Zhaoyang, et al. Artistic neural style transfer algorithms with activation smoothing[C]//Proceedings of the 2025 2nd International Conference on Informatics Education and Computer Technology Applications. New York: ACM, 2025: 1-6.
[3] WANG Zhizhong, ZHANG Zhanjie, ZHAO Lei, et al. AesUST: towards aesthetic-enhanced universal style transfer[C]//Proceedings of the 30th ACM International Conference on Multimedia. New York: ACM, 2022: 1095-1106.
[4] WANG Quan, LI Sheng, WANG Zichi, et al. Multi-source style transfer via style disentanglement network[J]. IEEE transactions on multimedia, 2024, 26: 1373-1383
[5] JING Yongcheng, YANG Yezhou, FENG Zunlei, et al. Neural style transfer: a review[J]. IEEE transactions on visualization and computer graphics, 2020, 26(11): 3365-3385
[6] CHEN Haibo, ZHAO Lei, WANG Zhizhong, et al. Artistic style transfer with internal-external learning and contrastive learning[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. New York: ACM, 2021: 26561-26573.
[7] DENG Yingying, TANG Fan, DONG Weiming, et al. StyTr2: image style transfer with transformers[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 11316-11326.
[8] DENG Yingying, TANG Fan, DONG Weiming, et al. Arbitrary style transfer via multi-adaptation network[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 2719-2727.
[9] GATYS L A, ECKER A S, BETHGE M. A neural algorithm of artistic style[EB/OL]. (2015-08-26)[2025-01-01]. https://arxiv.org/abs/1508.06576.
[10] HUANG X, BELONGIE S. Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice: ICCV, 2017: 1501-1510.
[11] LI Yijun, FANG Chen, YANG Jimei, et al. Universal style transfer via feature transforms[C]//Advances in Neural Information Processing Systems. Long Beach: NIPS, 2017: 386-396.
[12] ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 10674-10685.
[13] ZHANG Yuxin, TANG Fan, DONG Weiming, et al. Domain enhanced arbitrary image style transfer via contrastive learning[C]//Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings. New York: ACM, 2022: 1-8.
[14] LIU Songhua, LIN Tianwei, HE Dongliang, et al. AdaAttN: revisit attention mechanism in arbitrary neural style transfer[C]//2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 6629-6638.
[15] 贵向泉, 李琪, 李立, 等. 多尺度感知的单文本条件图像风格迁移[J]. 计算机技术与发展, 2025, 35(9): 46-54 GUI Xiangquan, LI Qi, LI Li, et al. Single text conditional image style transfer for multi-scale perception[J]. Computer technology and development, 2025, 35(9): 46-54
[16] 董心悦, 傅鹏. 基于改进生成对抗网络的人脸图像风格迁移方法[J]. 鄂州大学学报, 2025, 32(2): 94-97 DONG Xinyue, FU Peng. Face image style transfer method based on improved generative adversarial network[J]. Journal of Ezhou University, 2025, 32(2): 94-97
[17] CHEN Haibo, ZHAO Lei , WANG Zhizhong, et al. Artistic style transfer with internal-external learning and contrastive learning[J]. Advances in neural information processing systems, 2021, 34: 26561-26573
[18] 雷松林, 赵征鹏, 阳秋霞, 等. 基于可解耦扩散模型的零样本风格迁移[J]. 图学学报, 2025, 46(4): 727-738 LEI Songlin, ZHAO Zhengpeng, YANG Qiuxia, et al. Zero-shot style transfer based on decoupled diffusion models[J]. Journal of graphics, 2025, 46(4): 727-738
[19] WANG Zhizhong, ZHAO Lei, ZUO Zhiwen, et al. MicroAST: towards super-fast ultra-resolution arbitrary style transfer[J]. Proceedings of the AAAI conference on artificial intelligence, 2023, 37(3): 2742-2750
[20] MOU Chong, WANG Xintao, XIE Liangbin, et al. T2I-adapter: learning adapters to dig out more controllable ability for text-to-image diffusion models[J]. Proceedings of the AAAI conference on artificial intelligence, 2024, 38(5): 4296-4304
[21] LEI Mingkun, SONG Xue, ZHU Beier, et al. StyleStudio: text-driven style transfer with selective control of style elements[EB/OL]. (2024-12-11)[2025-01-01]. https://arxiv.org/abs/2412.08503.
[22] GAO Xiang, ZHANG Yuqi. SRAGAN: saliency regularized and attended generative adversarial network for Chinese ink-wash painting style transfer[J]. Pattern recognition, 2025, 162: 111344
[23] PATASHNIK O, COHEN A, SHECHTMAN E. StyleCLIP: text-driven manipulation of StyleGAN imagery[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2021: 2085-2094.
[24] GAL R, PATASHNIK O, COHEN-OR D. StyleGAN-NADA: CLIP-guided domain adaptation of image generators[EB/OL]. (2021-08-03)[2025-01-01]. https://arxiv.org/abs/2108.00946.
[25] ZHANG Lyumin, RAO Anyi, AGRAWALA M. Adding conditional control to text-to-image diffusion models[C]//2023 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2024: 3813-3824.
[26] AN Jie, HUANG Siyu, SONG Yibing, et al. ArtFlow: unbiased image style transfer via reversible neural flows[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 862-871.
[27] YU Zhenyu, WANG Jinnian, CHEN Hanqing, et al. QRS-trs: style transfer-based image-to-image translation for carbon stock estimation in quantitative remote sensing[J]. IEEE access, 2025, 13: 52726-52737
[28] ZHANG Yuxin, HUANG Nisha, TANG Fan, et al. Inversion-based style transfer with diffusion models[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 10146-10156.
[29] PARK D Y, LEE K H. Arbitrary style transfer with style-attentional networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5873-5881.
[30] SOHN K, RUIZ N, LEE K, et al. StyleDrop: text-to-image generation in any style[EB/OL]. (2023-06-01)[2025-01-01]. https://arxiv.org/abs/2306.00983.
[31] 王伟, 张静宜, 温玉辉, 等. 基于神经网络的图像风格迁移算法综述[J]. 电子学报, 2025, 53(5): 1692-1712 WANG Wei, ZHANG Jingyi, WEN Yuhui, et al. Neural network based image style transfer: a survey[J]. Acta electronica sinica, 2025, 53(5): 1692-1712
- 相似文献/References:
-
[1]曲海成,徐波.基于自适应图学习权重的多模态情感分析[J].智能系统学报,2025,20(2):516.[doi:10.11992/tis.202401001]
QU Haicheng,XU Bo.Multimodal sentiment analysis based on adaptive graph learning weight[J].CAAI Transactions on Intelligent Systems,2025,20():516.[doi:10.11992/tis.202401001]
[2]孟想,王博岳,高祎菡,等.基于视觉-语言关键线索挖掘的多模态假新闻检测模型[J].智能系统学报,2026,21(1):109.[doi:10.11992/tis.202505007]
MENG Xiang,WANG Boyue,GAO Yihan,et al.Visual-language key clue discovery-based multimodal fake news detection model[J].CAAI Transactions on Intelligent Systems,2026,21():109.[doi:10.11992/tis.202505007]
备注/Memo
收稿日期:2025-8-9。
基金项目:南通市自然科学基金面上项目(JC2025090).
作者简介:仇佳庆，硕士研究生，主要研究方向为多媒体、计算机视觉。E-mail：983576605@qq.com。;窦立云，讲师，博士，主要研究方向为图像处理、图像篡改，先后担任了ACMMM、CVPR、TMM和TKDE等国际顶会顶刊的审稿人。E-mail：liyun_dou@163.com。;王进，副教授，博士，中国计算机学会高级会员，南通市人工智能学会副理事长，主要研究方向为人工智能、计算机视觉，中国计算机学会(CCF)高级会员，南通市人工智能学会副理事长。发表学术论文40余篇，申请授权专利30余件。 E-mail：wj@ntu.edu.cn。
通讯作者:窦立云. E-mail：Liyun_dou@163.com
更新日期/Last Update:
1900-01-01