[1]QIU Jiaqing,DOU Liyun,WANG Jin.Multistage image style transfer based on cross-modal attention[J].CAAI Transactions on Intelligent Systems,2026,21(3):751-762.[doi:10.11992/tis.202508011]
Copy

Multistage image style transfer based on cross-modal attention

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 21 Number of periods: 2026 3 Page number: 751-762 Column: 学术论文—智能系统 Public date: 2026-05-05
Title:
Multistage image style transfer based on cross-modal attention
Author(s):
QIU Jiaqing;  DOU Liyun;  WANG Jin
School of Artificial Intelligence and Computer Science, Nantong University, Nantong 226019, China
Keywords:
image style transfer; cross-modal attention; adaptive style modulation; latent diffusion model; image generation; text-guided image generation; multimodal generation; artistic style transfer
CLC:
TP391；TH212
DOI:
10.11992/tis.202508011
Abstract:
Image style transfer (IST) aims to fuse image content with a target artistic style to generate semantically rational and visually expressive images, and it has been widely applied in art creation, personalized image editing, and other fields. Existing methods suffer from low computational efficiency, weak style controllability, loss of details, and disconnection between style and content when handling high-resolution images, complex textures, and cross-modal guided tasks. To address these issues, this paper proposes CAST-Diff, a multistage image style transfer framework built on latent diffusion models. CAST-Diff follows a decoupled collaborative design that combines cross-modal semantic guidance, adaptive regional style regulation, and latent space diffusion refinement so that text and image features can be aligned through cross-modal attention, regional style strength can be adjusted by the adaptive module, and denoising together with detail reconstruction can be completed by the latent diffusion model. Experimental comparisons with mainstream methods such as StyleGAN-Diffusion and ControlNet on the COCO and Flickr30k datasets show that CAST-Diff performs better in style consistency, detail fidelity, and visual naturalness while preserving image structure and fine textures in complex scenes, producing more natural and realistic style transfer results, and improving computational efficiency as well as generalization ability. These advantages make CAST-Diff a practical solution for text-guided high-precision style transfer.
References:
[1] YOSHIOKA D, YASUDA Y, TODA T. Nonparallel spoken-text-style transfer for linguistic expression control in speech generation[J]. IEEE transactions on audio, speech and language processing, 2025, 33: 333-346
[2] LI Xiangtian, CAO Han, ZHANG Zhaoyang, et al. Artistic neural style transfer algorithms with activation smoothing[C]//Proceedings of the 2025 2nd International Conference on Informatics Education and Computer Technology Applications. New York: ACM, 2025: 1-6.
[3] WANG Zhizhong, ZHANG Zhanjie, ZHAO Lei, et al. AesUST: towards aesthetic-enhanced universal style transfer[C]//Proceedings of the 30th ACM International Conference on Multimedia. New York: ACM, 2022: 1095-1106.
[4] WANG Quan, LI Sheng, WANG Zichi, et al. Multi-source style transfer via style disentanglement network[J]. IEEE transactions on multimedia, 2024, 26: 1373-1383
[5] JING Yongcheng, YANG Yezhou, FENG Zunlei, et al. Neural style transfer: a review[J]. IEEE transactions on visualization and computer graphics, 2020, 26(11): 3365-3385
[6] CHEN Haibo, ZHAO Lei, WANG Zhizhong, et al. Artistic style transfer with internal-external learning and contrastive learning[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. New York: ACM, 2021: 26561-26573.
[7] DENG Yingying, TANG Fan, DONG Weiming, et al. StyTr2: image style transfer with transformers[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 11316-11326.
[8] DENG Yingying, TANG Fan, DONG Weiming, et al. Arbitrary style transfer via multi-adaptation network[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 2719-2727.
[9] GATYS L A, ECKER A S, BETHGE M. A neural algorithm of artistic style[EB/OL]. (2015-08-26)[2025-01-01]. https://arxiv.org/abs/1508.06576.
[10] HUANG X, BELONGIE S. Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice: ICCV, 2017: 1501-1510.
[11] LI Yijun, FANG Chen, YANG Jimei, et al. Universal style transfer via feature transforms[C]//Advances in Neural Information Processing Systems. Long Beach: NIPS, 2017: 386-396.
[12] ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 10674-10685.
[13] ZHANG Yuxin, TANG Fan, DONG Weiming, et al. Domain enhanced arbitrary image style transfer via contrastive learning[C]//Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings. New York: ACM, 2022: 1-8.
[14] LIU Songhua, LIN Tianwei, HE Dongliang, et al. AdaAttN: revisit attention mechanism in arbitrary neural style transfer[C]//2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 6629-6638.
[15] 贵向泉, 李琪, 李立, 等. 多尺度感知的单文本条件图像风格迁移[J]. 计算机技术与发展, 2025, 35(9): 46-54 GUI Xiangquan, LI Qi, LI Li, et al. Single text conditional image style transfer for multi-scale perception[J]. Computer technology and development, 2025, 35(9): 46-54
[16] 董心悦, 傅鹏. 基于改进生成对抗网络的人脸图像风格迁移方法[J]. 鄂州大学学报, 2025, 32(2): 94-97 DONG Xinyue, FU Peng. Face image style transfer method based on improved generative adversarial network[J]. Journal of Ezhou University, 2025, 32(2): 94-97
[17] CHEN Haibo, ZHAO Lei , WANG Zhizhong, et al. Artistic style transfer with internal-external learning and contrastive learning[J]. Advances in neural information processing systems, 2021, 34: 26561-26573
[18] 雷松林, 赵征鹏, 阳秋霞, 等. 基于可解耦扩散模型的零样本风格迁移[J]. 图学学报, 2025, 46(4): 727-738 LEI Songlin, ZHAO Zhengpeng, YANG Qiuxia, et al. Zero-shot style transfer based on decoupled diffusion models[J]. Journal of graphics, 2025, 46(4): 727-738
[19] WANG Zhizhong, ZHAO Lei, ZUO Zhiwen, et al. MicroAST: towards super-fast ultra-resolution arbitrary style transfer[J]. Proceedings of the AAAI conference on artificial intelligence, 2023, 37(3): 2742-2750
[20] MOU Chong, WANG Xintao, XIE Liangbin, et al. T2I-adapter: learning adapters to dig out more controllable ability for text-to-image diffusion models[J]. Proceedings of the AAAI conference on artificial intelligence, 2024, 38(5): 4296-4304
[21] LEI Mingkun, SONG Xue, ZHU Beier, et al. StyleStudio: text-driven style transfer with selective control of style elements[EB/OL]. (2024-12-11)[2025-01-01]. https://arxiv.org/abs/2412.08503.
[22] GAO Xiang, ZHANG Yuqi. SRAGAN: saliency regularized and attended generative adversarial network for Chinese ink-wash painting style transfer[J]. Pattern recognition, 2025, 162: 111344
[23] PATASHNIK O, COHEN A, SHECHTMAN E. StyleCLIP: text-driven manipulation of StyleGAN imagery[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2021: 2085-2094.
[24] GAL R, PATASHNIK O, COHEN-OR D. StyleGAN-NADA: CLIP-guided domain adaptation of image generators[EB/OL]. (2021-08-03)[2025-01-01]. https://arxiv.org/abs/2108.00946.
[25] ZHANG Lyumin, RAO Anyi, AGRAWALA M. Adding conditional control to text-to-image diffusion models[C]//2023 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2024: 3813-3824.
[26] AN Jie, HUANG Siyu, SONG Yibing, et al. ArtFlow: unbiased image style transfer via reversible neural flows[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 862-871.
[27] YU Zhenyu, WANG Jinnian, CHEN Hanqing, et al. QRS-trs: style transfer-based image-to-image translation for carbon stock estimation in quantitative remote sensing[J]. IEEE access, 2025, 13: 52726-52737
[28] ZHANG Yuxin, HUANG Nisha, TANG Fan, et al. Inversion-based style transfer with diffusion models[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 10146-10156.
[29] PARK D Y, LEE K H. Arbitrary style transfer with style-attentional networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5873-5881.
[30] SOHN K, RUIZ N, LEE K, et al. StyleDrop: text-to-image generation in any style[EB/OL]. (2023-06-01)[2025-01-01]. https://arxiv.org/abs/2306.00983.
[31] 王伟, 张静宜, 温玉辉, 等. 基于神经网络的图像风格迁移算法综述[J]. 电子学报, 2025, 53(5): 1692-1712 WANG Wei, ZHANG Jingyi, WEN Yuhui, et al. Neural network based image style transfer: a survey[J]. Acta electronica sinica, 2025, 53(5): 1692-1712
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems