[1]潘莉文,王骏,施俊,等.基于条件扩散模型的双阶段恶劣天气图像复原[J].智能系统学报,2026,21(4):943-951.[doi:10.11992/tis.202509039]
PAN Liwen,WANG Jun,SHI Jun,et al.Two-stage adverse weather image restoration based on a conditional diffusion model[J].CAAI Transactions on Intelligent Systems,2026,21(4):943-951.[doi:10.11992/tis.202509039]
点击复制
PAN Liwen,WANG Jun,SHI Jun,et al.Two-stage adverse weather image restoration based on a conditional diffusion model[J].CAAI Transactions on Intelligent Systems,2026,21(4):943-951.[doi:10.11992/tis.202509039]
基于条件扩散模型的双阶段恶劣天气图像复原
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第4期
页码:
943-951
栏目:
学术论文—机器感知与模式识别
出版日期:
2026-07-05
- Title:
- Two-stage adverse weather image restoration based on a conditional diffusion model
- Keywords:
- deep learning; adverse weather image restoration; image restoration; diffusion model; wavelet transform; feature fusion; multi-scale features; encoder-decoder structure
- 分类号:
- TP391
- 摘要:
- 为了解决恶劣天气下图像复原任务中普遍存在的细节模糊、纹理丢失和显存占用高等问题，本研究提出了一种基于条件扩散模型的双阶段图像复原方法。该方法首先构建了基于双分支特征融合模块的编码器-解码器结构，充分利用卷积操作的局部感知能力与Transformer架构的全局依赖建模优势，实现对退化图像的初步重建。该阶段同时引入小波变换，有效提取图像的多尺度频率特征，在显著降低计算复杂度的前提下保持结构信息的完整性。将条件扩散模型嵌入编码-解码网络之间，以前一阶段输出作为结构条件，通过扩散过程的迭代去噪机制实现对纹理与细节的增强。该模块通过隐空间中的概率映射，引导退化图像稳定逼近真实图像分布，最终实现视觉感知质量显著提升的复原结果。实验结果表明，与若干主流图像复原方法相比，本文方法在去雨、去雪任务上均表现出优异的性能，在各数据集上的峰值信噪比（peak signal-to-noise ratio, PSNR）分别提升了0.06~1.13 dB 和 0.23~5.72 dB；同时，所提方法在模型参数量与显存占用之间取得了良好的平衡。综上所述，本研究为恶劣天气下的图像复原任务提供了一种全新、有效且可靠的解决方案。
- Abstract:
- To address common challenges in image restoration under inclement weather, such as blurred details, texture loss, and high video memory usage, this study proposes a two-stage image restoration method based on a conditional diffusion model. The first stage constructs an encoder–decoder architecture utilizing a dual-branch feature fusion module. This architecture leverages the local perception capabilities of convolutional operations and the global dependency modeling advantages of the transformer architecture to achieve preliminary reconstruction of degraded images. This stage also incorporates a wavelet transform to effectively extract multiscale frequency features, preserving structural information integrity while considerably reducing computational complexity. The second stage embeds a conditional diffusion model between the encoder and decoder networks, using the output of the previous stage as a structural condition. Texture and detail are enhanced through an iterative denoising mechanism via a diffusion process. This module guides the degraded image to steadily approximate the true image distribution through probabilistic mapping in the latent space, ultimately achieving restoration results with considerably improved visual quality. Experimental results demonstrate that our method outperforms several mainstream image restoration methods in rain and snow removal tasks, improving the peak signal-to-noise ratio by 0.06~1.13 and 0.23~5.72 dB, respectively, across various datasets. The proposed method also achieves a good balance between model parameter count and graphics memory usage. In short, this research provides a novel, effective, and reliable solution for image restoration in inclement weather.
- 参考文献/References:
-
[1] ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6230-6239.
[2] 谭睿俊, 赵志诚, 谢新林. 双层残差语义分割网络及交通场景应用[J]. 智能系统学报, 2022, 17(4): 780-787 TAN Ruijun, ZHAO Zhicheng, XIE Xinlin. Double-residual semantic segmentation network and traffic scenic application[J]. CAAI transactions on intelligent systems, 2022, 17(4): 780-787
[3] 陆军, 鲁林超, 翟晓阳, 等. 面向道路交通场景的高效3D目标检测[J]. 智能系统学报, 2025, 20(1): 91-100 LU Jun, LU Linchao, ZHAI Xiaoyang, et al. High-efficiency 3D object detection for road traffic scenes[J]. CAAI transactions on intelligent systems, 2025, 20(1): 91-100
[4] 贾亮, 戚丽瑾, 林铭文, 等. 改进的YOLOv7轻量化目标检测模型[J]. 计算机工程与设计, 2025, 46(8): 2350-2357 JIA Liang, QI Lijin, LIN Mingwen, et al. Improved YOLOv7 lightweight target detection model[J]. Computer engineering and design, 2025, 46(8): 2350-2357
[5] HU Yihan, YANG Jiazhi, CHEN Li, et al. Planning-oriented autonomous driving[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 17853-17862.
[6] 吕家威, 吕丹璇. 多模态交互技术在自动驾驶舱环境中的应用[J]. 汽车电器, 2025(9): 13-15 L? Jiawei, L? Danxuan. Application of multimodal interaction technology in automatic cockpit environment[J]. Auto electric parts, 2025(9): 13-15
[7] 王美华, 柯凡晖, 梁云, 等. 融合3D注意力和Transformer的图像去雨网络[J]. 中国图象图形学报, 2022, 27(5): 1509-1521 WANG Meihua, KE Fanhui, LIANG Yun, et al. 3D attention and Transformer based single image deraining network[J]. Journal of image and graphics, 2022, 27(5): 1509-1521
[8] JIANG Kui, WANG Zhongyuan, YI Peng, et al. Multi-scale progressive fusion network for single image deraining[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 8343-8352.
[9] LIANG Jingyun, CAO Jiezhang, SUN Guolei, et al. SwinIR: image restoration using swin transformer[C]//2021 IEEE/CVF International Conference on Computer Vision Workshops. Montreal: IEEE, 2021: 1833-1844.
[10] ZAMIR S W, ARORA A, KHAN S, et al. Restormer: efficient transformer for high-resolution image restoration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 5718-5729.
[11] WANG Zhendong, CUN Xiaodong, BAO Jianmin, et al. Uformer: a general U-shaped transformer for image restoration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 17662-17672.
[12] HE Kaiming, SUN Jian, TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 33(12): 2341-2353
[13] 王平, 张云峰, 包芳勋, 等. 基于雾天图像降质模型的优化去雾方法[J]. 中国图象图形学报, 2018, 23(4): 605-616 WANG Ping, ZHANG Yunfeng, BAO Fangxun, et al. Multivariate optimized haze removal method based on hazy image degradation model[J]. Journal of image and graphics, 2018, 23(4): 605-616
[14] ZHU Lei, FU C W, LISCHINSKI D, et al. Joint bi-layer optimization for single-image rain streak removal[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2545-2553.
[15] YOU Jing, JIA Shaocheng, PEI Xin, et al. DMRVisNet: deep multihead regression network for pixel-wise visibility estimation under foggy weather[J]. IEEE transactions on intelligent transportation systems, 2022, 23(11): 22354-22366
[16] DONG Chao, LOY C C, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]//Computer Vision-ECCV 2014. Cham: Springer, 2014: 184-199.
[17] ZHANG Kai, ZUO Wangmeng, CHEN Yunjin, et al. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising[J]. IEEE transactions on image processing, 2017, 26(7): 3142-3155
[18] VALANARASU J M J, YASARLA R, PATEL V M. Transweather: Transformer based restoration of images degraded by adverse weather conditions[C]//2022 IEEE Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 2353-2363.
[19] LIU Pengju, ZHANG Hongzhi, ZHANG Kai, et al. Multi-level wavelet-CNN for image restoration[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Boston: IEEE, 2018: 886-88609.
[20] 王维, 王习博, 廖万正, 等. 基于多尺度小波变换的雨天列车外观图像恢复[J/OL]. 北京: 自动化技术与应用, 2025. (2025-08-21)[2025-09-21]. https://link.cnki.net/urlid/23.1474.tp.20250821.1548.024. WANG Wei, WANG Xibo, LIAO Wanzheng, et al. R-ainy train appearance image restoration based on mul-tiscale wavelet transform[J/OL]. Automation technology and application, 2025. (2025-08-21)[2025-09-21]. https://link.cnki.net/urlid/23.1474.tp.20250821.1548.024.
[21] ?ZDENIZCI O, LEGENSTEIN R. Restoring vision in adverse weather conditions with patch-based denoising diffusion models[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(8): 10346-10357
[22] HUANG Yi, HUANG Jiancheng, LIU Jianzhuang, et al. WaveDM: wavelet-based diffusion models for image restoration[J]. IEEE transactions on multimedia, 2024, 26: 7058-7073
[23] 管凤旭, 张涵宇, 路斯棋, 等. 扩散模型在计算机视觉领域的研究现状[J]. 智能系统学报, 2025, 20(2): 265-282 GUAN Fengxu, ZHANG Hanyu, LU Siqi, et al. Research status of diffusion models in computer vision[J]. CAAI transactions on intelligent systems, 2025, 20(2): 265-282
[24] QIAN Rui, TAN R T, YANG Wenhan, et al. Attentive generative adversarial network for raindrop removal from a single image[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2482-2491.
[25] LIU Yunfu, JAW D W, HUANG S C, et al. DesnowNet: context-aware deep network for snow removal[J]. IEEE transactions on image processing, 2018, 27(6): 3064-3073
[26] HOR? A, ZIOU D. Image quality metrics: PSNR vs. SSIM[C]//2010 20th International Conference on Pattern Recognition. Istanbul: IEEE, 2010: 2366-2369.
[27] LIU Xing, SUGANUMA M, SUN Zhun, et al. Dual residual networks leveraging the potential of paired operations for image restoration[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 7000-7009.
[28] QUAN Ruijie, YU Xin, LIANG Yuanzhi, et al. Removing raindrops and rain streaks in one go[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 9143-9152.
[29] QUAN Yuhui, DENG Shijie, CHEN Yixin, et al. Deep learning for seeing through window with raindrops[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2020: 2463-2471.
[30] XIAO Jie, FU Xueyang, LIU Aiping, et al. Image de-raining transformer[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(11): 12978-12995
[31] GUO Hang, LI Jinmin, DAI Tao, et al. MambaIR: a simple baseline forImage restoration withState-space model[C]//Computer Vision-ECCV 2024. Cham: Springer, 2025: 222-241.
[32] CHEN Weiting, FANG Haoyu, DING J J, et al. JSTASR: joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal[C]//Computer Vision-ECCV 2020. Cham: Springer, 2020: 754-770.
[33] ZHANG Kaihao, LI Rongqing, YU Yanjiang, et al. Deep dense multi-scale network for snow removal using semantic and depth priors[J]. IEEE transactions on image processing, 2021, 30: 7419-7431
[34] CHU Xiaojie, CHEN Liangyu, YU Wenqing. NAFSSR: stereo image super-resolution using NAFNet[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New Orleans: IEEE, 2022: 1238-1247.
[35] LI Ruoteng, TAN R T, CHEONG L F. All in one bad weather removal using architectural search[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 3172-3182.
[36] CHEN Weiting, HUANG Zhikai, TSAI C C, et al. Learning multiple adverse weather removal via two-stage knowledge learning and multi-contrastive regularization: toward a unified model[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 17632-17641.
[37] ZHU Yurui, WANG Tianyu, FU Xueyang, et al. Learning weather-general and weather-specific features for image restoration under multiple adverse weather conditions[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 21747-21758.
[38] Wu Y, Kirillov A, Massa F, e tal. Detectron2[J/OL]. Facebook AI research, 2019. (2019-10-28)[2025-09-25]. https://github.com/facebookresearch/detectron2.
- 相似文献/References:
-
[1]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[2]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[3]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[4]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[5]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[6]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[7]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
[8]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[9]杨梦铎,栾咏红,刘文军,等.基于自编码器的特征迁移算法[J].智能系统学报,2017,12(6):894.[doi:10.11992/tis.201706037]
YANG Mengduo,LUAN Yonghong,LIU Wenjun,et al.Feature transfer algorithm based on an auto-encoder[J].CAAI Transactions on Intelligent Systems,2017,12():894.[doi:10.11992/tis.201706037]
[10]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
备注/Memo
收稿日期:2025-9-30。
基金项目:国家自然科学基金项目 (62301306); 上海市科技创新行动计划项目 (23ZR1422200, 23YF1412800).
作者简介:潘莉文，硕士研究生，主要研究方向为恶劣天气图像复原。E-mail：15779365007@163.com。;王骏，教授，博士，主要研究方向为机器学习、人工智能和医学影像智能处理。获江苏省高校科研成果自然科学一等奖、中国产学研创新成果一等奖。发表学术论文160余篇，谷歌学术引用6000余次。E-mail：wangjunshu@shu.edu.cn。;李俊诚，副教授，博士，主要研究方向为底层视觉和医学影像智能处理。上海市青年科技英才扬帆计划获得者，获中国产学研合作创新成果一等奖。发表学术论文70余篇，谷歌学术引用5000余次。E-mail：cvjunchengli@gmail.com。
通讯作者:李俊诚. E-mail：cvjunchengli@gmail.com
更新日期/Last Update:
1900-01-01