[1]王卫苹,刁亚鹏.融合金字塔和多尺度注意力的多曝光图像融合优化算法[J].智能系统学报,2025,20(4):916-927.[doi:10.11992/tis.202406032]
WANG Weiping,DIAO Yapeng.Optimized multi-exposure image fusion algorithm integrating pyramid and multi-scale attention[J].CAAI Transactions on Intelligent Systems,2025,20(4):916-927.[doi:10.11992/tis.202406032]
点击复制
WANG Weiping,DIAO Yapeng.Optimized multi-exposure image fusion algorithm integrating pyramid and multi-scale attention[J].CAAI Transactions on Intelligent Systems,2025,20(4):916-927.[doi:10.11992/tis.202406032]
融合金字塔和多尺度注意力的多曝光图像融合优化算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
20
期数:
2025年第4期
页码:
916-927
栏目:
学术论文—机器学习
出版日期:
2025-08-05
- Title:
- Optimized multi-exposure image fusion algorithm integrating pyramid and multi-scale attention
- Keywords:
- image fusion; attention mechanism; pyramid network; multi-exposure; detail extraction; color correction; multi-scale; deep learning
- 分类号:
- TP181
- 文献标志码:
- 2025-2-21
- 摘要:
- 为解决复杂光照条件下真实场景中导致的图像噪声、模糊和细节丢失问题，本文提出一种多曝光图像融合技术DPEPA-MEF（deep pyramid exposure pyramid attention-multi-exposure fusion）。该方法通过有效结合不同曝光水平的图像，解决高对比度、低光照以及色彩和亮度平衡等问题。该方法由3个模块组成，对DPE（deep pyramid exposure）进行改进。第1个模块专注于内容细节提取，第2个模块负责色彩映射和校正，第3个模块利用多尺度特征金字塔进行图像恢复。实验结果表明，在不同的光照条件和动态场景下，DPEPA-MEF能够更有效地融合多张曝光图像，生成的图像在细节保留、色彩还原和对比度等方面表现出色。通过定量评估指标和主观视觉评估，DPEPA-MEF均显示出显著的优势，证明了该方法在实际应用中的巨大潜力和优越性。
- Abstract:
- The article introduces a multi-exposure image fusion technique named DPEPA-MEF (deep pyramid exposure pyramid attention-multi-exposure fusion). This technique aims to address image noise, blur, and detail loss caused by complex lighting conditions in real scenes. The DPEPA-MEF method effectively combines images with varying exposure levels to solve issues such as high contrast, low light, and color and brightness balance. It consists of three modules, which improve upon deep pyramid exposure (DPE). The first module focuses on content detail extraction, the second on color mapping and correction, and the third on image recovery using a multi-scale feature pyramid. Experimental results indicate that under different lighting conditions and dynamic scenes, DPEPA-MEF can more effectively fuse multiple exposure images. The resulting images exhibit excellent detail preservation, color reproduction, and contrast. Both quantitative evaluation metrics and subjective visual assessments demonstrate the significant advantages of DPEPA-MEF, confirming its great potential and superiority in practical applications.
- 参考文献/References:
-
[1] YI Xunpeng, TANG Linfeng, ZHANG Hao, et al. Diff-IF: Multi-modality image fusion via diffusion model with fusion knowledge prior[J]. Information fusion, 2024, 110: 102450.
[2] ZHANG Xingchen. Benchmarking and comparing multi-exposure image fusion algorithms[J]. Information fusion, 2021, 74: 111-131.
[3] LEI Dajiang, RAN Gangsheng, ZHANG Liping, et al. A spatiotemporal fusion method based on multiscale feature extraction and spatial channel attention mechanism[J]. Remote sensing, 2022, 14(3): 461.
[4] LIAN Zuozheng, WANG Haizhen. An image deblurring method using improved U-Net model based on multilayer fusion and attention mechanism[J]. Scientific reports, 2023, 13: 21402.
[5] HAN Dong, LI Liang, GUO Xiaojie, et al. Multi-exposure image fusion via deep perceptual enhancement[J]. Information fusion, 2022, 79: 248-262.
[6] WANG Zhao, LI Feng, CONG Runmin, et al. Adaptive feature fusion network based on boosted attention mechanism for single image dehazing[J]. Multimedia tools and applications, 2022, 81(8): 11325-11339.
[7] ZHANG Jianming, FENG Wenjun, YUAN Tingyu, et al. SCSTCF: spatial-channel selection and temporal regularized correlation filters for visual tracking[J]. Applied soft computing, 2022, 118: 108485.
[8] CAI Jianrui, GU Shuhang, ZHANG Lei. Learning a deep single image contrast enhancer from multi-exposure images[J]. IEEE transactions on image processing, 2018, 27(4): 2049-2062.
[9] LI Weiqiang, MU Jiatong, LIU Guizhong. Multiple object tracking with motion and appearance cues[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop. Seoul: IEEE, 2019: 161–169.
[10] NIU Wenjia, ZHANG Kaihao, LUO Wenhan, et al. Deep robust image deblurring via blur distilling and information comparison in latent space[J]. Neurocomputing, 2021, 466: 69-79.
[11] LI Hui, MA Kede, YONG Hongwei, et al. Fast multi-scale structural patch decomposition for multi-exposure image fusion[J]. IEEE transactions on image processing, 2020, 29: 5805-5816.
[12] SHAO Mingwen, LI Le, MENG Deyu, et al. Uncertainty guided multi-scale attention network for raindrop removal from a single image[J]. IEEE transactions on image processing, 2021, 30: 4828-4839.
[13] LEE K H, CHEN Xi, HUA Gang, et al. Stacked cross attention for image-text matching[C]//Proceedings of the European conference on computer vision. New York: Springer, 2018: 201-216.
[14] KRISHNAMOORTHI R, BAI A, SRINIVAS A. Medical image fusion based on transformation domain approaches[C]//IOP Conference Series: Materials Science and Engineering. [S. l.]: IOP Publishing, 2020, 981(4): 042082.
[15] DENG Xin, ZHANG Yutong, XU Mai, et al. Deep coupled feedback network for joint exposure fusion and image super-resolution[J]. IEEE transactions on image processing, 2021, 30: 3098-3112.
[16] ZHANG Weixia, MA Kede, ZHAI Guangtao, et al. Uncertainty-aware blind image quality assessment in the laboratory and wild[J]. IEEE transactions on image processing, 2021, 30: 3474-3486.
[17] CHEN Tianhai, YANG Xichen, LI Nengxin, et al. Underwater image quality assessment method based on color space multi-feature fusion[J]. Scientific reports, 2023, 13: 16838.
[18] PAN Zhaoqing, YUAN Feng, LEI Jianjun, et al. VCRNet: visual compensation restoration network for No-reference image quality assessment[J]. IEEE transactions on image processing, 2022, 31: 1613-1627.
[19] PATHAN R K, BISWAS M, YASMIN S, et al. Sign language recognition using the fusion of image and hand landmarks through multi-headed convolutional neural network[J]. Scientific reports, 2023, 13(1): 16975.
[20] HOFFMANN M, BILLOT B, GREVE D N, et al. SynthMorph: learning contrast-invariant registration without acquired images[J]. IEEE transactions on medical imaging, 2022, 41(3): 543-558.
[21] MAR C C, ZIN T T, TIN P, et al. Cow detection and tracking system utilizing multi-feature tracking algorithm[J]. Scientific reports, 2023, 13(1): 17423.
[22] TANG Linfeng, ZHANG Hao, XU Han, et al. Rethinking the necessity of image fusion in high-level vision tasks: a practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity[J]. Information fusion, 2023, 99: 101870.
[23] 刘万军, 佟畅, 曲海成. 空洞卷积与注意力融合的对抗式图像阴影去除算法[J]. 智能系统学报, 2021, 16(6): 1081-1089.
LIU Wanjun, TONG Chang, QU Haicheng. An antagonistic image shadow removal algorithm based on dilated convolution and attention mechanism[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1081-1089.
[24] ZHANG Hao, XU Han, TIAN Xin, et al. Image fusion meets deep learning: a survey and perspective[J]. Information fusion, 2021, 76: 323-336.
[25] QIU Sen, ZHAO Hongkai, JIANG Nan, et al. Multi-sensor information fusion based on machine learning for real applications in human activity recognition: State-of-the-art and research challenges[J]. Information fusion, 2022, 80: 241-265.
[26] 王建, 吴锡生. 基于改进的稀疏表示和PCNN的图像融合算法研究[J]. 智能系统学报, 2019, 14(5): 922-928.
WANG Jian, WU Xisheng. Image fusion based on the improved sparse representation and PCNN[J]. CAAI transactions on intelligent systems, 2019, 14(5): 922-928.
[27] 姜义, 吕荣镇, 刘明珠, 等. 基于生成对抗网络的人脸口罩图像合成[J]. 智能系统学报, 2021, 16(6): 1073-1080.
JIANG Yi, LYU Rongzhen, LIU Mingzhu, et al. Masked face image synthesis based on a generative adversarial network[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1073-1080.
[28] 张心祎, 谭耀, 邢向磊. 基于物理先验的深度特征融合水下图像复原[J]. 智能系统学报, 2023, 18(6): 1185-1196.
ZHANG Xinyi, TAN Yao, XING Xianglei. Deep feature fusion for underwater-image restoration based on physical priors[J]. CAAI transactions on intelligent systems, 2023, 18(6): 1185-1196.
[29] LI Hui, WU Xiaojun, KITTLER J. RFN-Nest: an end-to-end residual fusion network for infrared and visible images[J]. Information fusion, 2021, 73: 72-86.
[30] TANG Linfeng, YUAN Jiteng, MA Jiayi. Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network[J]. Information fusion, 2022, 82: 28-42.
[31] QUAN Xiongwen, HOU Guangyao, YIN Wenya, et al. A multi-modal and multi-stage fusion enhancement network for segmentation based on OCT and OCTA images[J]. Information fusion, 2025, 113: 102594.
[32] HE Xiaoyu, WANG Yong, ZHAO Shuang, et al. Co-attention fusion network for multimodal skin cancer diagnosis[J]. Pattern recognition, 2023, 133: 108990.
[33] LIU Yun, ZHANG Xiaoming, ZHANG Qianyun, et al. Dual self-attention with co-attention networks for visual question answering[J]. Pattern recognition, 2021, 117: 107956.
[34] XIANG Fengtao, JIAN Zhang, LIANG Pan, et al. Robust image fusion with block sparse representation and online dictionary learning[J]. IET image processing, 2018, 12(3): 345-353.
- 相似文献/References:
-
[1]李洋,焦淑红,孙新童.基于IHS和小波变换的可见光与红外图像融合[J].智能系统学报,2012,7(6):554.
LI Yang,JIAO Shuhong,SUN Xintong.Fusion of visual and infrared images based on IHS and wavelet transforms[J].CAAI Transactions on Intelligent Systems,2012,7():554.
[2]任晓霞,孙秀明,耿鹏,等.多小波和NSDFB组合域递归滤波多聚焦图像融合[J].智能系统学报,2016,11(2):241.[doi:10.11992/tis.201509017]
REN Xiaoxia,SUN Xiuming,GENG Peng,et al.Multifocus image fusion using a recursive filter in the combined domain of multiwavelets and NSDFB[J].CAAI Transactions on Intelligent Systems,2016,11():241.[doi:10.11992/tis.201509017]
[3]王建,吴锡生.基于改进的稀疏表示和PCNN的图像融合算法研究[J].智能系统学报,2019,14(5):922.[doi:10.11992/tis.201805045]
WANG Jian,WU Xisheng.Image fusion based on the improved sparse representation and PCNN[J].CAAI Transactions on Intelligent Systems,2019,14():922.[doi:10.11992/tis.201805045]
[4]申凯,王晓峰,杨亚东.基于双向消息链路卷积网络的显著性物体检测[J].智能系统学报,2019,14(6):1152.[doi:10.11992/tis.201812003]
SHEN Kai,WANG Xiaofeng,YANG Yadong.Salient object detection based on bidirectional message link convolution neural network[J].CAAI Transactions on Intelligent Systems,2019,14():1152.[doi:10.11992/tis.201812003]
[5]赵文清,程幸福,赵振兵,等.注意力机制和Faster RCNN相结合的绝缘子识别[J].智能系统学报,2020,15(1):92.[doi:10.11992/tis.201907023]
ZHAO Wenqing,CHENG Xingfu,ZHAO Zhenbing,et al.Insulator recognition based on attention mechanism and Faster RCNN[J].CAAI Transactions on Intelligent Systems,2020,15():92.[doi:10.11992/tis.201907023]
[6]申翔翔,侯新文,尹传环.深度强化学习中状态注意力机制的研究[J].智能系统学报,2020,15(2):317.[doi:10.11992/tis.201809033]
SHEN Xiangxiang,HOU Xinwen,YIN Chuanhuan.State attention in deep reinforcement learning[J].CAAI Transactions on Intelligent Systems,2020,15():317.[doi:10.11992/tis.201809033]
[7]曾碧卿,韩旭丽,王盛玉,等.层次化双注意力神经网络模型的情感分析研究[J].智能系统学报,2020,15(3):460.[doi:10.11992/tis.201812017]
ZENG Biqing,HAN Xuli,WANG Shengyu,et al.Hierarchical double-attention neural networks for sentiment classification[J].CAAI Transactions on Intelligent Systems,2020,15():460.[doi:10.11992/tis.201812017]
[8]莫宏伟,田朋.基于注意力融合的图像描述生成方法[J].智能系统学报,2020,15(4):740.[doi:10.11992/tis.201910039]
MO Hongwei,TIAN Peng.An image caption generation method based on attention fusion[J].CAAI Transactions on Intelligent Systems,2020,15():740.[doi:10.11992/tis.201910039]
[9]鲍维克,袁春.面向推荐系统的分期序列自注意力网络[J].智能系统学报,2021,16(2):353.[doi:10.11992/tis.202005028]
BAO Weike,YUAN Chun.Recommendation system with long-term and short-term sequential self-attention network[J].CAAI Transactions on Intelligent Systems,2021,16():353.[doi:10.11992/tis.202005028]
[10]洪恺临,曹江涛,姬晓飞.改进Center-Net网络的自主喷涂机器人室内窗户检测[J].智能系统学报,2021,16(3):425.[doi:10.11992/tis.202005016]
HONG Kailin,CAO Jiangtao,JI Xiaofei.Indoor window detection of autonomous spraying robot based on improved CenterNet network[J].CAAI Transactions on Intelligent Systems,2021,16():425.[doi:10.11992/tis.202005016]
备注/Memo
收稿日期:2024-6-21。
基金项目:工业和信息化部高质量发展专项（TC220A04A-182）； 国 家自然科学基金项目（62271045）；国家国际科技合作重点项目 （2022YFE0112300）；北京科技大学青年教师学科交叉研究项目（FRF-IDRY-23-027）；CCF-绿盟科技“鲲鹏”科研基金项目（CCF-NSFOCUS 2024017）.
作者简介:王卫苹，教授，主要研究方向为类脑计算、忆阻神经网络、联想记忆感知模拟、复杂网络、网络安全和图像加密。主持和参与国家自然科学基金项目、国际联合基金重点项目等多项，获得中国通信学会最佳论文奖、河南省科技进步二等奖等荣誉。发表学术论文90余篇，拥有软著与专利20余项。E-mail：weipingwangjt@ustb.edu.cn。;刁亚鹏，硕士研究生，主要研究方向为类脑计算、脑机接口和图像处理领域。E-mail：ypdiao@xs.ustb.edu.cn。
通讯作者:王卫苹. E-mail：weipingwangjt@ustb.edu.cn
更新日期/Last Update:
1900-01-01