[1]程德强,朱星光,寇旗旗,等.融合分层特征与残差蒸馏连接的图像超分辨率重建[J].智能系统学报,2023,18(6):1173-1184.[doi:10.11992/tis.202304011]
CHENG Deqiang,ZHU Xingguang,KOU Qiqi,et al.Image super-resolution reconstruction by fusing layered features with residual distillation connections[J].CAAI Transactions on Intelligent Systems,2023,18(6):1173-1184.[doi:10.11992/tis.202304011]
点击复制
CHENG Deqiang,ZHU Xingguang,KOU Qiqi,et al.Image super-resolution reconstruction by fusing layered features with residual distillation connections[J].CAAI Transactions on Intelligent Systems,2023,18(6):1173-1184.[doi:10.11992/tis.202304011]
融合分层特征与残差蒸馏连接的图像超分辨率重建
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第6期
页码:
1173-1184
栏目:
学术论文—机器学习
出版日期:
2023-11-05
- Title:
- Image super-resolution reconstruction by fusing layered features with residual distillation connections
- Keywords:
- image processing; super-resolution reconstruction; U-network; residual connectivity; neural network; feature fusion; attention mechanism; subpixel convolution
- 分类号:
- TP391
- 摘要:
- 针对目前诸多图像超分辨率重建算法通过采用单一通道网络结构无法充分利用特征信息的问题，提出了一种可以高效利用特征信息的融合分层特征与残差蒸馏连接的超分辨率重建算法。该方法首先设计了一种将分层特征融合与残差连接相结合的连接方式，对图像深层特征与浅层特征进行充分融合，提升了网络对于特征信息的利用率；其次设计出一种残差蒸馏注意力模块，使网络更高效地关注图像关键特征，从而可以更好地恢复出重建图像的细节特征。实验结果表明，所提出的算法模型不仅在4种测试集上呈现出更优秀的客观评价指标，而且在主观视觉效果上也呈现出更好的重建效果。具体在Set14测试集上，该模型4倍重建结果的峰值信噪比相对于对比模型平均提升了0.85 dB，结构相似度平均提升了0.034，充分证明了该算法模型的有效性。
- Abstract:
- Aiming at the issue that several current image super-resolution reconstruction algorithms cannot fully utilize the feature information by adopting a single-channel network structure, a super-resolution reconstruction algorithm that fuses hierarchical features and residual distillation connections is proposed. In this method, a connection method that combines layered features with residual connections is adopted to fully fuse the deep and shallow features of an image, improving the utilization of feature information by the network. Further, a residual distillation attention module is used, which enables the network to focus on the key features of the image efficiently, allowing efficient recovery of the detailed features of the reconstructed image. The experimental results showed that the proposed algorithmic model exhibits better objective evaluation indexes on four test sets and has a superior reconstruction effect on the subjective visual effect. Specifically, on the Set14 test set, the peak signal-to-noise ratio of the four-fold reconstruction results of the model is improved by 0.85 dB on average relative to the comparison model and the structural similarity is improved by 0.034 on average, demonstrating the effectiveness of the algorithmic model.
- 参考文献/References:
-
[1] CHENG Deqiang, LI Jiahan, KOU Qiqi, et al. H-net: unsupervised domain adaptation person re-identification network based on hierarchy[J]. Image and vision computing, 2022, 124: 104493.
[2] 韩璐, 毕晓君. 多尺度特征融合网络的视网膜OCT图像分类[J]. 智能系统学报, 2022, 17(2): 360–367
HAN Lu, BI Xiaojun. Retinal optical coherence tomography image classification based on multiscale feature fusion[J]. CAAI transactions on intelligent systems, 2022, 17(2): 360–367
[3] 毕晓君, 潘梦迪. 基于生成对抗网络的机载遥感图像超分辨率重建[J]. 智能系统学报, 2020, 15(1): 74–83
BI Xiaojun, PAN Mengdi. Super-resolution reconstruction of airborne remote sensing images based on the generative adversarial networks[J]. CAAI transactions on intelligent systems, 2020, 15(1): 74–83
[4] 王宇昊, 王铸. 卫星遥感影像特定目标的超分辨率重建算法[J]. 遥感信息, 2022, 37(5): 108–115
WANG Yuhao, WANG Zhu. Super-resolution reconstruction algorithm for specific target in satellite remote sensing imagery[J]. Remote sensing information, 2022, 37(5): 108–115
[5] 程德强, 陈杰, 寇旗旗, 等. 融合层次特征和注意力机制的轻量化矿井图像超分辨率重建方法[J]. 仪器仪表学报, 2022, 43(8): 73–84
CHENG Deqiang, CHEN Jie, KOU Qiqi, et al. Lightweight super-resolution reconstruction method based on hierarchical features fusion and attention mechanism for mine image[J]. Chinese journal of scientific instrument, 2022, 43(8): 73–84
[6] TAO Hongjiu, TANG Xinjian, LIU Jian, et al. Superresolution remote sensing image processing algorithm based on wavelet transform and interpolation[J]. Image processing and pattern recognition in remote sensing, 2003, 4898: 259–263.
[7] CHENG Deqiang, CHEN Liangliang, LYU Chen, et al. Light-guided and cross-fusion U-net for anti-illumination image super-resolution[J]. IEEE transactions on circuits and systems for video technology, 2022, 32(12): 8436–8449.
[8] WANG Yetong, XING Kongduo, WANG Baji, et al. Image super-resolution reconstruction method based on residual mechanism[J]. Journal of electronic imaging, 2022, 31(3): 033010.
[9] 王凡超, 丁世飞. 基于广泛激活深度残差网络的图像超分辨率重建[J]. 智能系统学报, 2022, 17(2): 440–446
WANG Fanchao, DING Shifei. Image super-resolution reconstruction based on widely activated deep residual networks[J]. CAAI transactions on intelligent systems, 2022, 17(2): 440–446
[10] YANG Shuyuan, LIU Zhizhou, WANG Min, et al. Multitask dictionary learning and sparse representation based single-image super-resolution reconstruction[J]. Neurocomputing, 2011, 74(17): 3193–3203.
[11] DONG Chao, LOY C C, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision. Cham: Springer, 2014: 184-199.
[12] DONG Chao, LOY C C, TANG Xiaoou. Accelerating the super-resolution convolutional neural network[EB/OL]. (2016-08-01)[2023-04-06]. https://arxiv.org/abs/1608.00367.
[13] SHI Wenzhe, CABALLERO J, HUSZáR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1874-1883.
[14] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[15] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1646-1654.
[16] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE, 2017: 1132-1140.
[17] SEGU M, TONIONI A, TOMBARI F. Batch normalization embeddings for deep domain generalization[J]. Pattern recognition, 2023, 135: 109115.
[18] ZHANG Yulun, LI Kunpeng, LI Kai, et al. Image super-resolution using very deep residual channel attention networks[C]//European Conference on Computer Vision. Cham: Springer, 2018: 294-310.
[19] ZHANG Yulun, TIAN Yapeng, KONG Yu, et al. Residual dense network for image super-resolution[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2472-2481.
[20] CHEN Liangliang, KOU Qiqi, CHENG Deqiang, et al. Content-guided deep residual network for single image super-resolution[J]. Optik, 2020, 202: 163678.
[21] 程德强, 郭昕, 陈亮亮, 等. 多通道递归残差网络的图像超分辨率重建[J]. 中国图象图形学报, 2021, 26(3): 605–618
CHENG Deqiang, GUO Xin, CHEN Liangliang, et al. Image super-resolution reconstruction from multi-channel recursive residual network[J]. Journal of image and graphics, 2021, 26(3): 605–618
[22] HARIS M, SHAKHNAROVICH G, UKITA N. Deep back-projection networks for super-resolution[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 1664-1673.
[23] PUROHIT K, MANDAL S, RAJAGOPALAN A N. Mixed-dense connection networks for image and video super-resolution[J]. Neurocomputing, 2020, 398: 360–376.
[24] FARAMARZI A, AHMADYFARD A, KHOSRAVI H. Adaptive image super-resolution algorithm based on fractional Fourier transform[J]. Image analysis & stereology, 2022, 41(2): 133–144.
[25] AHN H, YIM C. Convolutional neural networks using skip connections with layer groups for super-resolution image reconstruction based on deep learning[J]. Applied sciences, 2020, 10(6): 1959.
[26] LIU Jie, TANG Jie, WU Gangshan. Residual feature distillation network for lightweight image super-resolution[M]//Computer Vision-ECCV 2020 Workshops. Cham: Springer International Publishing, 2020: 41-55.
[27] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of the British Machine Vision Conference 2012. Surrey: British Machine Vision Association, 2012.
[28] ROMANO Y, PROTTER M, ELAD M. Single image interpolation via adaptive nonlocal sparsity-based modeling[J]. IEEE transactions on image processing, 2014, 23(7): 3085–3098.
[29] LIU Yun, CHENG Mingming, HU Xiaowei, et al. Richer convolutional features for edge detection[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(8): 1939–1946.
[30] HUANG Jiabin, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 5197-5206.
[31] MEI Yiqun, FAN Yuchen, ZHOU Yuqian, et al. Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 5689-5698.
[32] MEI Yiqun, FAN Yuchen, ZHOU Yuqian. Image super-resolution with non-local sparse attention[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 3516-3525.
[33] WANG Yan. Edge-enhanced feature distillation network for efficient super-resolution[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New Orleans: IEEE, 2022: 776-784.
- 相似文献/References:
-
[1]刘? 琚,乔建苹.基于学习的超分辨率重建技术[J].智能系统学报,2009,4(3):199.
LIU Ju,QIAO Jian-ping.Learningbased superresolution reconstruction[J].CAAI Transactions on Intelligent Systems,2009,4():199.
[2]李庆武,蔡艳梅,徐立中.基于分块分类的智能视频监控背景更新算法[J].智能系统学报,2010,5(3):272.
LI Qing-wu,CAI Yan-mei,XU Li-zhong.Background update algorithm based on blocks classification forintelligent video surveillance[J].CAAI Transactions on Intelligent Systems,2010,5():272.
[3]邵真天,袁杰.一种基于曲波变换的图像去块算法[J].智能系统学报,2012,7(2):102.
SHAO Zhentian,YUAN Jie.An image deblocking algorithm based on curvelet transformation[J].CAAI Transactions on Intelligent Systems,2012,7():102.
[4]王锦榕,袁学海,刘增良.基于图像处理技术的瞳孔和角膜反射中心提取算法[J].智能系统学报,2012,7(5):423.
WANG Jinrong,YUAN Xuehai,LIU Zengliang.An extraction method of pupil and corneal reflection centers based on image processing technology[J].CAAI Transactions on Intelligent Systems,2012,7():423.
[5]傅博,姜勇,王洪光,等.输电线路巡检图像智能诊断系统[J].智能系统学报,2016,11(1):70.[doi:10.11992/tis.201503043]
FU Bo,JIANG Yong,WANG Hongguang,et al.Intelligent diagnosis system for patrol check images of power transmission lines[J].CAAI Transactions on Intelligent Systems,2016,11():70.[doi:10.11992/tis.201503043]
[6]任晓霞,孙秀明,耿鹏,等.多小波和NSDFB组合域递归滤波多聚焦图像融合[J].智能系统学报,2016,11(2):241.[doi:10.11992/tis.201509017]
REN Xiaoxia,SUN Xiuming,GENG Peng,et al.Multifocus image fusion using a recursive filter in the combined domain of multiwavelets and NSDFB[J].CAAI Transactions on Intelligent Systems,2016,11():241.[doi:10.11992/tis.201509017]
[7]梁义辉,战强.一种面向无线图像传输的视觉平台[J].智能系统学报,2016,11(5):608.[doi:10.11992/tis.201512014]
LIANG Yihui,ZHAN Qiang.A visual platform for wireless image transmission[J].CAAI Transactions on Intelligent Systems,2016,11():608.[doi:10.11992/tis.201512014]
[8]李霞丽,吴立成,樊艳明.易于硬件实现的压缩感知观测矩阵的研究与构造[J].智能系统学报,2017,12(3):279.[doi:10.11992/tis.201606037]
LI Xiali,WU Licheng,FAN Yanming.Study and construction of a compressed sensing measurement matrix that is easy to implement in hardware[J].CAAI Transactions on Intelligent Systems,2017,12():279.[doi:10.11992/tis.201606037]
[9]姜婷,袭肖明,岳厚光.基于分布先验的半监督FCM的肺结节分类[J].智能系统学报,2017,12(5):729.[doi:10.11992/tis.201706018]
JIANG Ting,XI Xiaoming,YUE Houguang.Classification of pulmonary nodules by semi-supervised FCM based on prior distribution[J].CAAI Transactions on Intelligent Systems,2017,12():729.[doi:10.11992/tis.201706018]
[10]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
备注/Memo
收稿日期:2023-4-6。
基金项目:中央高校基本科研业务费专项资金项目（2020QN49）；国家自然科学基金项目（52204177）.
作者简介:程德强,教授,博士生导师,博士,中国矿业大学信息与控制工程学院党委书记,中国煤炭工业协会信息化分会理事。主要研究方向为图像智能检测与模式识别、图像处理与视频编码。主持国家自然科学基金项目3项,作为技术负责人承担国家重点研发计划课题、江苏省科技成果转化专项资金项目和贵州省科技支撑计划重点项目等20余项。以第一发明人获得发明专利授权30余项,其中获授权美国发明专利2项。6项技术发明在企业转化应用。牵头制定中国煤炭工业协会团体标准2项,参与制定行业标准2项;朱星光,硕士研究生,主要研究方向为图像识别、图像超分辨率重建。获授权发明专利1篇,获授权软件著作权1篇;寇旗旗,讲师,博士,中国矿业大学人工智能研究院智慧矿山研究中心副主任,主要研究方向为视频/图像处理、智能检测与模式识别、工业机器人与机器视觉。2020年中国矿业大学研究生教育教学成果奖二等奖1 项,指导学生获得2021年第十七届江苏省大学生课外学术科技作品竞赛暨"挑战杯"全国竞赛江苏省选拔赛一等奖,获授权发明专利5 项,发表学术论文20余篇。
通讯作者:寇旗旗.E-mail:kouqiqi@cumt.edu.cn
更新日期/Last Update:
1900-01-01