[1]LIU Yukai,ZHOU Dengwen.Lightweight super-resolution reconstruction via progressive multi-path feature fusion and attention mechanism[J].CAAI Transactions on Intelligent Systems,2024,19(4):863-873.[doi:10.11992/tis.202209045]
Copy

Lightweight super-resolution reconstruction via progressive multi-path feature fusion and attention mechanism

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 4 Page number: 863-873 Column: 学术论文—机器感知与模式识别 Public date: 2024-07-05
Title:
Lightweight super-resolution reconstruction via progressive multi-path feature fusion and attention mechanism
Author(s):
LIU Yukai;  ZHOU Dengwen
School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China
Keywords:
image super-resolution; convolutional neural network; feature fusion; attention mechanism; deep learning; image restoration; peak signal-to noise ratio; structural similarity
CLC:
TP391
DOI:
10.11992/tis.202209045
Abstract:
In order to further explore the possibility of applying super-resolution methods on computing and storage resource-constrained devices, this study focuses on the application of deep convolutional neural network technology in single-image super-resolution, especially how to improve the performance of the network without significantly increasing the network size. In this paper, a novel lightweight single image super resolution (SISR) method via progressive multi-path feature fusion and attention mechanism (MPFFA) is proposed. MPFFA includes a multi-path FPF module, which can progressively guide and calibrate the learning of the following features through multiple paths. MPFFA also includes a multi-path feature attention mechanism (FAM), which can improve the utilization rate of feature information and the ability of feature expression by splicing multi-path features with weights. The experimental result shows that MPFFA significantly outperforms other representative methods, thus achieves a better balance between model complexity and performance. The proposed model can be better applied to computing and resource-constrained devices.
References:
[1] PARK S C, PARK M K, KANG M G. Super-resolution image reconstruction: a technical overview[J]. IEEE signal processing magazine, 2003, 20(3): 21–36.
[2] DONG Chao, LOY C C, HE Kaiming, et al. Image super-resolution using deep convolutional networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 38(2): 295–307.
[3] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Hawaii: IEEE, 2017: 136-144.
[4] ZHANG Yulun, LI Kunpeng, LI Kai, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision. Munich: ACM, 2018: 286-301.
[5] HUI Zheng, WANG Xiumei, GAO Xinbo. Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 723-731.
[6] ZHAO Hengyuan, KONG Xiangtao, HE Jingwen, et al. Efficient image super-resolution using pixel attention[C]// European Conference on Computer Vision. Glasgow: ACM, 2020: 56-72.
[7] WANG Xuehui, WANG Qing, ZHAO Yuzhi, et al. Lightweight single-image super-resolution network with attentive auxiliary feature learning[C]//Proceedings of the Asian Conference on Computer Vision. Kyoto: AFCV, 2021: 268-285.
[8] ZHAO Xiaole, LIAO Ying, HE Tian, et al. Fc2n: fully channel-concatenated network for single image super-resolution [EB/OL]. (2021-05-05)[2023-06-12].https://arxiv.org/pdf/1907.03221.pdf.
[9] DONG Chao, LOY C C, TANG Xiaoou. Accelerating the super-resolution convolutional neural network[C]// European Conference on Computer Vision. Amsterdam: ACM, 2016: 391-407.
[10] KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1637-1645.
[11] TAI Ying, YANG Jian, LIU Xiaoming, et al. MemNet: a persistent memory network for image restoration[C]// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4539-4547.
[12] 周登文, 赵丽娟. 基于递归残差网络的图像超分辨率重建[J]. 自动化学报, 2019, 45(6): 1157–1165
ZHOU Dengwen, ZHAO Lijuan. Image super-resolution based on recursive residual networks[J]. Acta automatica sinica, 2019, 45(6): 1157–1165
[13] HUI Zheng, GAO Xinbo, YANG Yuchu, et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia. Nice: ACM, 2019: 2024-2032.
[14] LIU Jie, TANG Jie, WU Gangshan. Residual feature distillation network for lightweight image super-resolution [C]//European Conference on Computer Vision. Glasgow: ACM, 2020: 41-55.
[15] 周登文, 王婉君. 基于区域互补注意力和多维注意力的轻量级图像超分辨率网络[J]. 模式识别与人工智能, 2022, 35(7): 625–636
ZHOU Dengwen, WANG Wanjun. Lightweight image supe-resolution network based on regional complementary attention and multi-dimensional attention[J]. Pattern recognition and artificial intelligence, 2022, 35(7): 625–636
[16] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1646-1654.
[17] ZHANG Yulun, TIAN Yapeng, KONG Yu, et al. Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2472-2481.
[18] WOO S, PARK J, LEE J Y, et al. Cbam: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision. Glasgow: ACM, 2018: 3-19.
[19] ZHANG Jiqing, LONG Chengjiang, WANG Yuxin, et al. A two-stage attentive network for single image super-resolution[J]. IEEE transactions on circuits and systems for video technology, 2021, 32: 1020–1033.
[20] TIMOFTE R, AGSTSSON E, VAN GOOL L, et al. Ntire 2017 challenge on single image super-resolution: methods and results[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Hawaii: IEEE, 2017: 114-125.
[21] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer VIsion and Pattern Recognition. Salt Lake City: IEEE, 2018: 586-595.
[22] HUANG Zhiyong, LI Wenbin, LI Jinxin, et al. Dual-path attention network for single image super-resolution[J]. Expert systems with applications, 2021, 169: 114450.
[23] LAI Weisheng, HUANG Jiabin, AHUJA N, et al. Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 624-632.
[24] TAI Ying, YANG Jian, LIU Xiaoming. Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 3147-3155.
[25] ZHANG Kai, ZUO Wangmeng, ZHANG Lei. Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3262-3271.
[26] AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[C]//Proceedings of the European Conference on Computer Vision. Munich: ACM, 2018: 252-268.
[27] LUO Xiaotong, XIE Yuan, ZHANG Yulun, et al. Latticenet: towards lightweight image super-resolution with lattice block[C]//Computer Vision–ECCV 2020: 16th European Conference. Glasgow: ACM, 2020: 272-289.
[28] MUQEET A, HWANG J, YANG S, et al. Multi-attention based ultra lightweight image super-resolution[C]// European Conference on Computer Vision. Glasgow: ACM, 2020: 103-118.
[29] ZHOU Dengwen, CHEN Yiming, LI Wenbin, et al. Image super-resolution based on adaptive cascading attention network[J]. Expert systems with applications, 2021, 186: 115815.
[30] LI Wenbo, ZHOU Kun, QI Lu, et al. Lapar: linearly-assembled pixel-adaptive regression network for single image super-resolution and beyond[C]//Proceedings of the Advances in Neural Information Processing Systems. Chicago: NIPS, 2021: 20343–20355.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems