[1]刘玉铠,周登文.基于多路特征渐进融合和注意力机制的轻量级图像超分辨率重建[J].智能系统学报,2024,19(4):863-873.[doi:10.11992/tis.202209045]
LIU Yukai,ZHOU Dengwen.Lightweight super-resolution reconstruction via progressive multi-path feature fusion and attention mechanism[J].CAAI Transactions on Intelligent Systems,2024,19(4):863-873.[doi:10.11992/tis.202209045]
点击复制
LIU Yukai,ZHOU Dengwen.Lightweight super-resolution reconstruction via progressive multi-path feature fusion and attention mechanism[J].CAAI Transactions on Intelligent Systems,2024,19(4):863-873.[doi:10.11992/tis.202209045]
基于多路特征渐进融合和注意力机制的轻量级图像超分辨率重建
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第4期
页码:
863-873
栏目:
学术论文—机器感知与模式识别
出版日期:
2024-07-05
- Title:
- Lightweight super-resolution reconstruction via progressive multi-path feature fusion and attention mechanism
- Keywords:
- image super-resolution; convolutional neural network; feature fusion; attention mechanism; deep learning; image restoration; peak signal-to noise ratio; structural similarity
- 分类号:
- TP391
- 摘要:
- 为进一步探索在计算和存储资源受限设备上应用超分辨率方法的可能性，本研究聚焦于深度卷积神经网络技术在单图像超分辨率中的应用，特别是如何在不显著增加网络规模的情况下，提升网络的性能。 本文提出一种新的基于多路特征渐进融合和注意力机制的轻量级单图像超分辨率方法(multi-path feature fusion and attention mechanism，MPFFA)。MPFFA包括一个多路特征渐进融合块 (multi-path feature progressive fusion，FPF)，可以通过前面的特征，多路渐进地引导和校准后面特征的学习；还包括一个多路特征注意力机制(multi-path feature attention mechanism，FAM)，通过加权拼接多路特征通道，可以提高特征信息的利用率和特征表达能力。实验结果表明：MPFFA显著优于当前其他代表性的方法，在模型复杂度和性能间达到了更好的平衡。本文提出的模型能够更好地应用于计算和资源受限的设备上。
- Abstract:
- In order to further explore the possibility of applying super-resolution methods on computing and storage resource-constrained devices, this study focuses on the application of deep convolutional neural network technology in single-image super-resolution, especially how to improve the performance of the network without significantly increasing the network size. In this paper, a novel lightweight single image super resolution (SISR) method via progressive multi-path feature fusion and attention mechanism (MPFFA) is proposed. MPFFA includes a multi-path FPF module, which can progressively guide and calibrate the learning of the following features through multiple paths. MPFFA also includes a multi-path feature attention mechanism (FAM), which can improve the utilization rate of feature information and the ability of feature expression by splicing multi-path features with weights. The experimental result shows that MPFFA significantly outperforms other representative methods, thus achieves a better balance between model complexity and performance. The proposed model can be better applied to computing and resource-constrained devices.
- 参考文献/References:
-
[1] PARK S C, PARK M K, KANG M G. Super-resolution image reconstruction: a technical overview[J]. IEEE signal processing magazine, 2003, 20(3): 21–36.
[2] DONG Chao, LOY C C, HE Kaiming, et al. Image super-resolution using deep convolutional networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 38(2): 295–307.
[3] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Hawaii: IEEE, 2017: 136-144.
[4] ZHANG Yulun, LI Kunpeng, LI Kai, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision. Munich: ACM, 2018: 286-301.
[5] HUI Zheng, WANG Xiumei, GAO Xinbo. Fast and accurate single image super-resolution via information distillation network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 723-731.
[6] ZHAO Hengyuan, KONG Xiangtao, HE Jingwen, et al. Efficient image super-resolution using pixel attention[C]// European Conference on Computer Vision. Glasgow: ACM, 2020: 56-72.
[7] WANG Xuehui, WANG Qing, ZHAO Yuzhi, et al. Lightweight single-image super-resolution network with attentive auxiliary feature learning[C]//Proceedings of the Asian Conference on Computer Vision. Kyoto: AFCV, 2021: 268-285.
[8] ZHAO Xiaole, LIAO Ying, HE Tian, et al. Fc2n: fully channel-concatenated network for single image super-resolution [EB/OL]. (2021-05-05)[2023-06-12].https://arxiv.org/pdf/1907.03221.pdf.
[9] DONG Chao, LOY C C, TANG Xiaoou. Accelerating the super-resolution convolutional neural network[C]// European Conference on Computer Vision. Amsterdam: ACM, 2016: 391-407.
[10] KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1637-1645.
[11] TAI Ying, YANG Jian, LIU Xiaoming, et al. MemNet: a persistent memory network for image restoration[C]// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4539-4547.
[12] 周登文, 赵丽娟. 基于递归残差网络的图像超分辨率重建[J]. 自动化学报, 2019, 45(6): 1157–1165
ZHOU Dengwen, ZHAO Lijuan. Image super-resolution based on recursive residual networks[J]. Acta automatica sinica, 2019, 45(6): 1157–1165
[13] HUI Zheng, GAO Xinbo, YANG Yuchu, et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia. Nice: ACM, 2019: 2024-2032.
[14] LIU Jie, TANG Jie, WU Gangshan. Residual feature distillation network for lightweight image super-resolution [C]//European Conference on Computer Vision. Glasgow: ACM, 2020: 41-55.
[15] 周登文, 王婉君. 基于区域互补注意力和多维注意力的轻量级图像超分辨率网络[J]. 模式识别与人工智能, 2022, 35(7): 625–636
ZHOU Dengwen, WANG Wanjun. Lightweight image supe-resolution network based on regional complementary attention and multi-dimensional attention[J]. Pattern recognition and artificial intelligence, 2022, 35(7): 625–636
[16] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1646-1654.
[17] ZHANG Yulun, TIAN Yapeng, KONG Yu, et al. Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2472-2481.
[18] WOO S, PARK J, LEE J Y, et al. Cbam: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision. Glasgow: ACM, 2018: 3-19.
[19] ZHANG Jiqing, LONG Chengjiang, WANG Yuxin, et al. A two-stage attentive network for single image super-resolution[J]. IEEE transactions on circuits and systems for video technology, 2021, 32: 1020–1033.
[20] TIMOFTE R, AGSTSSON E, VAN GOOL L, et al. Ntire 2017 challenge on single image super-resolution: methods and results[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Hawaii: IEEE, 2017: 114-125.
[21] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer VIsion and Pattern Recognition. Salt Lake City: IEEE, 2018: 586-595.
[22] HUANG Zhiyong, LI Wenbin, LI Jinxin, et al. Dual-path attention network for single image super-resolution[J]. Expert systems with applications, 2021, 169: 114450.
[23] LAI Weisheng, HUANG Jiabin, AHUJA N, et al. Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 624-632.
[24] TAI Ying, YANG Jian, LIU Xiaoming. Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 3147-3155.
[25] ZHANG Kai, ZUO Wangmeng, ZHANG Lei. Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3262-3271.
[26] AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[C]//Proceedings of the European Conference on Computer Vision. Munich: ACM, 2018: 252-268.
[27] LUO Xiaotong, XIE Yuan, ZHANG Yulun, et al. Latticenet: towards lightweight image super-resolution with lattice block[C]//Computer Vision–ECCV 2020: 16th European Conference. Glasgow: ACM, 2020: 272-289.
[28] MUQEET A, HWANG J, YANG S, et al. Multi-attention based ultra lightweight image super-resolution[C]// European Conference on Computer Vision. Glasgow: ACM, 2020: 103-118.
[29] ZHOU Dengwen, CHEN Yiming, LI Wenbin, et al. Image super-resolution based on adaptive cascading attention network[J]. Expert systems with applications, 2021, 186: 115815.
[30] LI Wenbo, ZHOU Kun, QI Lu, et al. Lapar: linearly-assembled pixel-adaptive regression network for single image super-resolution and beyond[C]//Proceedings of the Advances in Neural Information Processing Systems. Chicago: NIPS, 2021: 20343–20355.
- 相似文献/References:
-
[1]殷瑞,苏松志,李绍滋.一种卷积神经网络的图像矩正则化策略[J].智能系统学报,2016,11(1):43.[doi:10.11992/tis.201509018]
YIN Rui,SU Songzhi,LI Shaozi.Convolutional neural network’s image moment regularizing strategy[J].CAAI Transactions on Intelligent Systems,2016,11():43.[doi:10.11992/tis.201509018]
[2]龚震霆,陈光喜,任夏荔,等.基于卷积神经网络和哈希编码的图像检索方法[J].智能系统学报,2016,11(3):391.[doi:10.11992/tis.201603028]
GONG Zhenting,CHEN Guangxi,REN Xiali,et al.An image retrieval method based on a convolutional neural network and hash coding[J].CAAI Transactions on Intelligent Systems,2016,11():391.[doi:10.11992/tis.201603028]
[3]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[4]师亚亭,李卫军,宁欣,等.基于嘴巴状态约束的人脸特征点定位算法[J].智能系统学报,2016,11(5):578.[doi:10.11992/tis.201602006]
SHI Yating,LI Weijun,NING Xin,et al.A facial feature point locating algorithmbased on mouth-state constraints[J].CAAI Transactions on Intelligent Systems,2016,11():578.[doi:10.11992/tis.201602006]
[5]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[6]杨晓兰,强彦,赵涓涓,等.基于医学征象和卷积神经网络的肺结节CT图像哈希检索[J].智能系统学报,2017,12(6):857.[doi:10.11992/tis.201706035]
YANG Xiaolan,QIANG Yan,ZHAO Juanjuan,et al.Hashing retrieval for CT images of pulmonary nodules based on medical signs and convolutional neural networks[J].CAAI Transactions on Intelligent Systems,2017,12():857.[doi:10.11992/tis.201706035]
[7]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
[8]莫凌飞,蒋红亮,李煊鹏.基于深度学习的视频预测研究综述[J].智能系统学报,2018,13(1):85.[doi:10.11992/tis.201707032]
MO Lingfei,JIANG Hongliang,LI Xuanpeng.Review of deep learning-based video prediction[J].CAAI Transactions on Intelligent Systems,2018,13():85.[doi:10.11992/tis.201707032]
[9]王成济,罗志明,钟准,等.一种多层特征融合的人脸检测方法[J].智能系统学报,2018,13(1):138.[doi:10.11992/tis.201707018]
WANG Chengji,LUO Zhiming,ZHONG Zhun,et al.Face detection method fusing multi-layer features[J].CAAI Transactions on Intelligent Systems,2018,13():138.[doi:10.11992/tis.201707018]
[10]葛园园,许有疆,赵帅,等.自动驾驶场景下小且密集的交通标志检测[J].智能系统学报,2018,13(3):366.[doi:10.11992/tis.201706040]
GE Yuanyuan,XU Youjiang,ZHAO Shuai,et al.Detection of small and dense traffic signs in self-driving scenarios[J].CAAI Transactions on Intelligent Systems,2018,13():366.[doi:10.11992/tis.201706040]
备注/Memo
收稿日期:2022-09-23。
作者简介:刘玉铠，硕士，主要研究方向为计算机视觉和深度学习。E-mail：liuyk@ncepu.edu.cn;周登文，教授，主要研究方向为图像去噪、图像去马赛克、图像插值和图像超分辨率。发表学术论文50余篇。E-mail：zdw@ncepu.edu.cn
通讯作者:周登文. E-mail：zdw@ncepu.edu.cn
更新日期/Last Update:
1900-01-01