[1]SUN Lihui,CHEN Heng,SHANG Yueping.Video denoising based on optical flow and multi-scale features[J].CAAI Transactions on Intelligent Systems,2024,19(6):1593-1603.[doi:10.11992/tis.202306002]
Copy

Video denoising based on optical flow and multi-scale features

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 6 Page number: 1593-1603 Column: 人工智能院长论坛 Public date: 2024-12-05
Title:
Video denoising based on optical flow and multi-scale features
Author(s):
SUN Lihui1;  CHEN Heng1;  SHANG Yueping2
1. School of Information Technology, Hebei University of Economics and Business, Shijiazhuang 050061, China;
2. College of Mathematics and Statistics, Hebei University of Economics and Business, Shijiazhuang 050061, China
Keywords:
multi-frame noise reduction; video denoising; optical flow alignment; perceptual loss; non-local attention; image processing; computer vision; deep learning
CLC:
TP391
DOI:
10.11992/tis.202306002
Abstract:
To effectively eliminate noise from videos while preserving texture details, a cascade video denoising algorithm that integrates optical flow and multi-scale features is proposed. The process begins by accurately aligning sequence frames using a grouping strategy. These frames are then processed through a multi-scale architecture that combines residual refinement and selective skip connection. This approach not only preserves detailed features but also enhances alignment and fusion. Furthermore, a non-local attention mechanism is employed to deeply mine spatiotemporal features, enabling the reconstruction of high-quality videos. To preserve detailed textures, a target function supervision training method that combines perceptual loss is proposed. Experimental results show that the proposed algorithm retains more texture features and aligns well with human visual perception. It is also highly robust, has low computational complexity under strong noise, and meets real-time denoising requirements.
References:
[1] LI J, WU X, NIU Z, et al. Unidirectional video denoising by mimicking backward recurrent modules with look-ahead forward ones [C]//European Conference on Computer Vision. Berlin: ECCV,2022: 592-609.
[2] ZHAO Yaping, ZHENG Haitian, WANG Zhongrui, et al. Manet: improving video denoising with a multi-alignment network[C]//2022 IEEE International Conference on Image Processing. Bordeaux: IEEE, 2022: 2036-2040.
[3] 刘迪, 贾金露, 赵玉卿, 等. 基于深度学习的图像去噪方法研究综述[J]. 计算机工程与应用, 2021, 57(7): 1-13.
LIU Di, JIA Jinlu, ZHAO Yuqing, et al. Overview of image denoising methods based on deep learning[J]. Computer engineering and applications, 2021, 57(7): 1-13.
[4] ZHANG Kai, ZUO Wangmeng, CHEN Yunjin, et al. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising[J]. IEEE transactions on image processing: a publication of the IEEE signal processing society, 2017, 26(7): 3142-3155.
[5] GUO Shi, YAN Zifei, ZHANG Kai, et al. Toward convolutional blind denoising of real photographs[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 1712-1722.
[6] HELOU M, SUSSTRUNK S. Blind universal Bayesian image denoising with Gaussian noise level learning[J]. IEEE transactions on image processing: a publication of the IEEE signal processing society, 2020, 29: 4885-4897.
[7] REN Chao, HE Xiaohai, WANG Chuncheng, et al. Adaptive consistency prior based deep network for image denoising[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 8592-8602.
[8] DAVY A, EHRET T, MOREL J M, et al. A non-local CNN for video denoising[C]//2019 IEEE International Conference on Image Processing. Taipei: IEEE, 2019: 2409-2413.
[9] VAKSMAN G, ELAD M, MILANFAR P. Patch craft: video denoising by deep modeling and patch matching[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 2137-2146.
[10] MAGGIONI M, HUANG Yibin, LI Cheng, et al. Efficient multi-stage video denoising with recurrent spatio-temporal fusion[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 3465-3474.
[11] TASSANO M, DELON J, VEIT T. DVDNET: a fast network for deep video denoising[C]//2019 IEEE International Conference on Image Processing. Taipei: IEEE, 2019: 1805-1809.
[12] XUE Tianfan, CHEN Baian, WU Jiajun, et al. Video enhancement with task-oriented flow[J]. International journal of computer vision, 2019, 127(8): 1106-1125.
[13] CHAN K C K, WANG Xintao, YU Ke, et al. BasicVSR: the search for essential components in video super-resolution and beyond[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 4945-4954.
[14] TASSANO M, DELON J, VEIT T. FastDVDnet: towards real-time deep video denoising without flow estimation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1351-1360.
[15] XIANG Liuyu, ZHOU Jundong, LIU Jirui, et al. ReMoNet: recurrent multi-output network for efficient video denoising[C]//The Association for the Advancement of Artificial Intelligence. Vancouver: AAAI, 2022: 2786-2794.
[16] WANG Xintao, CHAN K C K, YU Ke, et al. EDVR: video restoration with enhanced deformable convolutional networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach: IEEE, 2019: 1954-1963.
[17] SUN Deqing, YANG Xiaodong, LIU Mingyu, et al. PWC-net: CNNs for optical flow using pyramid, warping, and cost volume[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8934-8943.
[18] BALLé J, LAPARRA V, SIMONCELLI E P. Density modeling of images using a generalized normalization transformation [C]//4th International Conference on Learning Representations. Puerto Rico: ICLR, 2016: 100-112.
[19] LI Xiang, WANG Wenhai, HU Xiaolin, et al. Selective kernel networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 510-519.
[20] CAO Yue, XU Jiarui, LIN S, et al. GCNet: non-local networks meet squeeze-excitation networks and beyond[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop. Seoul: IEEE, 2019: 1971-1980.
[21] HU Jie, SHEN Li, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2020, 42(8): 2011-2023.
[22] ZAMIR S W, ARORA A, KHAN S, et al. Learning enriched features for fast image restoration and enhancement[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(2): 1934-1948.
[23] WANG Haonan, CAO Peng, WANG Jiaqi, et al. UCTransNet: rethinking the skip connections in U-net from a channel-wise perspective with transformer[C]//The Association for the Advancement of Artificial Intelligence. Vancouver: AAAI, 2022: 2441-2449.
[24] 申屠敏健. 基于先验信息和卷积神经网络的视频去噪算法研究[D]. 成都: 电子科技大学, 2021.
SHENTU Minjian. Video denoising based on prior information and convolutional neural network[D]. Chengdu: University of Electronic Science and Technology of China, 2021.
[25] JOHNSON J, ALAHI A, LI Feifei. Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision. Cham: Springer, 2016: 694-711.
[26] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//3rd International Conference on Learning Representations. San Diego: ICLR, 2015: 1-14.
[27] PERAZZI F, PONT-TUSET J, MCWILLIAMS B, et al. A benchmark dataset and evaluation methodology for video object segmentation[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 724-732.
[28] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1026-1034.
[29] KINGMA D P, BA J L. Adam: a method for stochastic optimization[C]//3rd International Conference on Learning Representations. San Diego: ICLR, 2015: 131-142.
Similar References:

Memo

-

Last Update: 2024-11-05

Copyright © CAAI Transactions on Intelligent Systems