[1]ZHANG Xinyi,TAN Yao,XING Xianglei.Deep feature fusion for underwater-image restoration based on physical priors[J].CAAI Transactions on Intelligent Systems,2023,18(6):1185-1196.[doi:10.11992/tis.202304038]
Copy

Deep feature fusion for underwater-image restoration based on physical priors

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 6 Page number: 1185-1196 Column: 学术论文—机器学习 Public date: 2023-11-05
Title:
Deep feature fusion for underwater-image restoration based on physical priors
Author(s):
ZHANG Xinyi;  TAN Yao;  XING Xianglei
College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China
Keywords:
deep learning; underwater-image restoration; neural networks; information separation; encoder; decoder; feature extraction; image fusion
CLC:
TP391.41
DOI:
10.11992/tis.202304038
Abstract:
Due to interference factors such as suspended impurities of plankton and varying spectral absorption rates in an underwater environment, underwater images often suffer from degradation issues such as image blur, color distortion, and uneven illumination. This paper proposes an underwater-image reconstruction model that combines physical imaging principles with data-driven deep-learning methods. Using a deep neural network to infer the learnable parameters in the physical imaging model, the model generates data-driven restoration feature maps and physically informed restoration feature maps through modulated convolution and prior physical knowledge, respectively. Deep feature fusion with a mixed-attention mechanism is introduced to reconstruct the final image. Experimental results showed that this method can reduce noise, improve contrast, and restore image details, enhancing the visual quality and target detection accuracy of underwater images and increasing the robustness and generalizability of the underwater learning model.
References:
[1] IQBAL K, ODETAYO M, JAMES A, et al. Enhancing the low quality images using unsupervised colour correction method[C]//2010 IEEE International Conference on Systems, Man and Cybernetics. Istanbul: IEEE, 2010: 1703-1709.
[2] ABDUL GHANI A S, MAT ISA N A. Underwater image quality enhancement through integrated color model with Rayleigh distribution[J]. Applied soft computing, 2015, 27: 219–230.
[3] ANCUTI C O, ANCUTI C, DE VLEESCHOUWER C, et al. Color balance and fusion for underwater image enhancement[J]. IEEE transactions on image processing, 2017, 27(1): 379–393.
[4] ANCUTI C, ANCUTI C O, HABER T, et al. Enhancing underwater images and videos by fusion[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 81-88.
[5] GALDRAN A, PARDO D, PICóN A, et al. Automatic red-channel underwater image restoration[J]. Journal of visual communication and image representation, 2015, 26: 132–145.
[6] DREWS P L J J, NASCIMENTO E R, BOTELHO S S C, et al. Underwater depth estimation and image restoration based on single images[J]. IEEE computer graphics and applications, 2016, 36(2): 24–35.
[7] LI Chongyi, GUO Jichang, CONG Runmin, et al. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior[J]. IEEE transactions on image processing, 2016, 25(12): 5664–5677.
[8] LIU Ze, LIN Yutong, CAO Yue, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 9992-10002.
[9] LIU Zhuang, MAO Hanzi, WU Chaoyuan, et al. A ConvNet for the 2020s[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 11966-11976.
[10] LI Jie, SKINNER K A, EUSTICE R M, et al. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE robotics and automation letters, 2018, 3(1): 387–394.
[11] 李微, 毕晓君. 改进U-Net网络的水下图像增强[J]. 应用科技, 2021, 48(3): 34–40
LI Wei, BI Xiaojun. Underwater image enhancement based on improved U-Net model[J]. Applied science and technology, 2021, 48(3): 34–40
[12] LI Chongyi, GUO Jichang, GUO Chunle. Emerging from water: underwater image color correction based on weakly supervised color transfer[J]. IEEE signal processing letters, 2018, 25(3): 323–327.
[13] PENG Y T, CAO Keming, COSMAN P C. Generalization of the dark channel prior for single image restoration[J]. IEEE transactions on image processing, 2018, 27(6): 2856–2868.
[14] JAFFE J S. Computer modeling and the design of optimal underwater imaging systems[J]. IEEE journal of oceanic engineering, 1990, 15(2): 101–111.
[15] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[16] DONG Chao, LOY C C, HE Kaiming, et al. Image super-resolution using deep convolutional networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(2): 295–307.
[17] WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11531-11539.
[18] KARRAS T, LAINE S, AITTALA M, et al. Analyzing and improving the image quality of StyleGAN[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 8107-8116.
[19] WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE transactions on image processing:a publication of the IEEE signal processing society, 2004, 13(4): 600–612.
[20] KARRAS T, AILA T LAINE S, et al. Progressive growing of GANs for improved quality, stability, and variation[EB/OL]. (2017-10-27)[2023-04-18]. https://arxiv.org/abs/1710.10196.
[21] ARORA S, COHEN N, HAZAN E. On the optimization of deep networks: implicit acceleration by overparameterization[EB/OL]. (2018-02-19) [2023-04-18]. https://arxiv.org/abs/1802.06509.
[22] JUSTIN J, ALEXANDRE A, LI Feifei. Perceptual losses for real-time style transfer and super-resolution[C]//Computer Vision–ECCV 2016. Amsterdam: Springer International Publishing, 2016: 694-711.
[23] LI Chongyi, ANWAR S, PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern recognition, 2020, 98: 107038.
[24] UPLAVIKAR P, WU Zhenyu, WANG Zhangyang. All-in-one underwater image enhancement using domain-adversarial learning[EB/OL]. (2019-05-30) [2023-04-18]. https://arxiv.org/abs/1905.13342.
[25] HAN Junlin, SHOEIBY M, MALTHUS T, et al. Underwater image restoration via contrastive learning and a real-world dataset[J]. Remote sensing, 2022, 14(17): 4297.
[26] JERLOV N G. Marine optics[M]. [S. l. ]: Elsevier, 1976.
[27] NATHAN S, DEREK H, PUSHMEET K, et al. Indoor segmentation and support inference from RGBD images[C]//Computer Vision-ECCV 2012. Florence: Springer International Publishing, 2012: 746–760.
[28] LI Chongyi, GUO Jichang, CHEN Sanji, et al. Underwater image restoration based on minimum information loss principle and optical properties of underwater imaging[C]//2016 IEEE International Conference on Image Processing. Phoenix: IEEE, 2016: 1993-1997.
[29] PENG Yantsung, PAMELA C C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE transactions on image processing, 2017, 26(4): 1579–1594.
[30] QIN Xuebin, ZHANG Zichen, HUANG Chenyang, et al. BASNet: boundary-aware salient object detection[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 7471-7481.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems