[1]YAN Zheping,QU Siyu,XING Wen.An overview of underwater image enhancement methods[J].CAAI Transactions on Intelligent Systems,2022,17(5):860-873.[doi:10.11992/tis.202108022]
Copy

An overview of underwater image enhancement methods

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 17 Number of periods: 2022 5 Page number: 860-873 Column: 综述 Public date: 2022-09-05
Title:
An overview of underwater image enhancement methods
Author(s):
YAN Zheping1; 2;  QU Siyu1; 2;  XING Wen1; 2
1. Department of Intelligent Science and Engineering, Harbin Engineering University, Harbin 150001, China;
2. Qingdao Innovation and Development Center, Harbin Engineering University, Qingdao 266000, China
Keywords:
underwater imaging; image processing; underwater image enhancement; underwater image restoration; imaging model; generative adversarial network; convolutional neural network; image quality assessment
CLC:
TP391.41; TN911.73
DOI:
10.11992/tis.202108022
Abstract:
The underwater image enhancement technology plays an important role in deep-sea exploration missions for underwater manned/unmanned vehicles, which adopts relevant and comprehensive theoretical knowledge of signal processing, image processing, and machine learning to realize the flexible enhancement of underwater images. In this paper, a brief description of the research background, significance, and the hotspot of underwater image enhancement is provided first. Then, the underwater image enhancement methods are expounded, considering the image-formation-model-free, image-formation-model-based, and learning-based approaches. Particularly, the design principles and technical characteristics of different methods are analyzed. Finally, considering the shortcomings of the existing methods and recent major challenges of the underwater image enhancement technology, the future research direction and development trend of this technology are summarized and prospected.
References:
[1] SCHECHNER Y Y, KARPEL N. Recovery of underwater visibility and structure by polarization analysis[J]. IEEE journal of oceanic engineering, 2005, 30(3): 570–587.
[2] TREIBITZ T, SCHECHNER Y Y. Active polarization descattering[J]. IEEE transactions on pattern analysis and machine intelligence, 2009, 31(3): 385?399.
[3] LI Hailan, WANG Xia, BAI Tingzhu, et al. Speckle noise suppression of range gated underwater imaging system[C]// Proc SPIE 7443, applications of digital image processing XXXII, San Diego, 2009, 7443: 641?648.
[4] ROSER M, DUNBABIN M, GEIGER A. Simultaneous underwater visibility assessment, enhancement and improved stereo[C]// IEEE International Conference on Robotics and Automation, Hong Kong, China，2014: 3840?3847.
[5] LU Huimin, LI Yujie, ZHANG Yudong, et al. Underwater optical image processing: a comprehensive review[J]. Mobile networks and applications, 2017, 22(6): 1204–1211.
[6] 王浩, 张叶, 沈宏海, 等. 图像增强算法综述[J]. 中国光学, 2017, 10(4): 438–448
WANG Hao, ZHANG Ye, SHEN Honghai, et al. Review of image enhancement algorithms[J]. Chinese optics, 2017, 10(4): 438–448
[7] 丁畅, 董丽丽, 许文海. “直方图”均衡化图像增强技术研究综述[J]. 计算机工程与应用, 2017, 53(23): 12–17
DING Chang, DONG Lili, XU Wenhai. Review of “histogram” equalization technique for image enhancement[J]. Computer engineering and applications, 2017, 53(23): 12–17
[8] 董丽丽, 丁畅, 许文海. 基于直方图均衡化图像增强的两种改进方法[J]. 电子学报, 2018, 46(10): 2367–2375
DONG Lili, DING Chang, XU Wenhai. Two improved methods based on histogram equalization for image enhancement[J]. Acta electronica sinica, 2018, 46(10): 2367–2375
[9] HUANG S C, CHENG F C, CHIU Y S. Efficient contrast enhancement using adaptive gamma correction with weighting distribution[J]. IEEE transactions on image processing, 2013, 22(3): 1032–1041.
[10] 刘志成, 王殿伟, 刘颖, 等. 基于二维伽马函数的光照不均匀图像自适应校正算法[J]. 北京理工大学学报, 2016, 36(2): 191–196, 214
LIU Zhicheng, WANG Dianwei, LIU Ying, et al. Adaptive adjustment algorithm for non-uniform illumination images based on 2D gamma function[J]. Transactions of Beijing institute of technology, 2016, 36(2): 191–196, 214
[11] 冯维, 吴贵铭, 赵大兴, 等. 多图像融合Retinex用于弱光图像增强[J]. 光学精密工程, 2020, 28(3): 736–744
FENG Wei, WU Guiming, ZHAO Daxing, et al. Multi images fusion Retinex for low light image enhancement[J]. Optics and precision engineering, 2020, 28(3): 736–744
[12] 何笑, 王刚, 贺欢. 融合引导滤波和小波变换的水下图像增强算法[J]. 计算技术与自动化, 2021, 40(1): 114–118
HE Xiao, WANG Gang, HE Huan. Underwater image enhancement algorithm combining guided filtering and wavelet transform[J]. Computing technology and automation, 2021, 40(1): 114–118
[13] 李庆忠, 刘清. 基于小波变换的低照度图像自适应增强算法[J]. 中国激光, 2015, 42(2): 280–286
LI Qingzhong, LIU Qing. Adaptive enhancement algorithm for low illumination images based on wavelet transform[J]. Chinese journal of lasers, 2015, 42(2): 280–286
[14] WANG Jianhua, WANG Huibin, GAO Guowei, et al. Single underwater image enhancement based on: norm decomposition[J]. IEEE access, 2019(7): 145199–145213.
[15] 郭继昌, 李重仪, 郭春乐, 等. 水下图像增强和复原方法研究进展[J]. 中国图象图形学报, 2017, 22(3): 273–287
GUO Jichang, LI Chongyi, GUO Chunle, et al. Research progress of underwater image enhancement and restoration methods[J]. Journal of image and graphics, 2017, 22(3): 273–287
[16] 宋瑞霞, 李达, 王小春. 基于HSI色彩空间的低照度图像增强算法[J]. 图学学报, 2017, 38(2): 217–223
SONG Ruixia, LI Da, WANG Xiaochun. Low illumination image enhancement algorithm based on HSI color space[J]. Journal of graphics, 2017, 38(2): 217–223
[17] 贾芃, 李博, 赵晓龙. 基于HSI的改进Retinex水下图像增强算法[J]. 实验室研究与探索, 2020, 39(12): 1–4, 14
JIA Peng, LI Bo, ZHAO Xiaolong. Improved retinex underwater image enhancement algorithm based on HSI model[J]. Research and exploration in laboratory, 2020, 39(12): 1–4, 14
[18] 田会娟, 蔡敏鹏, 关涛, 等. 基于YCbCr颜色空间的Retinex低照度图像增强方法研究[J]. 光子学报, 2020, 49(2): 0210002
TIAN Huijuan, CAI Minpeng, GUAN Tao, et al. Low-light image enhancement method using retinex method based on YCbCr color space[J]. Acta photonica sinica, 2020, 49(2): 0210002
[19] 郝志成, 吴川, 杨航, 等. 基于双边纹理滤波的图像细节增强方法[J]. 中国光学, 2016, 9(4): 423–431
HAO Zhicheng, WU Chuan, YANG Hang, et al. Image detail enhancement method based on multi-scale bilateral texture filter[J]. Chinese optics, 2016, 9(4): 423–431
[20] SCHECHNER Y Y, KARPEL N. Clear underwater vision[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR. Washington, DC, IEEE, 2004: I.
[21] SCHETTINI R, CORCHS S. Underwater image processing: state of the art of restoration and image enhancement methods[J]. EURASIP journal on advances in signal processing, 2010, 2010: 14.
[22] HE Kaiming, SUN Jian, TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 33(12): 2341–2353.
[23] GALDRAN A, PARDO D, PICóN A, et al. Automatic Red-Channel underwater image restoration[J]. Journal of visual communication and image representation, 2015, 26: 132–145.
[24] DREWS P L J, NASCIMENTO E R, BOTELHO S S C, et al. Underwater depth estimation and image restoration based on single images[J]. IEEE computer graphics and applications, 2016, 36(2): 24–35.
[25] AKKAYNAK D, TREIBITZ T. Sea-thru: a method for removing water from underwater images[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, IEEE, 2019: 1682?1691.
[26] BERMAN D, LEVY D, AVIDAN S, et al. Underwater single image color restoration using haze-lines and a new quantitative dataset[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(8): 2822–2837.
[27] MOGHIMI M K, MOHANNA F. Real-time underwater image resolution enhancement using super-resolution with deep convolutional neural networks[J]. Journal of real-time image processing, 2021, 18(5): 1653–1667.
[28] 马悦. 基于深度学习的低照度图像增强方法[J]. 信息技术, 2021, 45(1): 85–89
MA Yue. Low-light image enhancement method based on deep learning[J]. Information technology, 2021, 45(1): 85–89
[29] UPLAVIKAR P, WU ZHENYU, WANG ZHANGYANG. All-in-one underwater image enhancement using domain-adversarial learning[EB/OL]. (2019?01?01)[2020?01?01].https: //arxiv. org/abs/1905.13342.
[30] 刘超, 张晓晖. 超低照度下微光图像的深度卷积自编码网络复原[J]. 光学 精密工程, 2018, 26(4): 951–961
LIU Chao, ZHANG Xiaohui. Deep convolutional autoencoder networks approach to low-light level image restoration under extreme low-light illumination[J]. Optics and precision engineering, 2018, 26(4): 951–961
[31] WU Zhenyu, WANG Zhangyang, WANG Zhaowen, et al. Towards privacy-preserving visual recognition via adversarial training: a pilot study[M]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 627?645.
[32] HUANG Xun, LIU Mingyu, BELONGIE S, et al. Multimodal unsupervised image-to-image translation[M]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 179?196.
[33] LIU Mingyu, BREUEL T, KAUTZ J. Unsupervised image-to-image translation networks[C]//NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 700?708.
[34] WANG Yang, ZHANG Jing, CAO Yang, et al. A deep CNN method for underwater image enhancement[C]//2017 IEEE International Conference on Image Processing. Beijing. IEEE, 2017: 1382?1386.
[35] CAO Keming, PENG Y T, COSMAN P C. Underwater image restoration using deep networks to estimate background light and scene depth[C]//2018 IEEE Southwest Symposium on Image Analysis and Interpretation. Las Vegas, IEEE, 2018: 1?4.
[36] FAYAZ S, PARAH S A, QURESHI G J, et al. Underwater image restoration: a state-of-the-art review[J]. IET image processing, 2021, 15(2): 269–285.
[37] 徐岩, 孙美双. 基于卷积神经网络的水下图像增强方法[J]. 吉林大学学报(工学版), 2018, 48(6): 1895–1903
XU Yan, SUN Meishuang. Enhancing underwater image based on convolutional neural networks[J]. Journal of Jilin university (engineering and technology edition), 2018, 48(6): 1895–1903
[38] SUN Xin, LIU Lipeng, LI Qiong, et al. Deep pixel-to-pixel network for underwater image enhancement and restoration[J]. IET image processing, 2019, 13(3): 469–474.
[39] 江泽涛, 覃露露. 一种基于U-Net生成对抗网络的低照度图像增强方法[J]. 电子学报, 2020, 48(2): 258–264
JIANG Zetao, QIN Lulu. Low-light image enhancement method based on U-net generative adversarial network[J]. Acta electronica sinica, 2020, 48(2): 258–264
[40] DENG Junjie, LUO Gege, ZHAO Caidan. UCT-GAN: underwater image colour transfer generative adversarial network[J]. IET image processing, 2020, 14(14): 3613–3622.
[41] ZHANG Tingting, LI Yujie, TAKAHASHI S. Underwater image enhancement using improved generative adversarial network[J]. Concurrency and computation: practice and experience, 2021, 33(22).
[42] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision. Venice, IEEE, 2017: 2242?2251.
[43] LU Jingyu, LI Na, ZHANG Shaoyong, et al. Multi-scale adversarial network for underwater image restoration[J]. Optics & laser technology, 2019, 110: 105–113.
[44] CHOI Y, CHOI M, KIM M, et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, IEEE, 2018: 8789?8797.
[45] LI Jie, SKINNER K A, EUSTICE R M, et al. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE robotics and automation letters, 2018, 3(1): 387–394.
[46] YE Xinchen, LI Zheng, SUN Baoli, et al. Deep joint depth estimation and color correction from monocular underwater images based on unsupervised adaptation networks[J]. IEEE transactions on circuits and systems for video technology, 2020, 30(11): 3995–4008.
[47] ZHAO Hang, GALLO O, FROSIO I, et al. Loss functions for image restoration with neural networks[J]. IEEE transactions on computational imaging, 2017, 3(1): 47–57.
[48] 程宇, 邓德祥, 颜佳, 等. 基于卷积神经网络的弱光照图像增强算法[J]. 计算机应用, 2019, 39(4): 1162–1169
CHENG Yu, DENG Dexiang, YAN Jia, et al. Weakly illuminated image enhancement algorithm based on convolutional neural network[J]. Journal of computer applications, 2019, 39(4): 1162–1169
[49] GUO Yecai, LI Hanyu, ZHUANG Peixian. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE journal of oceanic engineering, 2020, 45(3): 862–870.
[50] LIU Peng, WANG Guoyu, QI Hao, et al. Underwater image enhancement with a deep residual framework[J]. IEEE access, 2019(7): 94614–94629.
[51] HAN Yanling, HUANG Lihua, HONG Zhonghua, et al. Deep supervised residual dense network for underwater image enhancement[J]. Sensors (Basel, Switzerland), 2021, 21(9)–3289.
[52] WANG Jing, LI Ping, DENG Jianhua, et al. CA-GAN: class-condition attention GAN for underwater image enhancement[J]. IEEE access, 2020(8): 130719–130728.
[53] 王越, 王德兴, 袁红春, 等. 基于金字塔注意力机制和GAN的水下图像增强[J]. 激光与光电子学进展, 2021, 48(16): 161006–161010
WANG Yue, WANG Dexing, YUAN Hongchun, et al. Underwater image enhancement based on pyramid attention mechanism and generative adversarial network[J]. Laser and Optoelect-ronics Progress, 2021, 48(16): 161006–161010
[54] LI Yang, CHEN Rong. UDA-Net: densely attention network for underwater image enhancement[J]. IET image processing, 2021, 15(3): 774–785.
[55] YANG Xi, LI Hui, CHEN Rong. Underwater image enhancement with image colorfulness measure[J]. Signal processing:image communication, 2021, 95: 116225.
[56] WANG Kun, SHEN Liquan, LIN Yufei, et al. Joint iterative color correction and dehazing for underwater image enhancement[J]. IEEE robotics and automation letters, 2021, 6(3): 5121–5128.
[57] 黄鐄, 陶海军, 王海峰. 条件生成对抗网络的低照度图像增强方法[J]. 中国图象图形学报, 2019, 24(12): 2149–2158
HUANG Huang, TAO Haijun, WANG Haifeng. Low-illumination image enhancement using a conditional generative adversarial network[J]. Journal of image and graphics, 2019, 24(12): 2149–2158
[58] LIU Xiaodong, GAO Zhi, CHEN B M. MLFcGAN: multilevel feature fusion-based conditional GAN for underwater image color correction[J]. IEEE geoscience and remote sensing letters, 2020, 17(9): 1488–1492.
[59] YANG Miao, HU Ke, DU Yixiang, et al. Underwater image enhancement based on conditional generative adversarial network[J]. Signal processing:image communication, 2020, 81: 115723.
[60] ZONG Xianhui, CHEN Zhehan, WANG Dadong. Local-CycleGAN: a general end-to-end network for visual enhancement in complex deep-water environment[J]. Applied intelligence, 2021, 51(4): 1947–1958.
[61] 晋玮佩, 郭继昌, 祁清. 基于条件生成对抗网络的水下图像增强[J]. 激光与光电子学进展, 2020, 57(14): 141002.
JIN Weipei, GUO Jichang, QI Qing. Underwater image enhancement based on conditional generative adversarial network[J]. Laser & optoelectronics progress, 2020, 57(14): 141002.
[62] SIMONYAN K, ZISSERMAN A J C S. Very deep convolutional networks for large-scale image recognition[C]// IEEE International Conference on Learning Representations, San Diego, 2015.
[63] ZHANG Huiqing, SUN Luyu, WU Lifang, et al. DuGAN: an effective framework for underwater image enhancement[J]. IET image processing, 2021, 15(9): 2010–2019.
[64] PANETTA K, GAO Chen, AGAIAN S. Human-visual-system-inspired underwater image quality measures[J]. IEEE journal of oceanic engineering, 2016, 41(3): 541–551.
[65] YANG Miao, SOWMYA A. An underwater color image quality evaluation metric[J]. IEEE transactions on image processing, 2015, 24(12): 6062–6071.
[66] LI Zongying, GU Zhaorui, ZHENG Haiyong, et al. Underwater image sharpness assessment based on selective attenuation of color in the water[C]//OCEANS 2016 - Shanghai. Shanghai, IEEE, 2016: 1?4.
[67] WANG Yan, LI Na, LI Zongying, et al. An imaging-inspired no-reference underwater color image quality assessment metric[J]. Computers & electrical engineering, 2018, 70: 904–913.
[68] LI Chongyi, GUO Chunle, REN Wenqi, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE transactions on image processing, 2020, 29: 4376–4389.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems