[1]严浙平,曲思瑜,邢文.水下图像增强方法研究综述[J].智能系统学报,2022,17(5):860-873.[doi:10.11992/tis.202108022]
YAN Zheping,QU Siyu,XING Wen.An overview of underwater image enhancement methods[J].CAAI Transactions on Intelligent Systems,2022,17(5):860-873.[doi:10.11992/tis.202108022]
点击复制
YAN Zheping,QU Siyu,XING Wen.An overview of underwater image enhancement methods[J].CAAI Transactions on Intelligent Systems,2022,17(5):860-873.[doi:10.11992/tis.202108022]
水下图像增强方法研究综述
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第5期
页码:
860-873
栏目:
综述
出版日期:
2022-09-05
- Title:
- An overview of underwater image enhancement methods
- Keywords:
- underwater imaging; image processing; underwater image enhancement; underwater image restoration; imaging model; generative adversarial network; convolutional neural network; image quality assessment
- 分类号:
- TP391.41; TN911.73
- 文献标志码:
- 2022-06-17
- 摘要:
- 水下图像增强是水下有人/无人设备完成深海探测任务的重要支撑技术。该技术综合应用信号处理、图像处理以及机器学习的相关理论知识以实现对水下图像的灵活增强。在简述了水下图像增强的研究背景、意义及热点问题的基础上，按照不基于成像模型、基于成像模型与基于学习3个方向对水下图像增强技术的发展进行了详细的论述，重点分析了不同方法的原理和技术特点。最后，根据水下图像增强技术的难点与目前面临的主要问题对研究方向和发展趋势进行了归纳和展望。
- Abstract:
- The underwater image enhancement technology plays an important role in deep-sea exploration missions for underwater manned/unmanned vehicles, which adopts relevant and comprehensive theoretical knowledge of signal processing, image processing, and machine learning to realize the flexible enhancement of underwater images. In this paper, a brief description of the research background, significance, and the hotspot of underwater image enhancement is provided first. Then, the underwater image enhancement methods are expounded, considering the image-formation-model-free, image-formation-model-based, and learning-based approaches. Particularly, the design principles and technical characteristics of different methods are analyzed. Finally, considering the shortcomings of the existing methods and recent major challenges of the underwater image enhancement technology, the future research direction and development trend of this technology are summarized and prospected.
- 参考文献/References:
-
[1] SCHECHNER Y Y, KARPEL N. Recovery of underwater visibility and structure by polarization analysis[J]. IEEE journal of oceanic engineering, 2005, 30(3): 570–587.
[2] TREIBITZ T, SCHECHNER Y Y. Active polarization descattering[J]. IEEE transactions on pattern analysis and machine intelligence, 2009, 31(3): 385?399.
[3] LI Hailan, WANG Xia, BAI Tingzhu, et al. Speckle noise suppression of range gated underwater imaging system[C]// Proc SPIE 7443, applications of digital image processing XXXII, San Diego, 2009, 7443: 641?648.
[4] ROSER M, DUNBABIN M, GEIGER A. Simultaneous underwater visibility assessment, enhancement and improved stereo[C]// IEEE International Conference on Robotics and Automation, Hong Kong, China，2014: 3840?3847.
[5] LU Huimin, LI Yujie, ZHANG Yudong, et al. Underwater optical image processing: a comprehensive review[J]. Mobile networks and applications, 2017, 22(6): 1204–1211.
[6] 王浩, 张叶, 沈宏海, 等. 图像增强算法综述[J]. 中国光学, 2017, 10(4): 438–448
WANG Hao, ZHANG Ye, SHEN Honghai, et al. Review of image enhancement algorithms[J]. Chinese optics, 2017, 10(4): 438–448
[7] 丁畅, 董丽丽, 许文海. “直方图”均衡化图像增强技术研究综述[J]. 计算机工程与应用, 2017, 53(23): 12–17
DING Chang, DONG Lili, XU Wenhai. Review of “histogram” equalization technique for image enhancement[J]. Computer engineering and applications, 2017, 53(23): 12–17
[8] 董丽丽, 丁畅, 许文海. 基于直方图均衡化图像增强的两种改进方法[J]. 电子学报, 2018, 46(10): 2367–2375
DONG Lili, DING Chang, XU Wenhai. Two improved methods based on histogram equalization for image enhancement[J]. Acta electronica sinica, 2018, 46(10): 2367–2375
[9] HUANG S C, CHENG F C, CHIU Y S. Efficient contrast enhancement using adaptive gamma correction with weighting distribution[J]. IEEE transactions on image processing, 2013, 22(3): 1032–1041.
[10] 刘志成, 王殿伟, 刘颖, 等. 基于二维伽马函数的光照不均匀图像自适应校正算法[J]. 北京理工大学学报, 2016, 36(2): 191–196, 214
LIU Zhicheng, WANG Dianwei, LIU Ying, et al. Adaptive adjustment algorithm for non-uniform illumination images based on 2D gamma function[J]. Transactions of Beijing institute of technology, 2016, 36(2): 191–196, 214
[11] 冯维, 吴贵铭, 赵大兴, 等. 多图像融合Retinex用于弱光图像增强[J]. 光学精密工程, 2020, 28(3): 736–744
FENG Wei, WU Guiming, ZHAO Daxing, et al. Multi images fusion Retinex for low light image enhancement[J]. Optics and precision engineering, 2020, 28(3): 736–744
[12] 何笑, 王刚, 贺欢. 融合引导滤波和小波变换的水下图像增强算法[J]. 计算技术与自动化, 2021, 40(1): 114–118
HE Xiao, WANG Gang, HE Huan. Underwater image enhancement algorithm combining guided filtering and wavelet transform[J]. Computing technology and automation, 2021, 40(1): 114–118
[13] 李庆忠, 刘清. 基于小波变换的低照度图像自适应增强算法[J]. 中国激光, 2015, 42(2): 280–286
LI Qingzhong, LIU Qing. Adaptive enhancement algorithm for low illumination images based on wavelet transform[J]. Chinese journal of lasers, 2015, 42(2): 280–286
[14] WANG Jianhua, WANG Huibin, GAO Guowei, et al. Single underwater image enhancement based on: norm decomposition[J]. IEEE access, 2019(7): 145199–145213.
[15] 郭继昌, 李重仪, 郭春乐, 等. 水下图像增强和复原方法研究进展[J]. 中国图象图形学报, 2017, 22(3): 273–287
GUO Jichang, LI Chongyi, GUO Chunle, et al. Research progress of underwater image enhancement and restoration methods[J]. Journal of image and graphics, 2017, 22(3): 273–287
[16] 宋瑞霞, 李达, 王小春. 基于HSI色彩空间的低照度图像增强算法[J]. 图学学报, 2017, 38(2): 217–223
SONG Ruixia, LI Da, WANG Xiaochun. Low illumination image enhancement algorithm based on HSI color space[J]. Journal of graphics, 2017, 38(2): 217–223
[17] 贾芃, 李博, 赵晓龙. 基于HSI的改进Retinex水下图像增强算法[J]. 实验室研究与探索, 2020, 39(12): 1–4, 14
JIA Peng, LI Bo, ZHAO Xiaolong. Improved retinex underwater image enhancement algorithm based on HSI model[J]. Research and exploration in laboratory, 2020, 39(12): 1–4, 14
[18] 田会娟, 蔡敏鹏, 关涛, 等. 基于YCbCr颜色空间的Retinex低照度图像增强方法研究[J]. 光子学报, 2020, 49(2): 0210002
TIAN Huijuan, CAI Minpeng, GUAN Tao, et al. Low-light image enhancement method using retinex method based on YCbCr color space[J]. Acta photonica sinica, 2020, 49(2): 0210002
[19] 郝志成, 吴川, 杨航, 等. 基于双边纹理滤波的图像细节增强方法[J]. 中国光学, 2016, 9(4): 423–431
HAO Zhicheng, WU Chuan, YANG Hang, et al. Image detail enhancement method based on multi-scale bilateral texture filter[J]. Chinese optics, 2016, 9(4): 423–431
[20] SCHECHNER Y Y, KARPEL N. Clear underwater vision[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR. Washington, DC, IEEE, 2004: I.
[21] SCHETTINI R, CORCHS S. Underwater image processing: state of the art of restoration and image enhancement methods[J]. EURASIP journal on advances in signal processing, 2010, 2010: 14.
[22] HE Kaiming, SUN Jian, TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 33(12): 2341–2353.
[23] GALDRAN A, PARDO D, PICóN A, et al. Automatic Red-Channel underwater image restoration[J]. Journal of visual communication and image representation, 2015, 26: 132–145.
[24] DREWS P L J, NASCIMENTO E R, BOTELHO S S C, et al. Underwater depth estimation and image restoration based on single images[J]. IEEE computer graphics and applications, 2016, 36(2): 24–35.
[25] AKKAYNAK D, TREIBITZ T. Sea-thru: a method for removing water from underwater images[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, IEEE, 2019: 1682?1691.
[26] BERMAN D, LEVY D, AVIDAN S, et al. Underwater single image color restoration using haze-lines and a new quantitative dataset[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(8): 2822–2837.
[27] MOGHIMI M K, MOHANNA F. Real-time underwater image resolution enhancement using super-resolution with deep convolutional neural networks[J]. Journal of real-time image processing, 2021, 18(5): 1653–1667.
[28] 马悦. 基于深度学习的低照度图像增强方法[J]. 信息技术, 2021, 45(1): 85–89
MA Yue. Low-light image enhancement method based on deep learning[J]. Information technology, 2021, 45(1): 85–89
[29] UPLAVIKAR P, WU ZHENYU, WANG ZHANGYANG. All-in-one underwater image enhancement using domain-adversarial learning[EB/OL]. (2019?01?01)[2020?01?01].https: //arxiv. org/abs/1905.13342.
[30] 刘超, 张晓晖. 超低照度下微光图像的深度卷积自编码网络复原[J]. 光学 精密工程, 2018, 26(4): 951–961
LIU Chao, ZHANG Xiaohui. Deep convolutional autoencoder networks approach to low-light level image restoration under extreme low-light illumination[J]. Optics and precision engineering, 2018, 26(4): 951–961
[31] WU Zhenyu, WANG Zhangyang, WANG Zhaowen, et al. Towards privacy-preserving visual recognition via adversarial training: a pilot study[M]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 627?645.
[32] HUANG Xun, LIU Mingyu, BELONGIE S, et al. Multimodal unsupervised image-to-image translation[M]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 179?196.
[33] LIU Mingyu, BREUEL T, KAUTZ J. Unsupervised image-to-image translation networks[C]//NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 700?708.
[34] WANG Yang, ZHANG Jing, CAO Yang, et al. A deep CNN method for underwater image enhancement[C]//2017 IEEE International Conference on Image Processing. Beijing. IEEE, 2017: 1382?1386.
[35] CAO Keming, PENG Y T, COSMAN P C. Underwater image restoration using deep networks to estimate background light and scene depth[C]//2018 IEEE Southwest Symposium on Image Analysis and Interpretation. Las Vegas, IEEE, 2018: 1?4.
[36] FAYAZ S, PARAH S A, QURESHI G J, et al. Underwater image restoration: a state-of-the-art review[J]. IET image processing, 2021, 15(2): 269–285.
[37] 徐岩, 孙美双. 基于卷积神经网络的水下图像增强方法[J]. 吉林大学学报(工学版), 2018, 48(6): 1895–1903
XU Yan, SUN Meishuang. Enhancing underwater image based on convolutional neural networks[J]. Journal of Jilin university (engineering and technology edition), 2018, 48(6): 1895–1903
[38] SUN Xin, LIU Lipeng, LI Qiong, et al. Deep pixel-to-pixel network for underwater image enhancement and restoration[J]. IET image processing, 2019, 13(3): 469–474.
[39] 江泽涛, 覃露露. 一种基于U-Net生成对抗网络的低照度图像增强方法[J]. 电子学报, 2020, 48(2): 258–264
JIANG Zetao, QIN Lulu. Low-light image enhancement method based on U-net generative adversarial network[J]. Acta electronica sinica, 2020, 48(2): 258–264
[40] DENG Junjie, LUO Gege, ZHAO Caidan. UCT-GAN: underwater image colour transfer generative adversarial network[J]. IET image processing, 2020, 14(14): 3613–3622.
[41] ZHANG Tingting, LI Yujie, TAKAHASHI S. Underwater image enhancement using improved generative adversarial network[J]. Concurrency and computation: practice and experience, 2021, 33(22).
[42] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision. Venice, IEEE, 2017: 2242?2251.
[43] LU Jingyu, LI Na, ZHANG Shaoyong, et al. Multi-scale adversarial network for underwater image restoration[J]. Optics & laser technology, 2019, 110: 105–113.
[44] CHOI Y, CHOI M, KIM M, et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, IEEE, 2018: 8789?8797.
[45] LI Jie, SKINNER K A, EUSTICE R M, et al. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE robotics and automation letters, 2018, 3(1): 387–394.
[46] YE Xinchen, LI Zheng, SUN Baoli, et al. Deep joint depth estimation and color correction from monocular underwater images based on unsupervised adaptation networks[J]. IEEE transactions on circuits and systems for video technology, 2020, 30(11): 3995–4008.
[47] ZHAO Hang, GALLO O, FROSIO I, et al. Loss functions for image restoration with neural networks[J]. IEEE transactions on computational imaging, 2017, 3(1): 47–57.
[48] 程宇, 邓德祥, 颜佳, 等. 基于卷积神经网络的弱光照图像增强算法[J]. 计算机应用, 2019, 39(4): 1162–1169
CHENG Yu, DENG Dexiang, YAN Jia, et al. Weakly illuminated image enhancement algorithm based on convolutional neural network[J]. Journal of computer applications, 2019, 39(4): 1162–1169
[49] GUO Yecai, LI Hanyu, ZHUANG Peixian. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE journal of oceanic engineering, 2020, 45(3): 862–870.
[50] LIU Peng, WANG Guoyu, QI Hao, et al. Underwater image enhancement with a deep residual framework[J]. IEEE access, 2019(7): 94614–94629.
[51] HAN Yanling, HUANG Lihua, HONG Zhonghua, et al. Deep supervised residual dense network for underwater image enhancement[J]. Sensors (Basel, Switzerland), 2021, 21(9)–3289.
[52] WANG Jing, LI Ping, DENG Jianhua, et al. CA-GAN: class-condition attention GAN for underwater image enhancement[J]. IEEE access, 2020(8): 130719–130728.
[53] 王越, 王德兴, 袁红春, 等. 基于金字塔注意力机制和GAN的水下图像增强[J]. 激光与光电子学进展, 2021, 48(16): 161006–161010
WANG Yue, WANG Dexing, YUAN Hongchun, et al. Underwater image enhancement based on pyramid attention mechanism and generative adversarial network[J]. Laser and Optoelect-ronics Progress, 2021, 48(16): 161006–161010
[54] LI Yang, CHEN Rong. UDA-Net: densely attention network for underwater image enhancement[J]. IET image processing, 2021, 15(3): 774–785.
[55] YANG Xi, LI Hui, CHEN Rong. Underwater image enhancement with image colorfulness measure[J]. Signal processing:image communication, 2021, 95: 116225.
[56] WANG Kun, SHEN Liquan, LIN Yufei, et al. Joint iterative color correction and dehazing for underwater image enhancement[J]. IEEE robotics and automation letters, 2021, 6(3): 5121–5128.
[57] 黄鐄, 陶海军, 王海峰. 条件生成对抗网络的低照度图像增强方法[J]. 中国图象图形学报, 2019, 24(12): 2149–2158
HUANG Huang, TAO Haijun, WANG Haifeng. Low-illumination image enhancement using a conditional generative adversarial network[J]. Journal of image and graphics, 2019, 24(12): 2149–2158
[58] LIU Xiaodong, GAO Zhi, CHEN B M. MLFcGAN: multilevel feature fusion-based conditional GAN for underwater image color correction[J]. IEEE geoscience and remote sensing letters, 2020, 17(9): 1488–1492.
[59] YANG Miao, HU Ke, DU Yixiang, et al. Underwater image enhancement based on conditional generative adversarial network[J]. Signal processing:image communication, 2020, 81: 115723.
[60] ZONG Xianhui, CHEN Zhehan, WANG Dadong. Local-CycleGAN: a general end-to-end network for visual enhancement in complex deep-water environment[J]. Applied intelligence, 2021, 51(4): 1947–1958.
[61] 晋玮佩, 郭继昌, 祁清. 基于条件生成对抗网络的水下图像增强[J]. 激光与光电子学进展, 2020, 57(14): 141002.
JIN Weipei, GUO Jichang, QI Qing. Underwater image enhancement based on conditional generative adversarial network[J]. Laser & optoelectronics progress, 2020, 57(14): 141002.
[62] SIMONYAN K, ZISSERMAN A J C S. Very deep convolutional networks for large-scale image recognition[C]// IEEE International Conference on Learning Representations, San Diego, 2015.
[63] ZHANG Huiqing, SUN Luyu, WU Lifang, et al. DuGAN: an effective framework for underwater image enhancement[J]. IET image processing, 2021, 15(9): 2010–2019.
[64] PANETTA K, GAO Chen, AGAIAN S. Human-visual-system-inspired underwater image quality measures[J]. IEEE journal of oceanic engineering, 2016, 41(3): 541–551.
[65] YANG Miao, SOWMYA A. An underwater color image quality evaluation metric[J]. IEEE transactions on image processing, 2015, 24(12): 6062–6071.
[66] LI Zongying, GU Zhaorui, ZHENG Haiyong, et al. Underwater image sharpness assessment based on selective attenuation of color in the water[C]//OCEANS 2016 - Shanghai. Shanghai, IEEE, 2016: 1?4.
[67] WANG Yan, LI Na, LI Zongying, et al. An imaging-inspired no-reference underwater color image quality assessment metric[J]. Computers & electrical engineering, 2018, 70: 904–913.
[68] LI Chongyi, GUO Chunle, REN Wenqi, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE transactions on image processing, 2020, 29: 4376–4389.
- 相似文献/References:
-
[1]李庆武,蔡艳梅,徐立中.基于分块分类的智能视频监控背景更新算法[J].智能系统学报,2010,5(3):272.
LI Qing-wu,CAI Yan-mei,XU Li-zhong.Background update algorithm based on blocks classification forintelligent video surveillance[J].CAAI Transactions on Intelligent Systems,2010,5():272.
[2]邵真天,袁杰.一种基于曲波变换的图像去块算法[J].智能系统学报,2012,7(2):102.
SHAO Zhentian,YUAN Jie.An image deblocking algorithm based on curvelet transformation[J].CAAI Transactions on Intelligent Systems,2012,7():102.
[3]王锦榕,袁学海,刘增良.基于图像处理技术的瞳孔和角膜反射中心提取算法[J].智能系统学报,2012,7(5):423.
WANG Jinrong,YUAN Xuehai,LIU Zengliang.An extraction method of pupil and corneal reflection centers based on image processing technology[J].CAAI Transactions on Intelligent Systems,2012,7():423.
[4]傅博,姜勇,王洪光,等.输电线路巡检图像智能诊断系统[J].智能系统学报,2016,11(1):70.[doi:10.11992/tis.201503043]
FU Bo,JIANG Yong,WANG Hongguang,et al.Intelligent diagnosis system for patrol check images of power transmission lines[J].CAAI Transactions on Intelligent Systems,2016,11():70.[doi:10.11992/tis.201503043]
[5]任晓霞,孙秀明,耿鹏,等.多小波和NSDFB组合域递归滤波多聚焦图像融合[J].智能系统学报,2016,11(2):241.[doi:10.11992/tis.201509017]
REN Xiaoxia,SUN Xiuming,GENG Peng,et al.Multifocus image fusion using a recursive filter in the combined domain of multiwavelets and NSDFB[J].CAAI Transactions on Intelligent Systems,2016,11():241.[doi:10.11992/tis.201509017]
[6]梁义辉,战强.一种面向无线图像传输的视觉平台[J].智能系统学报,2016,11(5):608.[doi:10.11992/tis.201512014]
LIANG Yihui,ZHAN Qiang.A visual platform for wireless image transmission[J].CAAI Transactions on Intelligent Systems,2016,11():608.[doi:10.11992/tis.201512014]
[7]李霞丽,吴立成,樊艳明.易于硬件实现的压缩感知观测矩阵的研究与构造[J].智能系统学报,2017,12(3):279.[doi:10.11992/tis.201606037]
LI Xiali,WU Licheng,FAN Yanming.Study and construction of a compressed sensing measurement matrix that is easy to implement in hardware[J].CAAI Transactions on Intelligent Systems,2017,12():279.[doi:10.11992/tis.201606037]
[8]姜婷,袭肖明,岳厚光.基于分布先验的半监督FCM的肺结节分类[J].智能系统学报,2017,12(5):729.[doi:10.11992/tis.201706018]
JIANG Ting,XI Xiaoming,YUE Houguang.Classification of pulmonary nodules by semi-supervised FCM based on prior distribution[J].CAAI Transactions on Intelligent Systems,2017,12():729.[doi:10.11992/tis.201706018]
[9]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
[10]刘彪,黄蓉蓉,林和,等.基于卷积神经网络的盲文音乐识别研究[J].智能系统学报,2019,14(1):186.[doi:10.11992/tis.201805002]
LIU Biao,HUANG Rongrong,LIN He,et al.Research on braille music recognition based on convolutional neural networks[J].CAAI Transactions on Intelligent Systems,2019,14():186.[doi:10.11992/tis.201805002]
备注/Memo
收稿日期:2021-08-17。
基金项目:国家自然科学基金面上项目（52071102）.
作者简介:严浙平，教授，博士生导师，主要研究方向为多传感器数据融合理论及其应用、水下无人航行器集成与控制和无人现场智能控制、系统综合仿真与验证。发表学术论文100余篇，出版专著2部;曲思瑜，硕士研究生，主要研究方向为图像处理、机器学习;邢文，讲师，主要研究方向为水下无人集群与自主智能协同控制、复杂网络与牵制趋同控制、网络攻击与弹性事件触发分布式控制、水下无人集群协同导航定位与非线性滤波。主持国防重点实验室开放基金1项、中央高校基础科研基金项目1项。发表学术论文20余篇
通讯作者:邢文. E-mail：xingwen@hrbeu.edu.cn
更新日期/Last Update:
1900-01-01