[1]WANG Keping,YANG Yi,FEI Shumin.Review of hazy image sharpening methods[J].CAAI Transactions on Intelligent Systems,2023,18(2):217-230.[doi:10.11992/tis.202110029]
Copy

Review of hazy image sharpening methods

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 2 Page number: 217-230 Column: 综述 Public date: 2023-05-05
Title:
Review of hazy image sharpening methods
Author(s):
WANG Keping1;  YANG Yi1;  FEI Shumin2
1. School of Electrical Engineering and Automation, He’nan Polytechnic University, Jiaozuo 454003;
2. School of Automation, Southeast University, Nanjing 210096, China
Keywords:
image sharpening; image dehazing; ill-posed problem; image degradation; atmospheric scattering model; deep learning; model-free; uneven hazy image
CLC:
TP391.41
DOI:
10.11992/tis.202110029
Abstract:
Hazy images not only induce visual effects but also easily introduce difficulties to subsequent high-level tasks, such as image recognition and understanding. Image dehazing is a typical ill-posed problem, and accurately modeling the imaging process is difficult. Therefore, eliminating the haze in the image faces enormous challenges. Researchers have proposed numerous methods to overcome the hazy image degradation caused by haze. First, this paper summarizes the image dehazing methods to understand and organize them. Whether the haze degradation process is supported by the model, the clarity algorithm is generally divided into a Retinex-based model, atmospheric scattering model defogging algorithm, and model-free image dehazing algorithm. The atmospheric scattering model is the typical model-based dehazing. The imaging mechanism of the model is comprehensively analyzed. However, addressing the non-uniform hazy image problem is difficult because the atmospheric scattering model is easily restricted to an assumption that the atmospheric concentration is distributed uniformly. The deep learning-based model-free dehazing algorithm not only deals with the non-uniform hazy image but also gains a considerable improvement in dehazing performance. Second, this paper summarizes the commonly used image dehazing data sets in recent years and compares the data sets from multiple dimensions, such as the scope of application, scale, and expandability. Moreover, the influence of a synthetic hazy data set and the data set of images shot in reality on the image dehazing algorithm is qualitatively and quantitatively discussed in accordance with the formation mode of hazy images.
References:
[1] SHEKAR A K, GOU Liang, REN Liu, et al. Label-free robustness estimation of object detection CNNs for autonomous driving applications[J]. International journal of computer vision, 2021, 129(4): 1185–1201.
[2] XIAO H, MOSHREFI A, TURNER A, et al. Methods and systems for automobile security monitoring[P]. US, 20120162423A1, 2012.
[3] LI Yanshan, XIA Rongjie, LIU Xing. Learning shape and motion representations for view invariant skeleton-based action recognition[J]. Pattern recognition, 2020, 103: 107293.
[4] ZHAO Zhongqiu, ZHENG Peng, XU Shoutao, et al. Object detection with deep learning: a review[J]. IEEE transactions on neural networks and learning systems, 2019, 30(11): 3212–3232.
[5] WANG Guangting, LUO Chong, XIONG Zhiwei, et al. SPM-tracker: series-parallel matching for real-time visual object tracking[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3638?3647.
[6] LAND E H. The retinex[J]. American scientist, 1964, 52(2): 247–264.
[7] NAYAR S K, NARASIMHAN S G. Vision in bad weather[C]//Proceedings of the seventh IEEE International Conference on Computer Vision. Kerkyra: IEEE, 1999: 820-827.
[8] NARASIMHAN S G, NAYAR S K. Vision and the atmosphere[J]. International journal of computer vision, 2002, 48(3): 233–254.
[9] NARASIMHAN S G, NAYAR S K. Contrast restoration of weather degraded images[J]. IEEE transactions on pattern analysis and machine intelligence, 2003, 25(6): 713–724.
[10] 杨艳飞. 一般形式正则化的大规模离散线性不适定问题算法的研究[D]. 北京: 清华大学, 2018.
YANG Yanfei. Algorithms for large scale discrete linear ill-posed problems with general-form regularization[D]. Beijing: Tsinghua University, 2018.
[11] XU Zhiyuan, LIU Xiaoming, JI Na. Fog removal from color images using contrast limited adaptive histogram equalization[C]//2009 2nd International Congress on Image and Signal Processing. Tianjin: IEEE, 2009: 1?5.
[12] 牛宏侠, 王春智. 基于HSI空间的沙尘图像增强算法[J]. 北京交通大学学报, 2022, 46(5): 1–8
NIU Hongxia, WANG Chunzhi. Sand-dust image enhancement algorithm based on HSI space[J]. Journal of Beijing Jiaotong University, 2022, 46(5): 1–8
[13] 程新. 基于同态滤波的图像增强算法研究[D]. 西安: 西安邮电大学, 2016.
[14] LI Boyi, REN Wenqi, FU Dengpan, et al. Benchmarking single-image dehazing and beyond[J]. IEEE transactions on image processing, 2019, 28(1): 492–505.
[15] ZHANG Yanfu, DING Li, SHARMA G. HazeRD: an outdoor scene dataset and benchmark for single image dehazing[C]//2017 IEEE International Conference on Image Processing. Beijing: IEEE, 2017: 3205?3209.
[16] SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor Segmentation and Support Inference from RGBD Images[C]//European Conference on Computer Vision. Berlin: Springer, 2012: 746?760.
[17] ANCUTI C, ANCUTI C O, DE VLEESCHOUWER C. D-HAZY: a dataset to evaluate quantitatively dehazing algorithms[C]//2016 IEEE International Conference on Image Processing. Phoenix: IEEE, 2016: 2226-2230.
[18] CAI Bolun, XU Xiangmin, JIA Kui, et al. DehazeNet: an end-to-end system for single image haze removal[J]. IEEE transactions on image processing: a publication of the IEEE Signal Processing Society, 2016, 25(11): 5187–5198.
[19] ANCUTI C, ANCUTI C O, TIMOFTE R, et al. I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images[M]//Advanced Concepts for Intelligent Vision Systems. Cham: Springer International Publishing, 2018: 620?631.
[20] ANCUTI C O, ANCUTI C, TIMOFTE R, et al. O-HAZE: a dehazing benchmark with real hazy and haze-free outdoor images[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE, 2018: 867?8678.
[21] ANCUTI C O, ANCUTI C, TIMOFTE R. NH-HAZE: an image dehazing benchmark with non-homogeneous hazy and haze-free images[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE, 2020: 1798?1805.
[22] PETRO A B, SBERT C, MOREL J M. Multiscale retinex[J]. Image processing on line, 2014, 4: 71–88.
[23] XIE F, TANG M, ZHANG R. Review of image enhancement algorithms based on Retinex[J]. Journal of data acquisition and processing, 2019, 34(1): 1–11.
[24] LAND E H, MCCANN J J. Lightness and retinex theory[J]. Journal of the optical society of America, 1971, 61(1): 1.
[25] 顾振飞, 张登银. 基于变分Retinex模型的雾天图像增强方法[J]. 中国矿业大学学报, 2018, 47(6): 1386–1394
GU Zhenfei, ZHANG Dengyin. A hazy image enhancement method based on the variational Retinex model[J]. Journal of China University of Mining & Technology, 2018, 47(6): 1386–1394
[26] WANG Wei, HE Chuanjiang. A variational model with barrier functionals for retinex[J]. SIAM journal on imaging sciences, 2015, 8(3): 1955–1980.
[27] SWINEHART D F. The beer-lambert law[J]. Journal of chemical education, 1962, 39(7): 333.
[28] HE Kaiming, SUN Jian, TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 33(12): 2341–2353.
[29] 黄富瑜, 李刚, 邹昌帆, 等. 暗原色先验自适应图像去雾方法[J]. 光电子·激光, 2019, 30(12): 1323–1330
HUANG Fuyu, LI Gang, ZOU Changfan, et al. Adaptive image dehazing based on dark channel prior[J]. Journal of optoelectronics·laser, 2019, 30(12): 1323–1330
[30] 黄鹤, 李昕芮, 宋京, 等. 多尺度窗口的自适应透射率修复交通图像去雾方法[J]. 中国光学, 2019, 12(6): 1311–1320
HUANG He, LI Xinrui, SONG Jing, et al. A traffic image dehaze method based on adaptive transmittance estimation with multi-scale window[J]. Chinese optics, 2019, 12(6): 1311–1320
[31] 杨燕, 王志伟. 基于补偿透射率和自适应雾浓度系数的图像复原算法[J]. 通信学报, 2020, 41(1): 66–75
YANG Yan, WANG Zhiwei. Image restoration algorithm based on compensated transmission and adaptive haze concentration coefficient[J]. Journal on communications, 2020, 41(1): 66–75
[32] ZHU Qingsong, MAI Jiaming, SHAO Ling. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE transactions on image processing, 2015, 24(11): 3522–3533.
[33] 赵小强, 宋昭漾. 多级跳线连接的深度残差网络超分辨率重建[J]. 电子与信息学报, 2019, 41(10): 2501–2508
ZHAO Xiaoqiang, SONG Zhaoyang. Super-resolution reconstruction of deep residual network with multi-level skip connections[J]. Journal of electronics & information technology, 2019, 41(10): 2501–2508
[34] KAWULOK M, BENECKI P, PIECHACZEK S, et al. Deep learning for multiple-image super-resolution[J]. IEEE geoscience and remote sensing letters, 2020, 17(6): 1062–1066.
[35] MAEDA S. Unpaired image super-resolution using pseudo-supervision[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 288?297.
[36] LIU Hongyu, JIANG Bin, XIAO Yi, et al. Coherent semantic attention for image inpainting[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 4169?4178.
[37] REN Wenqi, LIU Si, ZHANG Hua, et al. Single image dehazing via multi-scale convolutional neural networks[M]//Computer Vision-ECCV 2016. Cham: Springer International Publishing, 2016: 154?169.
[38] LI Boyi, PENG Xiulian, WANG Zhangyang, et al. AOD-net: all-in-one dehazing network[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4780?4788.
[39] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137–1149.
[40] AMODIO M, ASSOUEL R, SCHMIDT V, et al. Image-to-image Mapping with Many Domains by Sparse Attribute Transfer[EB/OL].(2020?01?23)[2021?09?12].https://arxiv.org/abs/2006.13291.
[41] 张惊雷, 厚雅伟. 基于改进循环生成式对抗网络的图像风格迁移[J]. 电子与信息学报, 2020, 42(5): 1216–1222
ZHANG Jinglei, HOU Yawei. Image-to-image translation based on improved cycle-consistent generative adversarial network[J]. Journal of electronics & information technology, 2020, 42(5): 1216–1222
[42] PATHAK D, KR?HENBüHL P, DONAHUE J, et al. Context encoders: feature learning by inpainting[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2536?2544.
[43] ZHAO Lei, MO Qihang, LIN Sihuan, et al. UCTGAN: diverse image inpainting based on unsupervised cross-space translation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 5740?5749.
[44] LEDIG C, THEIS L, HUSZáR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[EB/OL].(2016?09?15)[2021?09?12].https://arxiv.org/abs/1609.04802v1.
[45] HYUN S, HEO J P. VarSR: variational super-resolution network for very low resolution images[C]//European Conference on Computer Vision. Cham: Springer, 2020: 431?447.
[46] ZHU H, PENG X, CHANDRASEKHAR V, et al. DehazeGAN: when image dehazing meets differential programming[C]// Twenty-Seventh International Joint Conference on Artificial Intelligence IJCAI-18. Stockholm: International Joint Conferences on Artificial Intelligence, 2018:1234?1240.
[47] ZHANG He, PATEL V M. Densely connected pyramid dehazing network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3194?3203.
[48] DAS S D, DUTTA S. Fast deep multi-patch hierarchical network for nonhomogeneous image dehazing[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE, 2020: 1994?2001.
[49] ZHANG Hongguang, DAI Yuchao, LI Hongdong, et al. Deep stacked hierarchical multi-patch network for image deblurring[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5971?5979.
[50] WANG Keping, YANG Yi, LI Bingfeng, et al. Uneven image dehazing by heterogeneous twin network[J]. IEEE access, 2020, 8: 118485–118496.
[51] LI Runde, PAN Jinshan, LI Zechao, et al. Single image dehazing via conditional generative adversarial network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8202?8211.
[52] ENGIN D, GENC A, EKENEL H K. Cycle-dehaze: enhanced CycleGAN for single image dehazing[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE, 2018: 938?9388.
[53] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2242?2251.
[54] BHARATH RAJ N, VENKETESWARAN N. Single image haze removal using a generative adversarial network[C]//2020 International Conference on Wireless Communications Signal Processing and Networking. Chennai: IEEE, 2020: 37?42.
[55] JéGOU S, DROZDZAL M, VAZQUEZ D, et al. The one hundred layers tiramisu: fully convolutional DenseNets for semantic segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE, 2017: 1175?1183.
[56] XU K, BA J, KIROS R, et. al. Show, attend and tell: Neural image caption generation with visual attention[EB/OL].(2015?02?10)[2021?09?12].https://arxiv.org/abs/1502.03044.
[57] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[EB/OL].(2017?06?12)[2021?09?12].https://arxiv.org/abs/1706.03762.
[58] WANG Xiaolong, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7794?7803.
[59] ZHANG Yulun, LI Kunpeng, LI Kai, et al. Image super-resolution using very deep residual channel attention networks[C]//European Conference on Computer Vision. Cham: Springer, 2018: 294?310.
[60] QIN Xu, WANG Zhilin, BAI Yuanchao, et al. FFA-net: feature fusion attention network for single image dehazing[J]. Proceedings of the AAAI conference on artificial intelligence, 2020, 34(7): 11908–11915.
[61] ANVARI Z, ATHITSOS V. Dehaze-GLCGAN: unpaired single image de-hazing via adversarial training[EB/OL].(2020?08?15)[2021?09?12].https://arxiv.org/abs/2008.06632.
[62] SCHARSTEIN D, HIRSCHMüLLER H, KITAJIMA Y, et al. High-resolution stereo datasets with subpixel-accurate ground truth[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2014: 31?42.
[63] ANCUTI C O, ANCUTI C, SBERT M, et al. Dense-haze: a benchmark for image dehazing with dense-haze and haze-free images[C]//2019 IEEE International Conference on Image Processing. Taipei: IEEE, 2019: 1014?1018.
[64] BERMAN D, TREIBITZ T, AVIDAN S. Non-local image dehazing[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1674?1682.
[65] GALDRAN A, BRIA A, ALVAREZ-GILA A, et al. On the duality between retinex and image dehazing[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8212?8221.
[66] ZHANG He, SINDAGI V, PATEL V M. Multi-scale single image dehazing using perceptual pyramid deep network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE, 2018: 1015?101509.
[67] LIU Xiaohong, MA Yongrui, SHI Zhihao, et al. GridDehazeNet: attention-based multi-scale network for image dehazing[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 7313?7322.
[68] ANCUTI C, ANCUTI C O, DE VLEESCHOUWER C, et al. Day and night-time dehazing by local airlight estimation[J]. IEEE transactions on image processing:a publication of the IEEE Signal Processing Society, 2020, 29: 6264–6275.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems