[1]LIU Wanjun,TONG Chang,QU Haicheng.An antagonistic image shadow removal algorithm based on dilated convolution and attention mechanism[J].CAAI Transactions on Intelligent Systems,2021,16(6):1081-1089.[doi:10.11992/tis.202011022]
Copy

An antagonistic image shadow removal algorithm based on dilated convolution and attention mechanism

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 6 Page number: 1081-1089 Column: 学术论文—智能系统 Public date: 2021-11-05
Title:
An antagonistic image shadow removal algorithm based on dilated convolution and attention mechanism
Author(s):
LIU Wanjun;  TONG Chang;  QU Haicheng
Software College, Liaoning Technical University, Huludao 125105, China
Keywords:
generative adversarial networks; hole convolution; multiple attention; residual network; multi-scale; autocoder; long short-term memory; shadow removal
CLC:
TP391.41
DOI:
10.11992/tis.202011022
Abstract:
To solve the problem of the unobvious shadow removal effect in dark areas or complex textured and penumbra areas, an antagonistic image shadow removal algorithm is proposed based on a dilated convolution and attention mechanism. The algorithm is based on the general idea of generative adversarial networks. First, the dilated convolution is introduced into the residual network, and the user-defined hole residual block is used for feature extraction, expanding the receptive field of feature extraction. Second, in the attention coding stage, four layers of dilated convolution with the same structure are added to provide more abstract and essential global semantic features for the decoding phase with a minimum calculation amount. Finally, the multiple attention mechanism is used to guide the discrimination network to identify the unshadowed image; thus, improving the discrimination network’s ability. The proposed algorithm is tested on image shadow triplets dataset and shadow removal public datasets and achieves the structural similarity of 97.77%. The image feature information of the algorithm is well preserved, the picture is clear, the shadow removal effect is good in the dark area and complex area, and the algorithm has good performance for the penumbra area.
References:
[1] HUANG Xiang, HUA Gang, TUMBLIN J, et al. What characterizes a shadow boundary under the sun and sky?[C]//Proceedings of the 2011 International Conference on Computer Vision. Barcelona, Spain, 2011: 898-905.
[2] FINLAYSON G D, HORDLEY S D, LU Cheng, et al. On the removal of shadows from images[J]. IEEE transactions on pattern analysis and machine intelligence, 2006, 28(1): 59-68.
[3] YU Xiaoming, LI Ge, YING Zhenqiang, et al. A new shadow removal method using color-lines[C]//International Conference on Computer Analysis of Images and Patterns. Ystad, Sweden, 2017: 307-319.
[4] 林雨准, 张保明, 郭海涛, 等. 结合多尺度分割和形态学运算的高分辨率遥感影像阴影检测[J]. 中国图象图形学报, 2018, 23(8): 1263-1272
LIN Yuzhun, ZHANG Baoming, GUO Haitao, et al. Shadow detection from high resolution remote sensing imagery based on multi-scale segmentation and morphology operation[J]. Journal of image and graphics, 2018, 23(8): 1263-1272
[5] GUO Ruiqi, DAI Qieyun, HOIEM D. Single-image shadow detection and removal using paired regions[C]//CVPR 2011. Colorado Springs, USA, 2011: 2033-2040.
[6] FAN Xinyun, WU Wenjun, ZHANG Ling, et al. Shading-aware shadow detection and removal from a single image[J]. The visual computer, 2020, 36(10/11/12): 2175-2188.
[7] 焦俊男, 石静, 田庆久, 等. 多光谱影像的NDVI阴影影响去除模型[J]. 遥感学报, 2020, 24(1): 53-66
JIAO Junnan, SHI Jing, TIAN Qingjiu, et al. Researchon multispectral-image-based NDVI shadow-effect-eliminating model[J]. Journal of remote sensing, 2020, 24(1): 53-66
[8] ZHANG Yindan, CHEN Gang, Vukomanovi J, et al. Recurrent Shadow Attention Model (RSAM) for shadow removal in high-resolution urban land-cover mapping[J]. Remote sensing of environment, 2020, 247: 111945.
[9] 吴文, 万毅. 基于低尺度细节恢复的单幅图像阴影去除方法[J]. 电子学报, 2020, 48(7): 1293-1302
WU Wen, WAN Yi. Single image shadow removal using low-scale detail recovering[J]. Acta electronica sinica, 2020, 48(7): 1293-1302
[10] 张德干, 陈晨, 董悦, 等. 一种基于机器学习的运动目标阴影检测新方法[J]. 光电子·激光, 2018, 29(12): 1317-1324
ZHANG Degan, CHEN Chen, DONG Yue, et al. A new method of moving object shadow detection based on machine learning[J]. Journal of optoelectronics·laser, 2018, 29(12): 1317-1324
[11] HOSSEINZADEH S, SHAKERI M, ZHANG H. Fast shadow detection from a single image using a patched convolutional neural network[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain, 2018: 1-5.
[12] HU Xiaowei, JIANG Yitong, FU C W, et al. Mask-ShadowGAN: learning to remove shadows from unpaired data[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South), 2019: 2472-2481.
[13] FAN Hui, HAN Meng, LI Jinjiang. Image shadow removal using end-to-end deep convolutional neural networks[J]. Applied sciences, 2019, 9(5): 1009-1026.
[14] HU Xiaowei, FU C W, ZHU Lei, et al. Direction-aware spatial context features for shadow detection and removal[J]. IEEE transactions on pattern analysis and machine intelligence, 2020, 42(11): 2795-2808.
[15] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada, 2014: 2672-2680.
[16] 张瑞倩, 邵振峰, PORTNOV A, 等. 多尺度空洞卷积的无人机影像目标检测方法[J]. 武汉大学学报(信息科学版), 2020, 45(6): 895-903
ZHANG Ruiqian, SHAO Zhenfeng, PORTNOV A, et al. Multi-scale dilated convolutional neural network for object detection in UAV images[J]. Journal of Wuhan University (information science edition), 2020, 45(6): 895-903
[17] 颜铭靖, 苏喜友. 基于三维空洞卷积残差神经网络的高光谱影像分类方法[J]. 光学学报, 2020, 40(16): 1628002
YAN Mingjing, SU Xiyou. Hyperspectral image classification based on three-dimensional dilated convolutional residual neural network[J]. Acta optica sinica, 2020, 40(16): 1628002
[18] 贾宇峰, 马力. 条件约束下的自我注意生成对抗网络[J]. 西安电子科技大学学报, 2019, 46(6): 163-170
JIA Yufeng, MA Li. Self-attention generative adversarial network with the conditional constraint[J]. Journal of Xidian University, 2019, 46(6): 163-170
[19] QIAN Rui, TAN R T, YANG Wenhan, et al. Attentive generative adversarial network for raindrop removal from a single image[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 2482-2491.
[20] 刘烨, 黄金筱, 马于涛. 基于混合神经网络和注意力机制的软件缺陷自动分派方法[J]. 计算机研究与发展, 2020, 57(3): 459-473
LIU Ye, HUANG Jinxiao, MA Yutao. An automatic method using hybrid neural networks and attention mechanism for software bug triaging[J]. Journal of Computer research and development, 2020, 57(3): 459-473
[21] WANG Jifeng, LI Xiang, YANG Jian. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 1788-1797.
[22] QU Liangqiong, TIAN Jiandong, HE Shengfeng, et al. DeshadowNet: a multi-context embedding deep network for shadow removal[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017: 2308-2316.
[23] DING Bin, LONG Chengjiang, ZHANG Ling, et al. ARGAN: attentive recurrent generative adversarial network for shadow detection and removal[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South), 2019: 10212-10221.
Similar References:

Memo

-

Last Update: 2021-12-25

Copyright © CAAI Transactions on Intelligent Systems