[1]王潇棠,闫河,刘建骐,等.一种边缘梯度插值的双分支deeplabv3+语义分割模型[J].智能系统学报,2023,18(3):604-612.[doi:10.11992/tis.202111023]
WANG Xiaotang,YAN He,LIU Jianqi,et al.A new deeplabv3+ semantic segmentation model of edge gradient interpolation with double branch structure[J].CAAI Transactions on Intelligent Systems,2023,18(3):604-612.[doi:10.11992/tis.202111023]
点击复制
WANG Xiaotang,YAN He,LIU Jianqi,et al.A new deeplabv3+ semantic segmentation model of edge gradient interpolation with double branch structure[J].CAAI Transactions on Intelligent Systems,2023,18(3):604-612.[doi:10.11992/tis.202111023]
一种边缘梯度插值的双分支deeplabv3+语义分割模型
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第3期
页码:
604-612
栏目:
学术论文—知识工程
出版日期:
2023-07-05
- Title:
- A new deeplabv3+ semantic segmentation model of edge gradient interpolation with double branch structure
- Keywords:
- edge gradient; image interpolation; bicubic interpolation; bilinear interpolation; deeplabv3+; dual branch structure; decoder; semantic segmentation
- 分类号:
- TP391；TP181
- 摘要:
- 针对deeplabv3+解码器采用双线性插值的单一分支结构易导致图像的高频分量损失、语义分割精度不高的问题，采用索伯(Sobel)算子计算各像素点沿不同方向的边缘梯度值并结合双三次插值算法，提出一种边缘梯度插值方法；在此基础上，对1/8输入图像与编码器输出采用边缘梯度2倍上插值再经特征融合和边缘梯度2倍上插值操作，并与1/4输入图像经特征融合后再进行边缘梯度4倍上插值操作，从而提出一种边缘梯度插值的双分支deeplabv3+意义分割模型。对比实验结果表明，本文方法在VOC2012数据集上较原分割模型平均交并比指标有2.2%的提升，且对图像边缘细节分割有较好的视觉效果。
- Abstract:
- Aiming at the problem that the single branch structure of deeplabv3 + decoder with bilinear interpolation is easy to lead to the loss of high-frequency components of an image and the low accuracy of semantic segmentation, the Sobel operator is used to calculate the edge gradient values of each pixel along different directions, and by combination with bicubic interpolation algorithm, an edge gradient interpolation method is proposed. On this basis, the 1/8 input image and the encoder output are interpolated up to twice the edge gradient, then interpolated up to twice the edge gradient through feature fusion and edge gradient interpolation, and then interpolated up to four times the edge gradient after feature fusion with the 1/4 input image, thereby a double branch deeplabv3 +semantic segmentation model with edge gradient interpolation is established. The comparative experimental results show that the proposed method has improved MIOU indicator by 2.2% on the VOC2012 dataset compared with the original model, and has a better visual effect on image edge detail segmentation.
- 参考文献/References:
-
[1] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL]. (2014?12?22)[2021?11?12].https://arxiv.org/abs/1412.7062. 2014.
[2] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 40(4): 834–848.
[3] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. (2017?06?17) [2021?11?12].https://arxiv.org/abs/1706.05587.
[4] CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision. Cham: Springer, 2018: 833?851.
[5] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(9): 1904–1916.
[6] 任凤雷, 何昕, 魏仲慧, 等. 基于DeepLabV3+与超像素优化的语义分割[J]. 光学精密工程, 2019, 27(12): 2722–2729
REN Fenglei, HE Xin, WEI Zhonghui, et al. Semantic segmentation based on DeepLabV3+ and superpixel optimization[J]. Optical precision engineering, 2019, 27(12): 2722–2729
[7] 吴涛, 董肖莉, 孟伟, 等. 基于语义分割的简洁线条肖像画生成方法[J]. 智能系统学报, 2021, 16(1): 134–141
WU Tao, DONG Xiaoli, MENG Wei, et al. Concise line portrait generation method based on semantic segmentation[J]. CAAI transactions on intelligent systems, 2021, 16(1): 134–141
[8] RUKUNDO O, CAO HANQIANG. Nearest neighbor value interpolation[EB/OL]. (2012?11?08)[2021?11?12].https://arxiv.org/abs/1211.1768.
[9] KIRKLAND E J. Bilinear interpolation[M]// Advanced Computing in Electron Microscopy. Boston: Springer, 2010: 261?263.
[10] HUANG Zhengzhong, CAO Liangcai. Bicubic interpolation and extrapolation iteration method for high resolution digital holographic reconstruction[J]. Optics and lasers in engineering, 2020, 130: 106090.
[11] RUKUNDO O. Effects of improved floor function on the accuracy of bilinear interpolation algorithm[EB/OL]. (2021?04?08)[2021?11?12].https://arxiv.org/abs/2104.03646.
[12] 王庆, 唐涛, 项德良, 等. 基于梯度比率的SAR图像局部特征提取方法研究[J]. 智能系统学报, 2017, 12(3): 286–292
WANG Qing, TANG Tao, XIANG Deliang, et al. Research on local feature extraction of SAR images based on gradient ratio[J]. CAAI transactions on intelligent systems, 2017, 12(3): 286–292
[13] 伍锡如, 凌星雨. 基于改进的Faster RCNN面部表情检测算法[J]. 智能系统学报, 2021, 16(2): 210–217
WU Xiru, LING Xingyu. Facial expression recognition based on improved Faster RCNN[J]. CAAI transactions on intelligent systems, 2021, 16(2): 210–217
[14] 徐聪, 王丽. 基于改进DeepLabv3+网络的图像语义分割方法[J]. 激光与光电子学进展, 2021, 58(16): 1610008
XU Cong, WANG Li. Image semantic segmentation method based on improved DeepPlabv 3+Network[J]. Laser & optoelectronics progress, 2021, 58(16): 1610008
[15] 王兴武, 雷涛, 王营博, 等. 基于多模态互补特征学习的遥感影像语义分割[J]. 智能系统学报, 2022, 17(6): 1123–1133
WANG Xingwu, LEI Tao, WANG Yingbo, et al. Semantic segmentation of remote sensing image based on multimodal complementary feature learning[J]. CAAI transactions on intelligent systems, 2022, 17(6): 1123–1133
[16] PANDA J, MEHER S. An efficient image interpolation using edge-error based sharpening[C]//2020 IEEE 17th India Council International Conference. New Delhi: IEEE, 2021: 1?6.
[17] VINCENT O, FOLORUNSO O. A descriptive algorithm for sobel image edge detection[C]//Proceedings of the 2009 InSITE Conference. Washington: Informing Science Institute. 2009, 40: 97?107.
[18] HOUBRECHTS K, DAS R, KEUPERS M, et al. A complete procedure to prepare virtual clinical trials in digital breast tomosynthesis[C]//Proceedings Volume 12463, Medical Imaging 2023: Physics of Medical Imaging. San Diego: SPIE. 2023, 12463: 267?274.
[19] HUANG Zhengzhong, CAO Liangcai. Bicubic interpolation and extrapolation iteration method for high resolution digital holographic reconstruction[J]. Optics and lasers in engineering, 2020, 130: 106090.
[20] HORé A, ZIOU D. Image quality metrics: PSNR vs. SSIM[C]//2010 20th International Conference on Pattern Recognition. Istanbul: IEEE, 2010: 2366?2369.
[21] HAENDEL A, ANDERSON J G, PILZ M, et al. A frequency-dependent model for the shape of the Fourier amplitude spectrum of acceleration at high frequencies[J]. Bulletin of the seismological society of America, 2020, 110(6): 2743–2754.
[22] YU Lei, DENG Qiuyue, LIU Bin, et al. Dual-branch feature learning network for single image super-resolution[J]. Multimedia tools and applications, 2023: 1?18.
[23] EVERINGHAM M, ALI ESLAMI S M, VAN GOOL L, et al. The pascal visual object classes challenge: a retrospective[J]. International journal of computer vision, 2015, 111(1): 98–136.
[24] LI YUANZHI, WEI C, MA TENGYU. Towards explaining the regularization effect of initial large learning rate in training neural networks[EB/OL]. (2019?07?10)[2021?11?12].https://arxiv.org/abs/1907.04595.
[25] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2999?3007.
备注/Memo
收稿日期:2021-11-12。
基金项目:国家重点研发计划“智能机器人”重点专项项目(2018YFB1308602)；国家自然科学基金面上项目(61173184)；重庆市自然科学基金项目(cstc2018jcyjAX0694).
作者简介:王潇棠,硕士研究生,主要研究方向为机器视觉、语义分割、图像处理、三维重建、人工智能;闫河,教授,博士,主要研究方向为小波分析、压缩感知、机器视觉与视觉测量、模式识别、人工智能。近5年主持国家自然科学基金面上项目1项、省部级科技项目4项,国家重点研发计划2018智能机器人重大专项项目子课题1项(合作单位负责人)、企业项目7项;获省部级科技进步三等奖1项;获重庆市科技进步三等奖1项。获发明专利授权5项,申请发明专利12项。出版学术专著7部(其中第一作者4部),发表学术论文70余篇。;刘建骐,硕士研究生,主要研究方向为图像处理、目标检测、视觉测量、人工智能
通讯作者:闫河.E-mail:cqyanhe@163.com
更新日期/Last Update:
1900-01-01