[1]庞荣,杨燕,冷雄进,等.基于双分支点流语义先验的路面病害分割模型[J].智能系统学报,2024,19(1):153-164.[doi:10.11992/tis.202306037]
PANG Rong,YANG Yan,LENG Xiongjin,et al.Segmentation model of pavement diseases based on semantic priori of double-branched point flow[J].CAAI Transactions on Intelligent Systems,2024,19(1):153-164.[doi:10.11992/tis.202306037]
点击复制
PANG Rong,YANG Yan,LENG Xiongjin,et al.Segmentation model of pavement diseases based on semantic priori of double-branched point flow[J].CAAI Transactions on Intelligent Systems,2024,19(1):153-164.[doi:10.11992/tis.202306037]
基于双分支点流语义先验的路面病害分割模型
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第1期
页码:
153-164
栏目:
学术论文—智能系统
出版日期:
2024-01-05
- Title:
- Segmentation model of pavement diseases based on semantic priori of double-branched point flow
- Keywords:
- semantic priori information; efficient attention mechanism; cross-covariance image transformers attention mechanism; sparse subject sampling point flow; category imbalance; semantic segmentation; pavement diseases; deep learning
- 分类号:
- TP391
- 文献标志码:
- 2024-01-03
- 摘要:
- 针对基于深度学习的真实路面病害图像识别算法主要面临的复杂道路背景与病害前景比例不同、病害尺度小等导致的类别严重不平衡、路面病害与道路的几何结构特征对比不明显导致其不易识别等问题，本文提出一种基于双分支语义先验网络，用于指导自注意力骨干特征网络挖掘背景与病害前景的复杂关系，运用高效自注意力机制和互协方差自注意力机制分别对二维空间和特征通道进行语义特征提取，并引入语义局部增强模块提高局部特征聚合能力。本文提出了一种新的稀疏主体点流模块，并与传统特征金字塔网络相结合，进一步缓解路面病害的类别不平衡问题；构建了一个真实场景的道路病害分割数据集，并在该数据集和公开数据集上与多个基线模型进行对比实验，实验结果验证了本模型的有效性。
- Abstract:
- At present, the main problems faced by real road disease image recognition algorithms based on deep learning include serious imbalance in categories caused by different proportions of complex road background and foreground of diseases, and small disease scales. What’s more, the inconspicuous contrast between pavement diseases and the geometric structure characteristics of roads leads to their difficulty in recognition. To address the above issues, we propose a semantic prior two-branch network to guide Transformer’s backbone feature network in mining the complex relationship between background and foreground of pavement disease. It uses high-efficiency self-attention mechanism and cross-covariance image transformers(XCiT) to extract semantic features from two-dimensional space and feature channels, respectively, and a semantic locally-enhanced feed-forward (SLeff) module to improve the ability of local feature aggregation. We also propose a new sparse subject sampling point stream module, which is combined with the traditional FPN structure to further alleviate the category imbalance problem of pavement diseases. Finally, we constructed the road disease segmentation dataset based on real scene and compared it with multiple baseline models on this dataset and public dataset. The experimental results demonstrated effectiveness of this model.
- 参考文献/References:
-
[1] “十四五”现代综合交通运输体系发展规划[J]. 铁道技术监督, 2022, 50(2): 9?23, 27.
Development plan of modern comprehensive transportation system in the 14th five-year plan[J]. Railway quality control, 2022, 50(2): 9?23, 27.
[2] The Central People’s Government of the People’s Republic of China, v1.0[EB/OL]. (2023?01?11)[2023?06?15]. http://www.gov.cn/.
[3] 侯越, 张慧婷, 高智伟, 等. 基于数据深度增强的路面病害智能检测方法研究及比较[J]. 北京工业大学学报, 2022, 48(6): 622–634
HOU Yue, ZHANG Huiting, GAO Zhiwei, et al. Research and comparison of intelligent detection methods of pavement distress based on deep data augmentation[J]. Journal of Beijing University of Technology, 2022, 48(6): 622–634
[4] 章世祥, 张汉成, 李西芝, 等. 基于机器视觉的路面裂缝病害多目标识别研究[J]. 公路交通科技, 2021, 38(3): 30–39
ZHANG Shixiang, ZHANG Hancheng, LI Xizhi, et al. Study on multi-objective identification of pavement cracks based on machine vision[J]. Journal of highway and transportation research and development, 2021, 38(3): 30–39
[5] 高建贞, 任明武, 唐振民, 等. 路面裂缝的自动检测与识别[J]. 计算机工程, 2003, 29(2): 149–150
GAO Jianzhen, REN Mingwu, TANG Zhenmin, et al. Automatic road crack detection and identification[J]. Computer engineering, 2003, 29(2): 149–150
[6] 徐威, 唐振民, 吕建勇. 基于图像显著性的路面裂缝检测[J]. 中国图象图形学报, 2013, 18(1): 69–77
XU Wei, TANG Zhenmin, LYU Jianyong. Pavement crack detection based on image saliency[J]. Journal of image and graphics, 2013, 18(1): 69–77
[7] 伯绍波, 闫茂德, 孙国军, 等. 沥青路面裂缝检测图像处理算法研究[J]. 微计算机信息, 2007, 23(15): 280–282
BO Shaobo, YAN Maode, SUN Guojun, et al. Research on crack detection image processing algorithm for asphalt pavement surface[J]. Control & automation, 2007, 23(15): 280–282
[8] SHENG Peng, CHEN Li, TIAN Jing. Learning-based road crack detection using gradient boost decision tree[C]//2018 13th IEEE Conference on Industrial Electronics and Applications. Piscataway: IEEE, 2018: 1228?1232.
[9] SUN Lu, KAMALIARDAKANI M, ZHANG Yongming. Weighted neighborhood pixels segmentation method for automated detection of cracks on pavement surface images[J]. Journal of computing in civil engineering, 2016, 30(2): 04015.
[10] 曹建农, 张昆, 元晨, 等. 用Mean Shift实现路面裂缝损伤自动识别与特征测量[J]. 计算机辅助设计与图形学学报, 2014, 26(9): 1450–1459
CAO Jiannong, ZHANG Kun, YUAN Chen, et al. Automatic road cracks detection and characterization based on mean shift[J]. Journal of computer-aided design & computer graphics, 2014, 26(9): 1450–1459
[11] WANG Yanyan, SONG Kechen, LIU Jie, et al. RENet: rectangular convolution pyramid and edge enhancement network for salient object detection of pavement cracks[J]. Measurement, 2021, 170: 108698.
[12] YANG Jing, FU Qin, NIE Mingxin. Road crack detection using deep neural network with receptive field block[J]. IOP conference series:materials science and engineering, 2020, 782(4): 042033.
[13] YAO Hui, LIU Yanhao, LI Xin, et al. A detection method for pavement cracks combining object detection and attention mechanism[J]. IEEE transactions on intelligent transportation systems, 2022, 23(11): 22179–22189.
[14] ZHANG Yujia, LI Qianzhong, ZHAO Xiaoguang, et al. TB-net: a three-stream boundary-aware network for fine-grained pavement disease segmentation[C]//2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway : IEEE, 2021: 3654?3663.
[15] WU Junxian, ZHANG Yujia, ZHAO Xiaoguang. Multi-task learning for pavement disease segmentation using wavelet transform[C]//2022 International Joint Conference on Neural Networks. Piscataway: IEEE, 2022: 1?8.
[16] YU Gui, DONG Juming, WANG Yihang, et al. RUC-net: a residual-unet-based convolutional neural network for pixel-level pavement crack segmentation[J]. Sensors, 2022, 23(1): 53.
[17] ZHANG Lei, YANG Fan, ZHANG Y D, et al. Road crack detection using deep convolutional neural network[C]//2016 IEEE International Conference on Image Processing. Piscataway: IEEE, 2016: 3708?3712.
[18] 韩静园, 王育坚, 谭卫雄, 等. 基于FCN的路面裂缝分割算法[J]. 传感器与微系统, 2022, 41(6): 146–149
HAN Jingyuan, WANG Yujian, TAN Weixiong, et al. Pavement crack segmentation algorithm based on FCN[J]. Transducer and microsystem technologies, 2022, 41(6): 146–149
[19] YANG Fan, ZHANG Lei, YU Sijia, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE transactions on intelligent transportation systems, 2020, 21(4): 1525–1535.
[20] 刘文婷, 卢新明. 基于计算机视觉的Transformer研究进展[J]. 计算机工程与应用, 2022, 58(6): 1–16
LIU Wenting, LU Xinming. Research progress of transformer based on computer vision[J]. Computer engineering and applications, 2022, 58(6): 1–16
[21] XIE Enze, WANG Wenhai, YU Zhiding, et al. SegFormer: simple and efficient design for semantic segmentation with transformers[EB/OL]. (2021?10?28)[2023?06?15]. https://arxiv.org/abs/2105.15203.pdf.
[22] STRUDEL R, GARCIA R, LAPTEV I, et al. Segmenter: transformer for semantic segmentation[C]//2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 7242?7252.
[23] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]// European Conference on Computer Vision. Cham: Springer, 2020: 213?229.
[24] JAIN J, SINGH A, ORLOV N, et al. SeMask: semantically masked transformers for semantic segmentation[C]//2023 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2023: 752?761.
[25] SHEN Zhuoran, ZHANG Mingyuan, ZHAO Haiyu, et al. Efficient attention: attention with linear complexities[C]//2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2021: 3530?3538.
[26] EL-NOUBY A, TOUVRON H, CARON M, et al. XCiT: cross-covariance image transformers[EB/OL]. (2021?06?18)[2023?05?15]. http://arxiv.org/abs/2106.09681.pdf
[27] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2021?07?03)[2023?06?15]. http://arxiv.org/abs/2010.11929.pdf.
[28] YUAN Kun, GUO Shaopeng, LIU Ziwei, et al. Incorporating convolution designs into visual transformers[C]//2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 559?568.
[29] LI Xiangtai, HE Hao, LI Xia, et al. PointFlow: flowing semantics through points for aerial image segmentation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 4215?4224.
[30] HUANG Zilong, WANG Xinggang, HUANG Lichao, et al. CCNet: criss-cross attention for semantic segmentation[C]//2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2020: 603?612.
[31] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. (2017?12?05)[2023?06?15]. http://arxiv.org/abs/1706.05587.pdf
[32] CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision. Cham: Springer, 2018: 833?851.
[33] SUN Ke, XIAO Bin, LIU Dong, et al. Deep high-resolution representation learning for human pose estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5686?5696.
[34] WU Tianyi, TANG Sheng, ZHANG Rui, et al. CGNet: a light-weight context guided network for semantic segmentation[J]. IEEE transactions on image processing, 2021, 30: 1169–1179.
[35] ZHENG Sixiao, LU Jiachen, ZHAO Hengshuang, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 6877?6886.
备注/Memo
收稿日期:2023-06-15。
基金项目:国家自然科学基金项目（61976247）；国家重大研发计划项目（2019YFB-1310400）；重庆市技术创新与应用发展专项重点项目(CSTB2022TIAD-KPX0100)；重庆市交通科技自筹项目（CQJT20-22ZC05）
作者简介:庞荣，博士研究生，主要研究方法为人工智能、深度学习、大数据分析与挖掘和高速公路智能检测。E-mail：519231410@qq.com;杨燕，教授，博士生导师，西南交通大学计算机与人工智能学院副院长、中国计算机学会杰出会员，主要研究方向为人工智能、大数据分析与挖掘、多视图学习、云计算和云服务。主持国家自然科学基金等项目10余项。发表学术论文230余篇。E-mail： yyang@swjtu.edu.cn;冷雄进，硕士研究生，主要研究方向为人工智能和计算机视觉。E-mail：2932985761@qq.com
通讯作者:杨燕. E-mail：yyang@swjtu.edu.cn
更新日期/Last Update:
1900-01-01