[1]PANG Rong,YANG Yan,LENG Xiongjin,et al.Segmentation model of pavement diseases based on semantic priori of double-branched point flow[J].CAAI Transactions on Intelligent Systems,2024,19(1):153-164.[doi:10.11992/tis.202306037]
Copy

Segmentation model of pavement diseases based on semantic priori of double-branched point flow

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 1 Page number: 153-164 Column: 学术论文—智能系统 Public date: 2024-01-05
Title:
Segmentation model of pavement diseases based on semantic priori of double-branched point flow
Author(s):
PANG Rong1; 2; 3; 4;  YANG Yan1; 2;  LENG Xiongjin1; 2;  ZHANG Peng3; 4;  LIU Yan1; 2
1. School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China;
2. Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, Chengdu 611756, China;
3. China Merchants Chongqing Road Engineering Inspection Center Co., Ltd, Chongqing 400067, China;
4. Mountain Highway Engineering Technology Research Center, Chongqing 400067, China
Keywords:
semantic priori information; efficient attention mechanism; cross-covariance image transformers attention mechanism; sparse subject sampling point flow; category imbalance; semantic segmentation; pavement diseases; deep learning
CLC:
TP391
DOI:
10.11992/tis.202306037
Abstract:
At present, the main problems faced by real road disease image recognition algorithms based on deep learning include serious imbalance in categories caused by different proportions of complex road background and foreground of diseases, and small disease scales. What’s more, the inconspicuous contrast between pavement diseases and the geometric structure characteristics of roads leads to their difficulty in recognition. To address the above issues, we propose a semantic prior two-branch network to guide Transformer’s backbone feature network in mining the complex relationship between background and foreground of pavement disease. It uses high-efficiency self-attention mechanism and cross-covariance image transformers(XCiT) to extract semantic features from two-dimensional space and feature channels, respectively, and a semantic locally-enhanced feed-forward (SLeff) module to improve the ability of local feature aggregation. We also propose a new sparse subject sampling point stream module, which is combined with the traditional FPN structure to further alleviate the category imbalance problem of pavement diseases. Finally, we constructed the road disease segmentation dataset based on real scene and compared it with multiple baseline models on this dataset and public dataset. The experimental results demonstrated effectiveness of this model.
References:
[1] “十四五”现代综合交通运输体系发展规划[J]. 铁道技术监督, 2022, 50(2): 9?23, 27.
Development plan of modern comprehensive transportation system in the 14th five-year plan[J]. Railway quality control, 2022, 50(2): 9?23, 27.
[2] The Central People’s Government of the People’s Republic of China, v1.0[EB/OL]. (2023?01?11)[2023?06?15]. http://www.gov.cn/.
[3] 侯越, 张慧婷, 高智伟, 等. 基于数据深度增强的路面病害智能检测方法研究及比较[J]. 北京工业大学学报, 2022, 48(6): 622–634
HOU Yue, ZHANG Huiting, GAO Zhiwei, et al. Research and comparison of intelligent detection methods of pavement distress based on deep data augmentation[J]. Journal of Beijing University of Technology, 2022, 48(6): 622–634
[4] 章世祥, 张汉成, 李西芝, 等. 基于机器视觉的路面裂缝病害多目标识别研究[J]. 公路交通科技, 2021, 38(3): 30–39
ZHANG Shixiang, ZHANG Hancheng, LI Xizhi, et al. Study on multi-objective identification of pavement cracks based on machine vision[J]. Journal of highway and transportation research and development, 2021, 38(3): 30–39
[5] 高建贞, 任明武, 唐振民, 等. 路面裂缝的自动检测与识别[J]. 计算机工程, 2003, 29(2): 149–150
GAO Jianzhen, REN Mingwu, TANG Zhenmin, et al. Automatic road crack detection and identification[J]. Computer engineering, 2003, 29(2): 149–150
[6] 徐威, 唐振民, 吕建勇. 基于图像显著性的路面裂缝检测[J]. 中国图象图形学报, 2013, 18(1): 69–77
XU Wei, TANG Zhenmin, LYU Jianyong. Pavement crack detection based on image saliency[J]. Journal of image and graphics, 2013, 18(1): 69–77
[7] 伯绍波, 闫茂德, 孙国军, 等. 沥青路面裂缝检测图像处理算法研究[J]. 微计算机信息, 2007, 23(15): 280–282
BO Shaobo, YAN Maode, SUN Guojun, et al. Research on crack detection image processing algorithm for asphalt pavement surface[J]. Control & automation, 2007, 23(15): 280–282
[8] SHENG Peng, CHEN Li, TIAN Jing. Learning-based road crack detection using gradient boost decision tree[C]//2018 13th IEEE Conference on Industrial Electronics and Applications. Piscataway: IEEE, 2018: 1228?1232.
[9] SUN Lu, KAMALIARDAKANI M, ZHANG Yongming. Weighted neighborhood pixels segmentation method for automated detection of cracks on pavement surface images[J]. Journal of computing in civil engineering, 2016, 30(2): 04015.
[10] 曹建农, 张昆, 元晨, 等. 用Mean Shift实现路面裂缝损伤自动识别与特征测量[J]. 计算机辅助设计与图形学学报, 2014, 26(9): 1450–1459
CAO Jiannong, ZHANG Kun, YUAN Chen, et al. Automatic road cracks detection and characterization based on mean shift[J]. Journal of computer-aided design & computer graphics, 2014, 26(9): 1450–1459
[11] WANG Yanyan, SONG Kechen, LIU Jie, et al. RENet: rectangular convolution pyramid and edge enhancement network for salient object detection of pavement cracks[J]. Measurement, 2021, 170: 108698.
[12] YANG Jing, FU Qin, NIE Mingxin. Road crack detection using deep neural network with receptive field block[J]. IOP conference series:materials science and engineering, 2020, 782(4): 042033.
[13] YAO Hui, LIU Yanhao, LI Xin, et al. A detection method for pavement cracks combining object detection and attention mechanism[J]. IEEE transactions on intelligent transportation systems, 2022, 23(11): 22179–22189.
[14] ZHANG Yujia, LI Qianzhong, ZHAO Xiaoguang, et al. TB-net: a three-stream boundary-aware network for fine-grained pavement disease segmentation[C]//2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway : IEEE, 2021: 3654?3663.
[15] WU Junxian, ZHANG Yujia, ZHAO Xiaoguang. Multi-task learning for pavement disease segmentation using wavelet transform[C]//2022 International Joint Conference on Neural Networks. Piscataway: IEEE, 2022: 1?8.
[16] YU Gui, DONG Juming, WANG Yihang, et al. RUC-net: a residual-unet-based convolutional neural network for pixel-level pavement crack segmentation[J]. Sensors, 2022, 23(1): 53.
[17] ZHANG Lei, YANG Fan, ZHANG Y D, et al. Road crack detection using deep convolutional neural network[C]//2016 IEEE International Conference on Image Processing. Piscataway: IEEE, 2016: 3708?3712.
[18] 韩静园, 王育坚, 谭卫雄, 等. 基于FCN的路面裂缝分割算法[J]. 传感器与微系统, 2022, 41(6): 146–149
HAN Jingyuan, WANG Yujian, TAN Weixiong, et al. Pavement crack segmentation algorithm based on FCN[J]. Transducer and microsystem technologies, 2022, 41(6): 146–149
[19] YANG Fan, ZHANG Lei, YU Sijia, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE transactions on intelligent transportation systems, 2020, 21(4): 1525–1535.
[20] 刘文婷, 卢新明. 基于计算机视觉的Transformer研究进展[J]. 计算机工程与应用, 2022, 58(6): 1–16
LIU Wenting, LU Xinming. Research progress of transformer based on computer vision[J]. Computer engineering and applications, 2022, 58(6): 1–16
[21] XIE Enze, WANG Wenhai, YU Zhiding, et al. SegFormer: simple and efficient design for semantic segmentation with transformers[EB/OL]. (2021?10?28)[2023?06?15]. https://arxiv.org/abs/2105.15203.pdf.
[22] STRUDEL R, GARCIA R, LAPTEV I, et al. Segmenter: transformer for semantic segmentation[C]//2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 7242?7252.
[23] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]// European Conference on Computer Vision. Cham: Springer, 2020: 213?229.
[24] JAIN J, SINGH A, ORLOV N, et al. SeMask: semantically masked transformers for semantic segmentation[C]//2023 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2023: 752?761.
[25] SHEN Zhuoran, ZHANG Mingyuan, ZHAO Haiyu, et al. Efficient attention: attention with linear complexities[C]//2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2021: 3530?3538.
[26] EL-NOUBY A, TOUVRON H, CARON M, et al. XCiT: cross-covariance image transformers[EB/OL]. (2021?06?18)[2023?05?15]. http://arxiv.org/abs/2106.09681.pdf
[27] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2021?07?03)[2023?06?15]. http://arxiv.org/abs/2010.11929.pdf.
[28] YUAN Kun, GUO Shaopeng, LIU Ziwei, et al. Incorporating convolution designs into visual transformers[C]//2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 559?568.
[29] LI Xiangtai, HE Hao, LI Xia, et al. PointFlow: flowing semantics through points for aerial image segmentation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 4215?4224.
[30] HUANG Zilong, WANG Xinggang, HUANG Lichao, et al. CCNet: criss-cross attention for semantic segmentation[C]//2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2020: 603?612.
[31] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. (2017?12?05)[2023?06?15]. http://arxiv.org/abs/1706.05587.pdf
[32] CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision. Cham: Springer, 2018: 833?851.
[33] SUN Ke, XIAO Bin, LIU Dong, et al. Deep high-resolution representation learning for human pose estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5686?5696.
[34] WU Tianyi, TANG Sheng, ZHANG Rui, et al. CGNet: a light-weight context guided network for semantic segmentation[J]. IEEE transactions on image processing, 2021, 30: 1169–1179.
[35] ZHENG Sixiao, LU Jiachen, ZHAO Hengshuang, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 6877?6886.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems