[1]HE Guohao,ZHAI Yong,GONG Jianwei,et al.Real-time stereo matching network for vehicle binocular vision based on dynamic cascade correction[J].CAAI Transactions on Intelligent Systems,2022,17(6):1145-1153.[doi:10.11992/tis.202111013]
Copy

Real-time stereo matching network for vehicle binocular vision based on dynamic cascade correction

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 17 Number of periods: 2022 6 Page number: 1145-1153 Column: 学术论文—机器感知与模式识别 Public date: 2022-11-05
Title:
Real-time stereo matching network for vehicle binocular vision based on dynamic cascade correction
Author(s):
HE Guohao1;  ZHAI Yong1;  GONG Jianwei1; 2;  WANG Yuchun1;  ZHANG Xi2
1. School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China;
2. Chongqing Innovation Center, Beijing Institute of Technology, Chongqing 401120, China
Keywords:
binocular vision; deep learning; stereo matching; disparity estimation; dynamic computation; feature fusion; on-board vision
CLC:
TP29
DOI:
10.11992/tis.202111013
Abstract:
Given the shortcoming of high-precision stereo matching networks based on binocular vision, such as high computing resource consumption, long operating time, and inability to be used in real-time navigation by intelligent driving systems, this study proposes a dynamic fusion stereo matching deep learning network that can meet real-time and accuracy requirements in vehicles. The network includes a global deep convolution-based attention module to complete feature extraction while reducing the number of network layers and parameters and optimizing 3D convolution calculations through dynamic cost cascade fusion, multi-scale fusion, and dynamic disparity change to accelerate the commonly used 3D feature fusion process. The trained model is tested on KITTI Stereo 2015 dataset using onboard hardware such as the NVIDIA Jetson TX2. Experiments show that the method can achieve the same operating accuracy as the state-of-the-art method currently in the leaderboard, 3 pixels error is less than 6.58%, and the operating duration is less than 0.1 seconds per frame, meeting real-time performance requirements.
References:
[1] MARTULL S, PERIS M, FUKUI K. Realistic CG stereo image dataset with ground truth disparity maps[J]. Technical report of ieice prmu, 2012(430): 117–118.
[2] HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE transactions on pattern analysis and machine intelligence, 2008, 30(2): 328–341.
[3] ZHANG Feihu, PRISACARIU V, YANG Ruigang, et al. GA-net: guided aggregation net for end-to-end stereo matching[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019 : 185-194.
[4] 张娣. 基于双目视觉的道路场景语义分割技术研究[D]. 南京: 南京理工大学, 2020.
ZHANG Di. Research on semantic segmentation of road scene based on binocular vision[D]. Nanjing: Nanjing University of Science and Technology, 2020.
[5] ?BONTAR J, LECUN Y. Stereo matching by training a convolutional neural network to compare image patches[J]. Journal of machine learning research, 2016, 17: 1–32.
[6] SEKI A, POLLEFEYS M. SGM-nets: semi-global matching with neural networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017 : 6640-6649.
[7] 王笛. 基于双目立体匹配的三维视觉方法研究[D]. 西安: 西安理工大学, 2021.
WANG Di. Research on 3D vision method based on binocular stereo matching[D]. Xi’an: Xi’an University of Technology, 2021.
[8] 吴玉晗. 基于双目立体视觉的立体匹配算法研究[D]. 成都: 电子科技大学, 2021.
WU Yuhan. Research on stereo matching algorithm based on binocular stereo vision[D]. Chengdu: University of Electronic Science and Technology of China, 2021.
[9] KENDALL A, MARTIROSYAN H, DASGUPTA S, et al. End-to-end learning of geometry and context for deep stereo regression[C]// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 66-75.
[10] CHANG Jiaren, CHEN Yongsheng. Pyramid stereo matching network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 5410-5418.
[11] XU Haofei, ZHANG Juyong. Aanet: adaptive aggregation network for efficient stereo matching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1959-1968.
[12] CHANG Jiaren, CHANG Peichun, CHEN Yongsheng. Attention-aware feature aggregation for real-time stereo matching on edge devices[C]//Asian Conference on Computer Vision. Cham: Springer, 2021: 365-380.
[13] MENZE M, GEIGER A. Object scene flow for autonomous vehicles[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3061-3070.
[14] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84–90.
[15] SANDLER M, HOWARD A, ZHU M, et al. Mobilenetv2: inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4510-4520.
[16] HAASE D, AMTHOR M. Rethinking depthwise separable convolutions: how intra-kernel correlations lead to improved MobileNets[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020 : 14588-14597.
[17] 郑秋梅, 温阳, 王风华. 基于注意力机制和可分离卷积的双目立体匹配算法[J]. 微电子学与计算机, 2021, 38(5): 42–47
ZHENG Qiumei, WEN Yang, WANG Fenghua. Stereo matching based on attention mechanism and separable convolution[J]. Microelectronics & computer, 2021, 38(5): 42–47
[18] CHEN Sheng, LIU Yang, GAO Xiang, et al. MobileFaceNets: efficient CNNs for accurate real-time face verification on mobile devices[C]//Chinese Conference on Biometric Recognition. Cham: Springer, 2018: 428-438.
[19] 吴俊劼, 陈震, 张聪炫, 等. 基于特征级联卷积网络的双目立体匹配[J]. 电子学报, 2021, 49(4): 690–695
WU Junjie, CHEN Zhen, ZHANG Congxuan, et al. Binocular stereo matching based on feature cascade convolutional network[J]. Acta electronica sinica, 2021, 49(4): 690–695
[20] MAYER N, ILG E, HAUSSER P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 4040-4048.
[21] 龚伟, 秦岭, 任高峰, 等. 基于多维特征融合的双目立体匹配算法研究[J]. 激光与光电子学进展, 2020, 57(16): 299–306
GONG Wei, QIN Ling, REN Gaofeng, et al. Binocular stereo matching algorithm based on multidimensional feature fusion[J]. Laser & optoelectronics progress, 2020, 57(16): 299–306
[22] KINGMA D, BA J. Adam: a method for stochastic optimization[EB/OL]. (2017-01-30)[2021-11-06]. https://arxiv. org/abs/1412.6980.
[23] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137–1149.
[24] NATEKIN A, KNOLL A. Gradient boosting machines, a tutorial[J]. Frontiers in neurorobotics, 2013, 7: 21.
[25] DUGGAL S, WANG Shenlong, MA W C, et al. DeepPruner: learning efficient stereo matching via differentiable PatchMatch[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019 : 4383-4392.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems