[1]周静,胡怡宇,黄心汉.形状补全引导的Transformer点云目标检测方法[J].智能系统学报,2023,18(4):731-742.[doi:10.11992/tis.202210038]
ZHOU Jing,HU Yiyu,HUANG Xinhan.Shape completion-guided Transformer point cloud object detection method[J].CAAI Transactions on Intelligent Systems,2023,18(4):731-742.[doi:10.11992/tis.202210038]
点击复制
ZHOU Jing,HU Yiyu,HUANG Xinhan.Shape completion-guided Transformer point cloud object detection method[J].CAAI Transactions on Intelligent Systems,2023,18(4):731-742.[doi:10.11992/tis.202210038]
形状补全引导的Transformer点云目标检测方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第4期
页码:
731-742
栏目:
学术论文—机器感知与模式识别
出版日期:
2023-07-15
- Title:
- Shape completion-guided Transformer point cloud object detection method
- Keywords:
- 3D object detection; low-quality object; feature separation; shape completion; Transformer; multi-scale; neighboring mask; feature enhancement
- 分类号:
- TP391.41
- 摘要:
- 针对雷达传感器采集到的场景点云中存在大量远距离或位于遮挡视角的形状缺失的低质量目标，其几何信息不足难以被识别，影响检测精度的问题，本文提出一种基于形状补全引导的Transformer点云目标检测方法(shape completion-guided transformer point cloud object detection method, STDet)，通过增强低质量目标形状特征来有效提升目标检测精度，利用Pointformer主干网络提取场景点云特征以生成初始候选框，基于特征分离预测的形状补全模块重构候选框中残缺目标的完整形状点云；构建Transformer几何特征增强模型，融合目标完整形状信息及空间位置信息至各目标点特征中，并感知各目标点不同邻域掩码范围内的局部结构信息与全局几何特征的注意力相关性，以获取关键几何信息增强的目标全局几何特征；基于该特征引导生成精细化的目标检测框。在KITTI数据集上的实验结果表明，该方法在存在大量形状残缺低质量目标的困难场景中检测精度较基准算法提升了4.96%，大量消融实验证明了该方法所构建的形状补全算法和Transformer几何特征增强模型的有效性。
- Abstract:
- Aiming at the problem that in the point cloud of scenes collected by the LIDAR sensor, there are lots of low-quality objects with missing shapes due to long distance or occlusion, whose geometric information are too insufficient to be recognized, so that the detection accuracy is affected. Hence, a shape completion-guided Transformer point cloud object detection method (STDet) is proposed to improve the object detection precision by enhancing shape features of the low-quality objects. The features of the point clouds are acquired by the Pointformer backbone network to generate the initial candidate box. Then, the shape completion module predicted based on feature separation is designed to reconstruct a complete shape of point clouds of the incomplete objects within the candidate box. A Transformer geometric feature enhancement module is established, which integrates the complete shape information and spatial location knowledge of the object into its point-wise feature to perceive the attention correlation between the local structure information and the global geometric features within different neighborhood masks, so as to acquire the global geometric feature with enhanced critical geometric knowledge of the objects. Finally, the refined object detection boxes are generated under the guidance of global geometric features. Experimental results on KITTI data set show that compared with the benchmark algorithm, the proposed method improves detection accuracy by 4.96% in scenes with abundant low-quality objects of incomplete shapes. Meanwhile, the effectiveness of the proposed shape completion algorithm and Transformer geometric feature encoding module is proved by extensive ablation experiments.
- 参考文献/References:
-
[1] 张新钰, 邹镇洪, 李志伟, 等. 面向自动驾驶目标检测的深度多模态融合技术[J]. 智能系统学报, 2020, 15(4): 758–771
ZHANG Xinyu, ZOU Zhenhong, LI Zhiwei, et al. Deep multi-modal fusion in object detection for autonomous driving[J]. CAAI transactions on intelligent systems, 2020, 15(4): 758–771
[2] LIANG Ming, YANG Bin, CHEN Yun, et al. Multi-task multi-sensor fusion for 3D object detection[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2019: 7337-7345.
[3] ZHOU Yin, SUN Pei, ZHANG Yu, et al. End-to-end multi-view fusion for 3D object detection in LiDAR point clouds[EB/OL]. (2019?10?15)[2022?10?29]. https://arxiv.org/abs/1910.06528.
[4] DENG Jiajun, ZHOU Wengang, ZHANG Yanyong, et al. From multi-view to hollow-3D: hallucinated hollow-3D R-CNN for 3D object detection[J]. IEEE transactions on circuits and systems for video technology, 2021, 31(12): 4722–4734.
[5] ZHOU Yin, TUZEL O. VoxelNet: end-to-end learning for point cloud based 3D object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4490?4499.
[6] YAN Yan, MAO Yuxing, LI Bo. SECOND: sparsely embedded convolutional detection[J]. Sensors, 2018, 18(10): 3337.
[7] LANG A H, VORA S, CAESAR H, et al. PointPillars: fast encoders for object detection from point clouds[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 12689-12697.
[8] YE Maosheng, XU Shuangjie, CAO Tongyi. HVNet: hybrid voxel network for LiDAR based 3D object detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 1628-1637.
[9] DENG Jiajun, SHI Shaoshuai, LI Peiwei, et al. Voxel R-CNN: towards high performance voxel-based 3D object detection[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2021: 1201?1209.
[10] HE Chenhang, ZENG Hui, HUANG Jianqiang, et al. Structure aware single-stage 3D object detection from point cloud[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2020: 11870-11879.
[11] SHI Shaoshuai, WANG Zhe, SHI Jianping, et al. From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(8): 2647–2664.
[12] SHI Shaoshuai, WANG Xiaogang, LI Hongsheng. PointRCNN: 3D object proposal generation and detection from point cloud[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 770-779.
[13] YANG Zetong, SUN Yanan, LIU Shu, et al. 3DSSD: point-based 3D single stage object detector[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2020: 11037-11045.
[14] ZHANG Yifan, HU Qingyong, XU Guoquan, et al. Not all points are equal: learning highly efficient point-based detectors for 3D LiDAR point clouds[EB/OL]. (2022?03?21)[2022?10?29]. https://arxiv.org/abs/2203.11139.
[15] SHI Weijing, RAJKUMAR R. Point-GNN: graph neural network for 3D object detection in a point cloud[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 1708-1716.
[16] LIU Ze, ZHANG Zheng, CAO Yue, et al. Group-free 3D object detection via transformers[C]//2021 IEEE/CVF International Conference on Computer Vision . Piscataway: IEEE, 2021: 2929-2938.
[17] MISRA I, GIRDHAR R, JOULIN A. An end-to-end transformer model for 3D object detection[C]//2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 2886-2897.
[18] PAN Xuran, XIA Zhuofan, SONG Shiji, et al. 3D object detection with pointformer[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 7459-7468.
[19] 韩磊, 高永彬, 史志才. 基于稀疏Transformer的雷达点云三维目标检测[J]. 计算机工程, 2022, 48(11): 104?110, 144.
HAN Lei, GAO Yongbin, SHI Zhicai. Three-dimensional object detection of LiDAR point cloud based on sparse transformer[J]. Computer engineering, 2022, 48(11): 104-110, 144.
[20] ZHOU Dingfu, FANG Jin, SONG Xibin, et al. IoU loss for 2D/3D object detection[C]//2019 International Conference on 3D Vision . Piscataway: IEEE, 2019: 85-94.
[21] 赵毅强, 艾西丁·艾克白尔, 陈瑞, 等. 基于体素化图卷积网络的三维点云目标检测方法[J]. 红外与激光工程, 2021, 50(10): 281–289
ZHAO Yiqiang, ARXIDIN A, CHEN Rui, et al. 3D point cloud object detection method in view of voxel based on graph convolution network[J]. Infrared and laser engineering, 2021, 50(10): 281–289
[22] SHI Shaoshuai, GUO Chaoxu, JIANG Li, et al. PV-RCNN: point-voxel feature set abstraction for 3D object detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 10526-10535.
[23] LI Jiale, DAI Hang, SHAO Ling, et al. Anchor-free 3D single stage detector with mask-guided attention for point cloud[C]//Proceedings of the 29th ACM International Conference on Multimedia. New York: ACM, 2021: 553?562.
[24] NOH J, LEE S, HAM B. HVPR: hybrid voxel-point representation for single-stage 3D object detection[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 14600-14609.
[25] ZARZAR J, GIANCOLA S, GHANEM B. PointRGCN: graph convolution networks for 3D vehicles detection refinement[EB/OL]. (2019?11?27)[2022?10?29]. https://arxiv.org/abs/1911.12236.
备注/Memo
收稿日期:2022-10-29。
基金项目:国家自然科学基金项目（62106086）；湖北省自然科学基金项目（2021CFB564）.
作者简介:周静，教授，主要研究方向为深度学习与智能算法、智能机器视觉、点云目标检测、图像处理与模式识别。主持国家自然科学基金项目、湖北省自然科学基金项目和横向科研项目20余项，发明专利10余项，武汉市优秀青年教师。发表学术论文30余篇。;胡怡宇，硕士研究生，主要研究方向为智能机器视觉、深度学习与智能算法、三维目标检测。;黄心汉，教授，博士生导师，中国人工智能学会会士，智能机器人专业委员会名誉主任，享受国务院政府特殊津贴，湖北省有突出贡献的中青年专家。主要研究方向为智能控制、智能机器人、信息融合、图像处理与模式识别。主持国家自然科学基金项目、国家 863计划、国家科技支撑计划及省部级和横向科研项目 60 余项，授权发明专利 11 项。发表学术论文 300 余篇，出版专著 4 部、译著 2 本。
通讯作者:周静.E-mail:zhj131@jhun.edu.cn
更新日期/Last Update:
1900-01-01