[1]陆军,王文豪,杜宏劲.基于特征融合和网络采样的点云配准[J].智能系统学报,2025,20(3):621-630.[doi:10.11992/tis.202403022]
LU Jun,WANG Wenhao,DU Hongjin.Point cloud registration based on feature fusion and network sampling[J].CAAI Transactions on Intelligent Systems,2025,20(3):621-630.[doi:10.11992/tis.202403022]
点击复制
LU Jun,WANG Wenhao,DU Hongjin.Point cloud registration based on feature fusion and network sampling[J].CAAI Transactions on Intelligent Systems,2025,20(3):621-630.[doi:10.11992/tis.202403022]
基于特征融合和网络采样的点云配准
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
20
期数:
2025年第3期
页码:
621-630
栏目:
学术论文—机器感知与模式识别
出版日期:
2025-05-05
- Title:
- Point cloud registration based on feature fusion and network sampling
- Keywords:
- point cloud registration; feature fusion; deep learning; network sampling; 3D vision; local feature; global feature; feature extraction
- 分类号:
- TP391.4
- 摘要:
- 针对点云配准过程中，下采样时容易丢失关键点、影响配准精度的问题，本文提出一种基于特征融合和网络采样的配准方法，提高了配准的精度和速度。在PointNet分类网络基础上，引入小型注意力机制，设计一种基于深度学习网络的关键点提取方法，将局部特征和全局特征融合，得到混合特征的特征矩阵。通过深度学习实现对应矩阵求解中相关参数的自动优化，最后利用加权奇异值分解(singular value decomposition，SVD)得到变换矩阵，完成配准。在ModelNet40数据集上的实验表明，和最远点采样相比，所提算法耗时减少45.36%；而配准结果和基于特征学习的鲁棒点匹配(robust point matching using learned features，RPM-Net)相比，平移矩阵均方误差降低5.67%，旋转矩阵均方误差降低13.1%。在自制点云数据上的实验，证实了算法在真实物体上配准的有效性。
- Abstract:
- To solve the issue of easily losing key points during lower sampling, which affects the registration accuracy during point cloud registration, a registration method is proposed based on network sampling and feature fusion, and this method improves registration accuracy and speed. Based on the PointNet classification network, we design a deep learning (DL) network-based method for key point extraction. The method fuses local features with global features to obtain the feature matrix with fixed characteristics and uses DL to automatically optimize the parameters when calculating the corresponding matrix. Finally, we use weighted singular value decomposition to obtain the transformation matrix and complete the registration. Our experiments using the ModelNet40 dataset reveal that the time consumed for the process is reduced by 45.36% compared with that consumed by farthest point sampling. Compared with the RPM-Net algorithm, the mean square errors of the translation and rotation matrices obtained by the proposed method are reduced by 5.67% and 13.1%, respectively. Further, the designed model was subjected to experiments, which proved the effectiveness of the algorithm in registering real objects.
- 参考文献/References:
-
[1] 张新钰, 邹镇洪, 李志伟, 等. 面向自动驾驶目标检测的深度多模态融合技术[J]. 智能系统学报, 2020, 15(4): 758-771.
ZHANG Xinyu, ZOU Zhenhong, LI Zhiwei, et al. Deep multi-modal fusion in object detection for autonomous driving[J]. CAAI transactions on intelligent systems, 2020, 15(4): 758-771.
[2] CHEN Kai, HONG Lanqing, XU Hang, et al. MultiSiam: self-supervised multi-instance Siamese representation learning for autonomous driving[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 7526-7534.
[3] MA Gang, WEI Hui. A novel sketch-based framework utilizing contour cues for efficient point cloud registration[J]. IEEE transactions on geoscience and remote sensing, 2023, 61: 5703616.
[4] PEREZ-GONZALEZ J, LUNA-MADRIGAL F, PI?A-RAMIREZ O. Deep learning point cloud registration based on distance features[J]. IEEE Latin America transactions, 2019, 17(12): 2053-2060.
[5] SEGAL A, HAEHNEL D, THRUN S. Generalized-ICP[C]//Robotics: Science and Systems. Seattle: MIT Press, 2009.
[6] YANG Jiaolong, LI Hongdong, JIA Yunde. Go-ICP: solving 3D registration efficiently and globally optimally[C]//2013 IEEE International Conference on Computer Vision. Sydney: IEEE, 2013: 1457-1464.
[7] RUSU R B, BLODOW N, BEETZ M. Fast point feature histograms (FPFH) for 3D registration[C]//2009 IEEE International Conference on Robotics and Automation. Kobe: IEEE, 2009: 3212-3217.
[8] SANG Mengting, WANG Wei, PAN Yani. RGB-ICP method to calculate ground three-dimensional deformation based on point cloud from airborne LiDAR[J]. Remote sensing, 2022, 14(19): 4851.
[9] SHI Weijing, RAJKUMAR R. Point-GNN: graph neural network for 3D object detection in a point cloud[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1708-1716.
[10] XIE Yifan, ZHU Jihua, LI Shiqi, et al. Cross-modal information-guided network using contrastive learning for point cloud registration[J]. IEEE robotics and automation letters, 2024, 9(1): 103-110.
[11] 陆军, 邵红旭, 王伟, 等. 基于关键点特征匹配的点云配准方法[J]. 北京理工大学学报, 2020, 40(4): 409-415.
LU Jun, SHAO Hongxu, WANG Wei, et al. Point cloud registration method based on key point extraction with small overlap[J]. Transactions of Beijing Institute of Technology, 2020, 40(4): 409-415.
[12] ISHIHARA K, KANERVISTO A, MIURA J, et al. Multi-task learning with attention for end-to-end autonomous driving[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE, 2021: 2902-2911.
[13] WANG Li, DU Liang, YE Xiaoqing, et al. Depth-conditioned dynamic message propagation for monocular 3D object detection[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 454-463.
[14] YAN Zihao, YI Zimu, HU Ruizhen, et al. Consistent two-flow network for tele-registration of point clouds[J]. IEEE transactions on visualization and computer graphics, 2022, 28(12): 4304-4318.
[15] DENG Haowen, BIRDAL T, ILIC S. PPFNet: global context aware local features for robust 3D point matching[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 195-205.
[16] CHOY C, GWAK J, SAVARESE S. 4D spatio-temporal ConvNets: minkowski convolutional neural networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3070-3079.
[17] BAI Xuyang, LUO Zixin, ZHOU Lei, et al. D3Feat: joint learning of dense detection and description of 3D local features[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6358-6366.
[18] HUANG Shengyu, GOJCIC Z, USVYATSOV M, et al. PREDATOR: registration of 3D point clouds with low overlap[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 4267-4276.
[19] LU Weixin, WAN Guowei, ZHOU Yao, et al. DeepVCP: an end-to-end deep neural network for point cloud registration[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 12-21.
[20] YEW Z J, LEE G H. RPM-net: robust point matching using learned features[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11821-11830.
[21] SARODE V, LI Xueqian, GOFORTH H, et al. PCRNet: point cloud registration network using PointNet encoding[EB/OL]. (2019-11-04)[2024-01-01]. https://arxiv.org/abs/1908.07906v2.
[22] GE Xuming, HU Han, WU Bo. Image-guided registration of unordered terrestrial laser scanning point clouds for urban scenes[J]. IEEE transactions on geoscience and remote sensing, 2019, 57(11): 9264-9276.
[23] YUAN Hanjie, HE Yang, DONG Limeng, et al. Multi-spectral registration fusion based on laser point cloud[C]//2022 IEEE International Conference on Electrical Engineering, Big Data and Algorithms. Changchun: IEEE, 2022: 50-53.
[24] WANG Huaqiang, HUANG Lu, YU Kang, et al. Deep-learning-based multiview RGBD sensor system for 3-D face point cloud registration[J]. IEEE sensors letters, 2023, 7(5): 7001804.
[25] HE Chenhang, ZENG Hui, HUANG Jianqiang, et al. Structure aware single-stage 3D object detection from point cloud[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11870-11879.
[26] WU Yue, YAO Qianlin, FAN Xiaolong, et al. PANet: a point-attention based multi-scale feature fusion network for point cloud registration[J]. IEEE transactions on instrumentation and measurement, 2023, 72: 2512913.
[27] KADAM P, ZHANG Min, LIU Shan, et al. R-PointHop: a green, accurate, and unsupervised point cloud registration method[J]. IEEE transactions on image processing, 2022, 31: 2710-2725.
[28] WANG Yue, SOLOMON J. Deep closest point: learning representations for point cloud registration[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3522-3531.
[29] YANG Zetong, SUN Yanan, LIU Shu, et al. 3DSSD: point-based 3D single stage object detector[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11037-11045.
[30] QI C R, YI Li, SU Hao, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[EB/OL]. (2017-06-07)[2024-01-01]. https://arxiv.org/abs/1706.02413v1.
[31] WANG Yongqiang, ZHANG Di, NI Lihua, et al. A novel keypoint detection method for radar point cloud registration[J]. IEEE sensors letters, 2023, 7(12): 3503204.
- 相似文献/References:
-
[1]孙倩茹,王文敏,刘宏.视频序列的人体运动描述方法综述[J].智能系统学报,2013,8(3):189.
SUN Qianru,WANG Wenmin,LIU Hong.Study of human action representation in video sequences[J].CAAI Transactions on Intelligent Systems,2013,8():189.
[2]吴茜茵,严云洋,杜静,等.多特征融合的火焰检测算法[J].智能系统学报,2015,10(2):240.[doi:10.3969/j.issn.1673-4785.201406022]
WU Xiyin,YAN Yunyang,DU Jing,et al.Fire detection based on fusion of multiple features[J].CAAI Transactions on Intelligent Systems,2015,10():240.[doi:10.3969/j.issn.1673-4785.201406022]
[3]张毅,陈起,罗元.室内环境下移动机器人三维视觉SLAM[J].智能系统学报,2015,10(4):615.[doi:10.3969/j.issn.1673-4785.201504003]
ZHANG Yi,CHEN Qi,LUO Yuan.Three dimensional visual SLAM for mobile robots in indoor environments[J].CAAI Transactions on Intelligent Systems,2015,10():615.[doi:10.3969/j.issn.1673-4785.201504003]
[4]孙劲光,孟凡宇.一种特征加权融合人脸识别方法[J].智能系统学报,2015,10(6):912.[doi:10.11992/tis.201509025]
SUN Jinguang,MENG Fanyu.Face recognition by weighted fusion of facial features[J].CAAI Transactions on Intelligent Systems,2015,10():912.[doi:10.11992/tis.201509025]
[5]柳培忠,阮晓虎,田震,等.一种基于多特征融合的视频目标跟踪方法[J].智能系统学报,2014,9(3):319.[doi:10.3969/j.issn.1673-4785.201309085]
LIU Peizhong,RUAN Xiaohu,TIAN Zhen,et al.A video tracking method based on object multi-feature fusion[J].CAAI Transactions on Intelligent Systems,2014,9():319.[doi:10.3969/j.issn.1673-4785.201309085]
[6]路子祥,屠黎阳,祖辰,等.基于脑连接网络的阿尔茨海默病临床变量值预测[J].智能系统学报,2017,12(3):355.[doi:10.11992/tis.201607020]
LU Zixiang,TU Liyang,ZU Chen,et al.Prediction of clinical variables in Alzheimer’s disease using brain connective networks[J].CAAI Transactions on Intelligent Systems,2017,12():355.[doi:10.11992/tis.201607020]
[7]吴钟强,张耀文,商琳.基于语义特征的多视图情感分类方法[J].智能系统学报,2017,12(5):745.[doi:10.11992/tis.201706026]
WU Zhongqiang,ZHANG Yaowen,SHANG Lin.Multi-view sentiment classification of microblogs based on semantic features[J].CAAI Transactions on Intelligent Systems,2017,12():745.[doi:10.11992/tis.201706026]
[8]王成济,罗志明,钟准,等.一种多层特征融合的人脸检测方法[J].智能系统学报,2018,13(1):138.[doi:10.11992/tis.201707018]
WANG Chengji,LUO Zhiming,ZHONG Zhun,et al.Face detection method fusing multi-layer features[J].CAAI Transactions on Intelligent Systems,2018,13():138.[doi:10.11992/tis.201707018]
[9]李亚飞,董红斌.基于卷积神经网络的遥感图像分类研究[J].智能系统学报,2018,13(4):550.[doi:10.11992/tis.201706078]
LI Yafei,DONG Hongbin.Classification of remote-sensing image based on convolutional neural network[J].CAAI Transactions on Intelligent Systems,2018,13():550.[doi:10.11992/tis.201706078]
[10]贾鹤鸣,朱传旭,张森,等.对偶树复小波与空域信息的手势识别分类研究[J].智能系统学报,2018,13(4):619.[doi:10.11992/tis.201708003]
JIA Heming,ZHU Chuanxu,ZHANG Sen,et al.Research on gesture recognition and classification of dual-tree complex wavelet and spatial information[J].CAAI Transactions on Intelligent Systems,2018,13():619.[doi:10.11992/tis.201708003]
[11]郭大伟,李景浩,陆军.三维点云配准的多尺度深度学习方法[J].智能系统学报,2024,19(4):817.[doi:10.11992/tis.202304007]
GUO Dawei,LI Jinghao,LU Jun.Multi-scale deep learning method for 3D point cloud registration[J].CAAI Transactions on Intelligent Systems,2024,19():817.[doi:10.11992/tis.202304007]
备注/Memo
收稿日期:2024-3-12。
基金项目:黑龙江省自然科学基金项目(F201123).
作者简介:陆军，教授，博士生导师，主要研究方向为计算机视觉、图像处理、高性能船舶控制。主持和承担国家及省部级科研项目多项，参与或承担的项目获国防科学技术进步奖一等奖3项、省部级二等奖1项、省部级三等奖3项，发表学术论文80余篇。E-mail： lujun0260@sina.com。;王文豪，硕士研究生，主要研究方向为三维机器视觉、图像处理。E-mail：wenhao-wang@qq.com。;杜宏劲，硕士研究生，主要研究方向为三维点云、图像处理、目标识别。E-mail：dhjmuchen@163.com。
通讯作者:陆军. E-mail：lujun0260@sina.com
更新日期/Last Update:
1900-01-01