[1]WANG Lipeng,WANG Xiaochen,QI Yao,et al.Indoor robot semantic VI-SLAM based on feature fusion and dynamic background removal[J].CAAI Transactions on Intelligent Systems,2024,19(6):1438-1448.[doi:10.11992/tis.202309025]
Copy

Indoor robot semantic VI-SLAM based on feature fusion and dynamic background removal

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 6 Page number: 1438-1448 Column: 学术论文—机器人 Public date: 2024-12-05
Title:
Indoor robot semantic VI-SLAM based on feature fusion and dynamic background removal
Author(s):
WANG Lipeng;  WANG Xiaochen;  QI Yao;  ZHANG Jiapeng
College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China
Keywords:
indoor robot; VI-SLAM; feature dynamic removing; semantic map; feature fusion.; dense point cloud; point cloud segmentation; dynamic scene
CLC:
TP242.6
DOI:
10.11992/tis.202309025
Abstract:
An indoor robot semantic VI-SLAM algorithm based on feature fusion and dynamic background removal is proposed to improve the positioning accuracy of indoor robots in dynamic scenes and build a three-dimensional (3D) semantic map with rich details. The framework of the ORB-SLAM3 algorithm is improved, and a VI-SLAM algorithm for real-time construction of 3D dense point cloud maps is designed. The algorithm fuses target recognition algorithms YOLOv5 and VI-SLAM to obtain two-dimensional (2D) semantic information. Dynamic features are then removed by combining the 2D semantic information with the epipolar constraint principle. Subsequently, the 2D semantic information is mapped into a 3D semantic tag, constructing a 3D semantic map by fusing the semantic features with the point-cloud features. Finally, experiments in 3D semantic map construction were conducted in indoor scenes using public data sets and a mobile robot platform. Results verify the feasibility and effectiveness of the semantic VI-SLAM algorithm in dynamic environments.
References:
[1] SüNDERHAUF N, PHAM T T, LATIF Y, et al. Meaningful maps with object-oriented semantic mapping[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE, 2017: 5079-5085.
[2] WHELAN T, SALAS-MORENO R F, GLOCKER B, et al. ElasticFusion: real-time dense SLAM and light source estimation[J]. The international journal of robotics research, 2016, 35(14): 1697-1716.
[3] MCCORMAC J, HANDA A, DAVISON A, et al. SemanticFusion: dense 3D semantic mapping with convolutional neural networks[C]//2017 IEEE International Conference on Robotics and Automation. New York: IEEE, 2017: 4628-4635.
[4] WANG Yutong, XU Bin, FAN Wei, et al. QISO-SLAM: object-oriented SLAM using dual quadrics as landmarks based on instance segmentation[J]. IEEE robotics and automation letters, 2023, 8(4): 2253-2260.
[5] SALAS-MORENO R F, NEWCOMBE R A, STRASDAT H, et al. SLAM: simultaneous localisation and mapping at the level of objects[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2013: 1352-1359.
[6] DAME A, PRISACARIU V A, REN C Y, et al. Dense reconstruction using 3D object shape priors[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2013: 1288-1295.
[7] YU Chao, LIU Zuxin, LIU Xinjun, et al. DS-SLAM: a semantic visual SLAM towards dynamic environments[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE, 2018: 1168-1174.
[8] KANEKO M, IWAMI K, OGAWA T, et al. Mask-SLAM: robust feature-based monocular SLAM by masking using semantic segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE, 2018: 371-378.
[9] WANG Kai, LIN Yimin, WANG Luowei, et al. A unified framework for mutual improvement of SLAM and semantic segmentation[C]//2019 International Conference on Robotics and Automation. Montreal: IEEE, 2019: 5224-5230.
[10] BESCOS B, FáCIL J M, CIVERA J, et al. DynaSLAM: tracking, mapping, and inpainting in dynamic scenes[J]. IEEE robotics and automation letters, 2018, 3(4): 4076-4083.
[11] BESCOS B, CAMPOS C, TARDóS J D, et al. DynaSLAM II: tightly-coupled multi-object tracking and SLAM[J]. IEEE robotics and automation letters, 2021, 6(3): 5191-5198.
[12] LI Ao, WANG Jikai, XU Meng, et al. DP-SLAM: a visual SLAM with moving probability towards dynamic environments[J]. Information sciences, 2021, 556: 128-142.
[13] ZHAO Yao, XIONG Zhi, ZHOU Shuailin, et al. KSF-SLAM: a key segmentation frame based semantic SLAM in dynamic environments[J]. Journal of intelligent & robotic systems, 2022, 105(1): 3.
[14] HU Zhangfang, ZHAO Jiang, LUO Yuan, et al. Semantic SLAM based on improved DeepLabv3+ in dynamic scenarios[J]. IEEE access, 2022, 10: 21160-21168.
[15] HUANG Shisheng, MA Zeyu, MU Taijiang, et al. Supervoxel convolution for online 3D semantic segmentation[J]. ACM transactions on graphics, 2021, 40(3): 1-15.
[16] MCCORMAC J, CLARK R, BLOESCH M, et al. Fusion: volumetric object-level SLAM[C]//2018 International Conference on 3D Vision. New York: IEEE, 2018: 32-41.
[17] HE Kaiming, GKIOXARI G, DOLLáR P, et al. Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision. New York: IEEE, 2017: 2980-2988.
[18] NAKAJIMA Y, SAITO H. Efficient object-oriented semantic mapping with object detector[J]. IEEE access, 2019, 7: 3206-3213.
[19] PHAM Q H, HUA B S, NGUYEN T, et al. Real-time progressive 3D semantic segmentation for indoor scenes[C]//2019 IEEE Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2019: 1089-1098.
[20] TATENO K, TOMBARI F, LAINA I, et al. CNN-SLAM: real-time dense monocular SLAM with learned depth prediction[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6565-6574.
[21] CLARK R, WANG Sen, WEN Hongkai, et al. VINet: visual-inertial odometry as a sequence-to-sequence learning problem [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2017: 3995-4001.
[22] DETONE D, MALISIEWICZ T, RABINOVICH A. SuperPoint: self-supervised interest point detection and description[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City : IEEE, 2018: 337-33712.
[23] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence. New York: IEEE, 2018: 834-848.
[24] SONG Chengqun, ZENG Bo, SU Tong, et al. Data association and loop closure in semantic dynamic SLAM using the table retrieval method[J]. Applied intelligence, 2022, 52(10): 11472-11488.
[25] QIAN Zhentian, FU Jie, XIAO Jing. Towards accurate loop closure detection in semantic SLAM with 3D semantic covisibility graphs[J]. IEEE robotics and automation letters, 2022, 7(2): 2455-2462.
Similar References:

Memo

-

Last Update: 2024-11-05

Copyright © CAAI Transactions on Intelligent Systems