[1]YANG Hui,ZHANG Ting,JIN Sheng,et al.Visual loop closure detection based on binary generative adversarial network[J].CAAI Transactions on Intelligent Systems,2021,16(4):673-682.[doi:10.11992/tis.202007007]
Copy

Visual loop closure detection based on binary generative adversarial network

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 4 Page number: 673-682 Column: 学术论文—机器感知与模式识别 Public date: 2021-07-05
Title:
Visual loop closure detection based on binary generative adversarial network
Author(s):
YANG Hui;  ZHANG Ting;  JIN Sheng;  CHEN Liang;  SUN Rongchuan;  SUN Lining
School of Mechanical and Electric Engineering, Soochow University, Suzhou 215021, China
Keywords:
loop closure detection; unsupervised learning; binary descriptor; BoVW; visual SLAM; generative adversarial; feature extraction; deep learning
CLC:
TP181
DOI:
10.11992/tis.202007007
Abstract:
In view of the problem that the existing loop closure detection models are mostly trained based on supervised learning and require a large amount of labeled data, this paper proposes a new method for visual loop closure detection. The idea of the generative adversarial network is adopted, and thus, a deep neural network is designed and trained through unsupervised learning methods to extract more discriminative binary feature descriptors with low dimensions. The distance propagation loss function and a binarized representation entropy loss function are introduced into the neural network. The first loss function can help spread the Hamming distance relationship of the high-dimensional feature space to the low-dimensional feature space, and the second one increases the diversity of the low-dimensional feature representation. The extracted local features are fused into global features by using the BoVW model for further loop closure detection. Experimental results show that the proposed method has better performance than feature extraction algorithms such as SIFT and ORB in complex scenes that have a strong viewpoint and appearance changes, and its performance is comparable with that of supervised deep networks such as AlexNet and AMOSNet. It is especially suitable for loop closure detection in large-scale open scenes because the time-consuming and tedious process of supervised data annotation is completely avoided with the use of unsupervised learning. Moreover, the binary feature descriptors can greatly save storage space and computing resources.
References:
[1] KONOLIGE K, AGRAWAL M. FrameSLAM: from bundle adjustment to real-time visual mapping[J]. IEEE transactions on robotics, 2008, 24(5): 1066-1077.
[2] 张毅, 沙建松. 基于图优化的移动机器人视觉SLAM[J]. 智能系统学报, 2018, 13(2): 290-295
ZHANG Yi, SHA Jiansong. Visual-SLAM for mobile robot based on graph optimization[J]. CAAI transactions on intelligent systems, 2018, 13(2): 290-295
[3] HO K L, NEWMAN P. Detecting loop closure with scene sequences[J]. International journal of computer vision, 2007, 74(3): 261-286.
[4] 刘强, 段富海, 桑勇, 等. 复杂环境下视觉SLAM闭环检测方法综述[J]. 机器人, 2019, 41(1): 112-123, 136
LIU Qiang, DUAN Fuhai, SANG Yong, et al. A survey of loop-closure detection method of visual SLAM in complex environments[J]. Robot, 2019, 41(1): 112-123, 136
[5] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the 17th IEEE International Conference on Computer Vision. Kerkyra, Greece, 2002: 1150-1157.
[6] BAY H, TUYTELAARS T, VAN GOOL L. SURF: speeded up robust features[C]//Computer vision-ECCV 2006. Graz, Austria, 2006: 404-417.
[7] SIVIC J, ZISSERMAN A. Video Google: A text retrieval approach to object matching in videos[C]//Proceedings of the 9th IEEE International Conference on Computer Vision. Nice, France, 2003: 1470-1477.
[8] CHEN Zetao, LAM O, JACOBSON A, et al. Convolutional neural network-based place recognition[C]//Australasian Conference on Robotics and Automati-on. Melbourne, Australasian, 2014: 8-14.
[9] SüNDERHAUF N, SHIRAZI S, DAYOUB F, et al. On the performance of ConvNet features for place recognition[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems. Hamburg, Germany, 2015: 4297-4304.
[10] GAO Xiang, ZHANG Tao. Unsupervised learning to detect loops using deep neural networks for visual SLAM system[J]. Autonomous robots, 2017, 41(1): 1-18.
[11] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada, 2014: 2672-2680.
[12] HINTON G E, ZEME R S. Autoencoders, minimum description length and Helmholtz free energy[C]//Proceedings of the 6th International Conference on Neural Information Processing Systems. Denver, Colorado, USA, 1993: 3-10.
[13] SMOLENSKY P. Information processing in dynamical systems: foundations of harmony theory[M]//RUMELHART D E, MCCLELLAND J L. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge: MIT Press, 1986.
[14] SHIN D W, HO Y S, KIM E S. Loop closure detection in simultaneous localization and mapping using descriptor from generative adversarial network[J]. Journal of electronic imaging, 2019, 28(1): 013014.
[15] RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[C]//Proceedings of the 4th International Conference on Learning Representations. San Juan, Puerto Rico, 2016: 97-108.
[16] SALIMANS T, GOODFELLOW I, ZAREMBA W, et al. Improved techniques for training GANs[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain, 2016: 2234-2242.
[17] DONG Haowen, YANG Y H. Training generative adversarial networks with binary neurons by end-to-end backpropagation[EB/OL]. (2018-12-12) [2020-01-01] https://arxiv.org/abs/1810.04714.
[18] CAO Yanshuai, DING G W, LUI K Y C, et al. Improving GAN training via binarized representation entropy (BRE) regularization[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2018: 1-22.
[19] ZHOU Bolei, LAPEDRIZA A, KHOSLA A, et al. Places: A 10 million image database for scene recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 40(6): 1452-1464.
[20] CUMMINS M, NEWMAN P. FAB-MAP: Probabilistic localization and mapping in the space of appearance[J]. The international journal of robotics research, 2008, 27(6): 647-665.
[21] CHOI Y, KIM N, PARK K, et al. All-day visual place recognition: benchmark dataset and baseline[C]//Proceedings of 2015 IEEE International Conference on Computer Vision and Pattern Recognition Workshops. Boston, USA, 2015: 8-13.
[22] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA, 2012: 1097-1105.
[23] CHEN Zetao, JACOBSON A, SüNDERHAUF N, et al. Deep learning features at scale for visual place recognition[C]//Proceedings of 2017 IEEE International Conference on Robotics and Automation. Singapore, 2017: 3223-3230.
[24] ZAFFAR M, KHALIQ A, EHSAN S, et al. Levelling the playing field: A comprehensive comparison of visual place recognition approaches under changing conditions[EB/OL]. (2019-04-29) [2020-02-01] https://arxiv.org/abs/1903.09107?context=cs.CV.
[25] MEMON A R, WANG Hesheng, HUSSAIN A. Loop closure detection using supervised and unsupervised deep neural networks for monocular SLAM systems[J]. Robotics and autonomous systems, 2020, 126: 103470.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems