融合深度学习与神经隐式表征的视觉SLAM系统

PDF下载HTML

Title:

Deep-learning-enhanced visual SLAM with neural implicit scene representation

作者:

张含笑, 邢向磊

哈尔滨工程大学 智能科学与工程学院, 黑龙江 哈尔滨 150001

Author(s):

ZHANG Hanxiao, XING Xianglei

College of Intelligent Science and Engineering, Harbin Engineering University, Harbin 150001, China

关键词:

神经辐射场; 视觉SLAM; 回环检测; 位姿估计; 深度学习; 三维重建; 语义嵌入; 轨迹预测

Keywords:

neural radiation field; visual SLAM; loop detection; pose estimation; deep learning; 3D reconstruction; semantic embedding; trajectory prediction

分类号:

TP391.41

DOI:

10.11992/tis.202505029

摘要:

近年来，神经辐射场在三维重建任务中展现出卓越性能。然而，应用在视觉同时定位与地图构建(simultaneous localization and mapping, SLAM)中因缺乏全局优化机制容易导致系统定位精度不足以及重建失败。针对该问题，本文提出一种融合深度学习位姿估计与神经隐式表征的视觉SLAM系统。通过稠密束调整层以及高效的全局优化机制对相机位姿和深度进行像素级的循环迭代，并基于神经辐射场方法更新全局一致的隐式重建表面，使得系统在精准定位的同时能够重建高保真场景，并且在此基础上引入语言查询机制，增强系统的交互能力。在EuRoC和Replica数据集上进行大量实验，在不同的输入条件下，分别与3类基准方法进行对比，结果表明该系统在跟踪鲁棒性和重建精度方面相较于现有方法表现更优。本方法可为后续基于神经辐射场的视觉SLAM方法提供参考。

Abstract:

In recent years, neural radiation fields have demonstrated strong capability in high-fidelity three-dimensional scene reconstruction. However, visual simultaneous localization and mapping(SLAM) systems that employ neural radiance fields still face challenges in localization accuracy and the flexibility of explicit scene representation. To address these limitations, this work proposes a visual SLAM system that integrates deep-learning-based pose estimation with neural implicit scene representation. Through dense bundle adjustment layers and efficient global optimization mechanisms, the camera pose and depth are iteratively optimized at the pixel level, and a globally consistent implicit reconstruction surface is incrementally updated based on neural radiation fields, enabling the system to reconstruct high-fidelity scenes while achieving accurate localization. Furthermore, a language query mechanism was introduced to enhance the system’s interactive capability. Extensive experiments were conducted on the EuRoC and Replica datasets, and the results were compared with those of three benchmark methods under different input conditions. The results showed that the proposed system outperformed existing methods in terms of tracking robustness and reconstruction accuracy, providing a reference for subsequent visual SLAM methods based on neural radiation fields.