[1]辛菁,杜柯楠,王媛媛,等.基于迁移学习的移动机器人单帧图像坡度检测算法[J].智能系统学报,2021,16(1):81-91.[doi:10.11992/tis.202009009]
XIN Jing,DU Kenan,Wang Yuanyuan,et al.Single frame image slope detection algorithm for mobile robots based on transfer learning[J].CAAI Transactions on Intelligent Systems,2021,16(1):81-91.[doi:10.11992/tis.202009009]
点击复制
XIN Jing,DU Kenan,Wang Yuanyuan,et al.Single frame image slope detection algorithm for mobile robots based on transfer learning[J].CAAI Transactions on Intelligent Systems,2021,16(1):81-91.[doi:10.11992/tis.202009009]
基于迁移学习的移动机器人单帧图像坡度检测算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第1期
页码:
81-91
栏目:
学术论文—智能系统
出版日期:
2021-01-05
- Title:
- Single frame image slope detection algorithm for mobile robots based on transfer learning
- Keywords:
- unknown environment; mobile robot; slope detection; outdoor; single image; depth estimation; transfer learning; deep convolutional network
- 分类号:
- TP391
- 摘要:
- 针对未知环境下移动机器人平稳上坡控制对坡度感知精度的要求，本文提出了一种基于迁移学习的移动机器人单帧图像坡度检测算法。利用室内图像标准数据集训练深度卷积神经场?全连接超像素池化网络(deep convolutional neural field-fully connected superpixel pooling network, DCNF-FCSP)并获得室内单帧图像深度估计网络模型；将DCNF-FCSP模型中前5个图像特征提取层的网络参数迁移至室外图像深度估计网络中；固定室外图像深度估计网络中图像特征提取部分的网络参数，利用室外图像数据集对剩余5层的网络参数进行训练，从而得到室外单帧图像深度估计网络；将其应用在移动机器人坡度检测中，根据单帧斜坡图像估计出斜坡角度。标准数据集和实际场景的深度估计和坡度检测实验表明：本文所提出的基于迁移学习的移动机器人单帧图像坡度检测算法能够仅根据移动机器人车载相机采集的单帧斜坡RGB图像就可估计出精确的斜坡角度，满足移动机器人在未知环境中对坡度感知精度的要求。
- Abstract:
- To meet the requirement of slope perception accuracy for stable uphill control of mobile robots in an unknown environment, a single frame image slope detection algorithm for mobile robots is proposed in this paper based on transfer learning. First, the deep convolutional neural field-fully connected superpixel-pooling network (DCNF-FCSP) is trained using a standard indoor image dataset, and the depth estimation network model of indoor single frame images is obtained. Second, the network parameters of the first five image feature extraction layers in the DCNF-FCSP model are transferred to the outdoor image depth estimation network. Then, the network parameters of the image feature extraction part in the outdoor image depth estimation network are fixed, and the network parameters of the remaining five layers are trained using the outdoor image dataset; thus the outdoor single frame image depth estimation network is obtained. Finally, it is applied to the slope detection of a mobile robot, and the slope angle is estimated according to the single frame slope image. The depth estimation and slope detection experiments on a standard dataset and in actual scenes show that the proposed algorithm can estimate the accurate slope angle according to only a single frame slope RGB image captured by the vehicle-mounted camera of a mobile robot. The proposed algorithm meets the requirements of the slope perception accuracy of a mobile robot in an unknown environment.
- 参考文献/References:
-
[1] 朱江, 王耀南, 余洪山, 等. 未知环境下移动机器人自主感知斜坡地形方法[J]. 仪器仪表学报, 2010, 31(8):1916-1920
ZHU Jiang, WANG Yaonan, YU Hongshan, et al. Mobile robot autonomous perceiving slope terrain under unknown environment[J]. Chinese journal of scientific instrument, 2010, 31(8):1916-1920
[2] YU Jinxia, CAI Zixing, DUAN Zhuohua. Dead reckoning of mobile robot in complex terrain based on proprioceptive sensors[C]//Proceedings of 2008 International Conference on Machine Learning and Cybernetics. Kunming, China, 2008:1930-1935.
[3] LI Zhibin, TSAGARAKIS N G, CALDWELL D G. Stabilizing humanoids on slopes using terrain inclination estimation[C]//Proceedings of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo, Japan, 2013:4124-4129.
[4] LU Jixin, KOBAYASHI Y, EMARU T, et al. Indoor slope and edge detection by using two-dimensional EKF-SLAM with orthogonal assumption[J]. International journal of advanced robotic systems, 2015, 12(4):44.
[5] HARA S, SHIMIZU T, KONISHI M, et al. Autonomous mobile robot for outdoor slope using 2D LiDAR with uniaxial gimbal mechanism[J]. Journal of robotics and mechatronics, 2020, 32(6):1173-1182.
[6] TAREEN S A K, KHAN H M. Novel slope detection and calculation techniques for mobile robots[C]//Proceedings of the 2nd IEEE International Conference on Robotics and Artificial Intelligence. Rawalpindi, Pakistan, 2016:158-163.
[7] 王泉德, 张松涛. 基于多尺度特征融合的单目图像深度估计[J]. 华中科技大学学报(自然科学版), 2020, 48(5):7-12
WANG Quande, ZHANG Songtao. Monocular depth estimation with multi-scale feature fusion[J]. Journal of Huazhong University of Science and Technology (natural science edition), 2020, 48(5):7-12
[8] 毕天腾, 刘越, 翁冬冬, 等. 基于监督学习的单幅图像深度估计综述[J]. 计算机辅助设计与图形学学报, 2018, 30(8):1383-1393
BI Tianteng, LIU Yue, WENG Dongdong, et al. Survey on supervised learning based depth estimation from a single image[J]. Journal of computer-aided design and computer graphics, 2018, 30(8):1383-1393
[9] 何磊, 苏松志, 李绍滋. 单摄像头下基于样本学习的人体深度估计[J]. 智能系统学报, 2014, 9(2):161-167
HE Lei, SU Songzhi, LI Shaozi. Human depth estimation on the basis of the sample learning method under a single camera[J]. CAAI transactions on intelligent systems, 2014, 9(2):161-167
[10] 孙蕴瀚, 史金龙, 孙正兴. 利用自监督卷积网络估计单图像深度信息[J]. 计算机辅助设计与图形学学报, 2020,32(4):643-651
SUN Yunhan, SHI Jinlong, SUN Zhengxing. Estimating depth from single image using unsupervised convolutional network[J]. Journal of computer-aided design and computer graphics, 2020,32(4):643-651
[11] EIGEN D, PUHRSCH C, FERGUS R. Depth map prediction from a single image using a multi-scale deep network[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Quebec, Canada, 2014:2366-2374.
[12] LIU Fayao, SHEN Chunhua, LIN Guosheng, et al. Learning depth from single monocular images using deep convolutional neural fields[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(10):2024-2039.
[13] RANFTL R, LASINGER K, HAFNER D, et al. Towards robust monocular depth estimation:mixing datasets for zero-shot cross-dataset transfer[J]. IEEE transactions on pattern analysis and machine intelligence, 2020, 42(8):1939-3539.
[14] HU Junjie, OZAY M, ZHANG Yan, et al. Revisiting single image depth estimation:toward higher resolution maps with accurate object boundaries[J]. arXiv:1803.0867, 2018.
[15] KUZNIETSOV Y, STüCKLER J, LEIBE B. Semi-supervised deep learning for monocular depth map prediction[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA:CVPR, 2017:2215-2223.
[16] Fu H, Gong M, Wang C, et al. A Compromise Principle in Deep Monocular Depth Estimation[J]. arXiv preprint arXiv:1708.0826, 2018.
[17] 梁煜, 张金铭, 张为. 一种改进的卷积神经网络的室内深度估计方法[J]. 天津大学学报(自然科学与工程技术版), 2020, 53(8):840-846
LIANG Yu, ZHANG Jinming, ZHANG Wei. An improved indoor depth estimation method using convolutional neural networks[J]. Journal of Tianjin University(science and technology), 2020, 53(8):840-846
[18] GODARD C, MAC AODHA O, BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017:6602-6611.
[19] LAINA I, RUPPRECHT C, BELAGIANNIS V, et al. Deeper depth prediction with fully convolutional residual networks[C]//Proceedings of the 4th International Conference on 3D Vision. Stanford, CA, USA, 2016:239-248.
[20] 许敏, 史荧中, 葛洪伟, 等. 一种具有迁移学习能力的RBF-NN算法及其应用[J]. 智能系统学报, 2018, 13(6):959-966
XU Min, SHI Yingzhong, GE Hongwei, et al. A RBF-NN algorithm with transfer learning ability and its application[J]. CAAI transactions on intelligent systems, 2018, 13(6):959-966
[21] 李耀宇, 王宏民, 张一帆, 等. 基于结构化深度学习的单目图像深度估计[J]. 机器人, 2017, 39(6):812-819
LI Yaoyu, WANG Hongmin, ZHANG Yifan, et al. Structured deep learning based depth estimation from a monocular image[J]. Robot, 2017, 39(6):812-819
[22] SAXENA A, SUN Min, NG A Y. Make3D:learning 3D scene structure from a single still image[J]. IEEE transactions on pattern analysis and machine intelligence, 2009, 31(5):824-840.
- 相似文献/References:
-
[1]杨甜甜,刘志远,陈 虹,等.移动机器人编队控制的现状与问题[J].智能系统学报,2007,2(4):21.
YANG Tian-tian,LIU Zhi-yuan,CHEN Hong,et al.Formation control of mobile robots: state and open prob lems[J].CAAI Transactions on Intelligent Systems,2007,2():21.
[2]李润伟,蔡自兴,童宇,等.基于ATM的提高狭窄环境探测精度的改进方法[J].智能系统学报,2008,3(4):283.
LI Run-wei,CAI Zi-xing,TONG Yu.Improving the accuracy of exploring the narrow environment by using ATM[J].CAAI Transactions on Intelligent Systems,2008,3():283.
[3]霍成立,谢 凡,秦世引.面向室内移动机器人的无迹滤波实时导航方法[J].智能系统学报,2009,4(4):295.
HUO Cheng-li,XIE Fan,QIN Shi-yin.A case study in realtime UKFbased navigation for indoor autonomous travel of mobile robots[J].CAAI Transactions on Intelligent Systems,2009,4():295.
[4]海 丹,李 勇,张 辉,等.无线传感器网络环境下基于粒子滤波的移动机器人SLAM算法[J].智能系统学报,2010,5(5):425.[doi:10.3969/j.issn.1673-4785.2010.05.008]
HAI Dan,LI Yong,ZHANG Hui,et al.Simultaneous localization and mapping of a mobile robot in wireless sensor networks based on particle filtering[J].CAAI Transactions on Intelligent Systems,2010,5():425.[doi:10.3969/j.issn.1673-4785.2010.05.008]
[5]房立金,王洪光.架空线移动机器人行走越障特点[J].智能系统学报,2010,5(6):492.
FANG Li-jin,WANG Hong-guang.Research on the characteristics of the movement and obstacleclearing processes of a wiresuspended mobile robot[J].CAAI Transactions on Intelligent Systems,2010,5():492.
[6]任立敏,王伟东,杜志江.移动机器人队形控制关键技术及其进展[J].智能系统学报,2013,8(5):381.[doi:10.3969/j.issn.1673-4785.201302011]
REN Limin,WANG Weidong,DU Zhijiang.Key technologies and development of? formation control of mobile robots[J].CAAI Transactions on Intelligent Systems,2013,8():381.[doi:10.3969/j.issn.1673-4785.201302011]
[7]贺超,刘华平,孙富春,等.采用Kinect的移动机器人目标跟踪与避障[J].智能系统学报,2013,8(5):426.[doi:10.3969/j.issn.1673-4785.201301028]
HE Chao,LIU Huaping,SUN Fuchun,et al.Target tracking and obstacle avoidance of mobile robot using Kinect[J].CAAI Transactions on Intelligent Systems,2013,8():426.[doi:10.3969/j.issn.1673-4785.201301028]
[8]阮晓钢,庞涛,张晓平,等.一种基于情感智能的机器人自主趋光行为研究[J].智能系统学报,2015,10(1):97.[doi:10.3969/j.issn.1673-4785.201312035]
RUAN Xiaogang,PANG Tao,ZHANG Xiaoping,et al.Research on the autonomous phototaxis behavior of a robot based on emotion intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():97.[doi:10.3969/j.issn.1673-4785.201312035]
[9]沈博闻,于宁波,刘景泰.仓储物流机器人集群的智能调度和路径规划[J].智能系统学报,2014,9(6):659.[doi:10.3969/j.issn.1673-4785.201312048]
SHEN Bowen,YU Ningbo,LIU Jingtai.Intelligent scheduling and path planning of warehouse mobile robots[J].CAAI Transactions on Intelligent Systems,2014,9():659.[doi:10.3969/j.issn.1673-4785.201312048]
[10]李贞辉,王洪光,王越超.一种双臂巡检机器人的质心调整策略[J].智能系统学报,2014,9(6):665.[doi:10.3969/j.issn.1673-4785.201311045]
LI Zhenhui,WANG Hongguang,WANG Yuechao.Centroid adjustment strategy for a dual arm inspection robot[J].CAAI Transactions on Intelligent Systems,2014,9():665.[doi:10.3969/j.issn.1673-4785.201311045]
[11]蔡自兴,王 勇,王 璐.基于角点聚类的移动机器人自然路标检测与识别[J].智能系统学报,2006,1(1):52.
CAI Zi-xing,WANG Yong,WANG Lu.Corner clustering based detection and recognition of natural landmark for mobile robot[J].CAAI Transactions on Intelligent Systems,2006,1():52.
备注/Memo
收稿日期:2020-09-01。
基金项目:国家自然科学基金项目(61873200，61833013，U20A20225)
作者简介:辛菁，教授，主要研究方向为机器人智能感知与自主运动，主持国家自然科学基金项目2项、省部级基金项目多项。发表学术论文40余篇;杜柯楠，硕士研究生，主要研究方向为语义地图构建;刘丁，教授，主要研究方向为复杂系统建模与优化控制，主持国家自然科学基金项目3项、国家科技重大专项及863项目多项，获国家技术发明奖二等奖1项、省科技进步一等奖3项。发表学术论文200余篇，出版专著3部
通讯作者:刘丁.E-mail：liud@xaut.edu.cn
更新日期/Last Update:
2021-02-25