[1]周方波,赵怀林,刘华平.基于场景图谱的室内移动机器人目标搜索[J].智能系统学报,2022,17(5):1032-1038.[doi:10.11992/tis.202109011]
ZHOU Fangbo,ZHAO Huailin,LIU Huaping.Indoor mobile robot target search based on the scene graphs[J].CAAI Transactions on Intelligent Systems,2022,17(5):1032-1038.[doi:10.11992/tis.202109011]
点击复制
ZHOU Fangbo,ZHAO Huailin,LIU Huaping.Indoor mobile robot target search based on the scene graphs[J].CAAI Transactions on Intelligent Systems,2022,17(5):1032-1038.[doi:10.11992/tis.202109011]
基于场景图谱的室内移动机器人目标搜索
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第5期
页码:
1032-1038
栏目:
吴文俊人工智能科学技术奖论坛
出版日期:
2022-09-05
- Title:
- Indoor mobile robot target search based on the scene graphs
- Keywords:
- mobile robot; scene graph; navigation map; semantic map; semantic relationship graph; target search; landmark object; dynamic object
- 分类号:
- TP242
- 文献标志码:
- 2022-06-22
- 摘要:
- 在移动机器人执行日常家庭任务时，首先需要其能够在环境中避开障碍物，自主地寻找到房间中的物体。针对移动机器人如何有效在室内环境下对目标物体进行搜索的问题，提出了一种基于场景图谱的室内移动机器人目标搜索，其框架结合了导航地图、语义地图和语义关系图谱。在导航地图的基础上建立了包含地标物体位置信息的语义地图，机器人可以轻松对地标物体进行寻找。对于动态的物体，机器人根据语义关系图中物体之间的并发关系，优先到关系强度比较高的地标物体旁寻找。通过物理实验展示了机器人在语义地图和语义关系图的帮助下可以实现在室内环境下有效地寻找到目标，并显著地减少了搜索的路径长度，证明了该方法的有效性。
- Abstract:
- When a mobile robot performs daily household tasks, it must avoid obstacles in the environment and find objects in the room autonomously. To enable mobile robots to effectively search for target objects in indoor environments, an indoor mobile robot target search method based on a scene graph is proposed. Its framework combines a navigation map, a semantic map, and a semantic relationship graph. Based on the navigation map, a semantic map containing the location information of the landmark objects is established, so the robot can easily search for these objects. For dynamic objects, the robot will first look for landmark objects with a higher relationship strength based on the concurrent relationship between the objects in the semantic relationship graph. Physical experiments show that the robot can effectively find the target in an indoor environment using the semantic map and the semantic relationship graph, significantly reducing the search path length and hence proving the effectiveness of the method.
- 参考文献/References:
-
[1] 张毅, 陈起, 罗元. 室内环境下移动机器人三维视觉SLAM[J]. 智能系统学报, 2015, 10(4): 615–619
ZHANG Yi, CHEN Qi, LUO Yuan. Three dimensional visual SLAM for mobile robots in indoor environments[J]. CAAI transactions on intelligent systems, 2015, 10(4): 615–619
[2] ZHU Yuke, MOTTAGHI R, KOLVE E, et al. Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]//2017 IEEE International Conference on Robotics and Automation. Singapore. IEEE, 2017: 3357–3364.
[3] WORTSMAN M, EHSANI K, RASTEGARI M, et al. Learning to learn how to learn: self-adaptive visual navigation using meta-learning[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, IEEE, 2019: 6743–6752.
[4] MOUSAVIAN A, TOSHEV A, FI?ER M, et al. Visual representations for semantic target driven navigation[C]//2019 International Conference on Robotics and Automation (ICRA). Montreal, IEEE, 2019: 8846–8852.
[5] 刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1): 1–27
LIU Quan, ZHAI JianWei, ZHANG ZongZhang, et al. A survey on deep reinforcement learning[J]. Chinese journal of computers, 2018, 41(1): 1–27
[6] CODEVILLA F, MüLLER M, LóPEZ A, et al. End-to-end driving via conditional imitation learning[C]//2018 IEEE International Conference on Robotics and Automation. Brisbane, IEEE, 2018: 4693–4700.
[7] 贺超, 刘华平, 孙富春, 等. 采用Kinect的移动机器人目标跟踪与避障[J]. 智能系统学报, 2013, 8(5): 426–432
HE Chao, LIU Huaping, SUN Fuchun, et al. Target tracking and obstacle avoidance of mobile robot using Kinect[J]. CAAI transactions on intelligent systems, 2013, 8(5): 426–432
[8] MNIH V, BADIA A P, MIRZA M, et al. Asynchronous methods for deep reinforcement learning[C]//ICML’16: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48. New York: ACM, 2016: 1928–1937.
[9] 蔡军, 陈科宇, 张毅. 基于Kinect的改进移动机器人视觉SLAM[J]. 智能系统学报, 2018, 13(5): 734–740
CAI Jun, CHEN Keyu, ZHANG Yi. Improved V-SLAM for mobile robots based on Kinect[J]. CAAI transactions on intelligent systems, 2018, 13(5): 734–740
[10] KHATIB O. Real-time obstacle avoidance for manipulators and mobile robots[C]//Proceedings of 1985 IEEE International Conference on Robotics and Automation. St. Louis, IEEE, 1985: 500–505.
[11] FUENTES-PACHECO J, RUIZ-ASCENCIO J, RENDóN-MANCHA J M. Visual simultaneous localization and mapping: a survey[J]. Artificial intelligence review, 2015, 43(1): 55–81.
[12] HART P E, NILSSON N J, RAPHAEL B. A formal basis for the heuristic determination of minimum cost paths[J]. IEEE transactions on systems science and cybernetics, 1968, 4(2): 100–107.
[13] BERKHEMER O A, FRANSEN P S S, BEUMER D, et al. A randomized trial of intraarterial treatment for acute ischemic stroke[J]. New England journal of medicine, 2015, 372(1): 11–20.
[14] KARAMAN S, FRAZZOLI E. Sampling-based algorithms for optimal motion planning[J]. The international journal of robotics research, 2011, 30(7): 846–894.
[15] DU Heming, YU Xin, ZHENG Liang. Learning Object Relation Graph and Tentative Policy for Visual Navigation[C]//European Conference on Computer Vision. Cham: Springer, 2020: 19-34.
[16] DRUON R, YOSHIYASU Y, KANEZAKI A, et al. Visual object search by learning spatial context[J]. IEEE robotics and automation letters, 2020, 5(2): 1279–1286.
[17] QIU Y, PAL A, CHRISTENSEN H I. Learning hierarchical rela-tionships for object-goal navigation[C]//2020 Conference on Robot Learning (CoRL), Cambridge MA, USA, 2020: 503–514.
[18] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, IEEE, 2016: 770–778.
[19] HE Kaiming, GKIOXARI G, DOLLáR P, et al. Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision. Venice, Italy. IEEE, 2017: 2980–2988.
[20] KRISHNA R, ZHU Yuke, GROTH O, et al. Visual genome: connecting language and vision using crowdsourced dense image annotations[J]. International journal of computer vision, 2017, 123(1): 32–73.
[21] ZHANG Chengyao, ZHANG Xuqing, WANG Minshui, et al. A shale gas exploitation platform detection and positioning method based on YOLOv5[C]//2021 3rd International Conference on Intelligent Control, Measurement and Signal Processing and Intelligent Oil Field (ICMSP). Xi’an, IEEE, 2021: 459–464.
[22] LIN Tsungyi, MAIRE M, BELONGIE S, et al. Microsoft coco: Common objects in context[C]//European conference on computer vision. Springer, Cham. 2014: 740–755.
- 相似文献/References:
-
[1]蔡自兴,王 勇,王 璐.基于角点聚类的移动机器人自然路标检测与识别[J].智能系统学报,2006,1(1):52.
CAI Zi-xing,WANG Yong,WANG Lu.Corner clustering based detection and recognition of natural landmark for mobile robot[J].CAAI Transactions on Intelligent Systems,2006,1():52.
[2]杨甜甜,刘志远,陈 虹,等.移动机器人编队控制的现状与问题[J].智能系统学报,2007,2(4):21.
YANG Tian-tian,LIU Zhi-yuan,CHEN Hong,et al.Formation control of mobile robots: state and open prob lems[J].CAAI Transactions on Intelligent Systems,2007,2():21.
[3]李润伟,蔡自兴,童宇,等.基于ATM的提高狭窄环境探测精度的改进方法[J].智能系统学报,2008,3(4):283.
LI Run-wei,CAI Zi-xing,TONG Yu.Improving the accuracy of exploring the narrow environment by using ATM[J].CAAI Transactions on Intelligent Systems,2008,3():283.
[4]霍成立,谢 凡,秦世引.面向室内移动机器人的无迹滤波实时导航方法[J].智能系统学报,2009,4(4):295.
HUO Cheng-li,XIE Fan,QIN Shi-yin.A case study in realtime UKFbased navigation for indoor autonomous travel of mobile robots[J].CAAI Transactions on Intelligent Systems,2009,4():295.
[5]海 丹,李 勇,张 辉,等.无线传感器网络环境下基于粒子滤波的移动机器人SLAM算法[J].智能系统学报,2010,5(5):425.[doi:10.3969/j.issn.1673-4785.2010.05.008]
HAI Dan,LI Yong,ZHANG Hui,et al.Simultaneous localization and mapping of a mobile robot in wireless sensor networks based on particle filtering[J].CAAI Transactions on Intelligent Systems,2010,5():425.[doi:10.3969/j.issn.1673-4785.2010.05.008]
[6]房立金,王洪光.架空线移动机器人行走越障特点[J].智能系统学报,2010,5(6):492.
FANG Li-jin,WANG Hong-guang.Research on the characteristics of the movement and obstacleclearing processes of a wiresuspended mobile robot[J].CAAI Transactions on Intelligent Systems,2010,5():492.
[7]任立敏,王伟东,杜志江.移动机器人队形控制关键技术及其进展[J].智能系统学报,2013,8(5):381.[doi:10.3969/j.issn.1673-4785.201302011]
REN Limin,WANG Weidong,DU Zhijiang.Key technologies and development of? formation control of mobile robots[J].CAAI Transactions on Intelligent Systems,2013,8():381.[doi:10.3969/j.issn.1673-4785.201302011]
[8]贺超,刘华平,孙富春,等.采用Kinect的移动机器人目标跟踪与避障[J].智能系统学报,2013,8(5):426.[doi:10.3969/j.issn.1673-4785.201301028]
HE Chao,LIU Huaping,SUN Fuchun,et al.Target tracking and obstacle avoidance of mobile robot using Kinect[J].CAAI Transactions on Intelligent Systems,2013,8():426.[doi:10.3969/j.issn.1673-4785.201301028]
[9]阮晓钢,庞涛,张晓平,等.一种基于情感智能的机器人自主趋光行为研究[J].智能系统学报,2015,10(1):97.[doi:10.3969/j.issn.1673-4785.201312035]
RUAN Xiaogang,PANG Tao,ZHANG Xiaoping,et al.Research on the autonomous phototaxis behavior of a robot based on emotion intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():97.[doi:10.3969/j.issn.1673-4785.201312035]
[10]沈博闻,于宁波,刘景泰.仓储物流机器人集群的智能调度和路径规划[J].智能系统学报,2014,9(6):659.[doi:10.3969/j.issn.1673-4785.201312048]
SHEN Bowen,YU Ningbo,LIU Jingtai.Intelligent scheduling and path planning of warehouse mobile robots[J].CAAI Transactions on Intelligent Systems,2014,9():659.[doi:10.3969/j.issn.1673-4785.201312048]
备注/Memo
收稿日期:2021-09-02。
基金项目:国家自然科学基金项目 (U1613212)
作者简介:周方波，硕士研究生，主要研究方向为视觉语言导航和目标驱动导航;赵怀林，教授，博士，主要研究方向为机器人和人工智能。主持和参与包括国家高技术研究发展计划863项目在内的纵向横向科研项目20项，发表学术论文70篇，出版专著1部;刘华平，副教授，博士生导师，中国人工智能学会理事、中国人工智能学会认知系统与信息处理专业委员会秘书长，主持国家自然科学基金重点项目2 项。吴文俊人工智能科学技术奖获得者,主要研究方向为机器人感知、学习与控制、多模态信息融合。发表学术论文340 余篇。
通讯作者:刘华平. E-mail：hpliu@tsinghua.edu.cn
更新日期/Last Update:
1900-01-01