[1]田顺钰,欧阳勇平,魏长赟.融合专家纠偏策略的移动机器人动态环境避障方法[J].智能系统学报,2024,19(6):1492-1502.[doi:10.11992/tis.202304056]
TIAN Shunyu,OUYANG Yongping,WEI Changyun.Collision avoidance approach with heuristic correction policy for mobile robot navigation in dynamic environments[J].CAAI Transactions on Intelligent Systems,2024,19(6):1492-1502.[doi:10.11992/tis.202304056]
点击复制
TIAN Shunyu,OUYANG Yongping,WEI Changyun.Collision avoidance approach with heuristic correction policy for mobile robot navigation in dynamic environments[J].CAAI Transactions on Intelligent Systems,2024,19(6):1492-1502.[doi:10.11992/tis.202304056]
融合专家纠偏策略的移动机器人动态环境避障方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第6期
页码:
1492-1502
栏目:
学术论文—智能系统
出版日期:
2024-12-05
- Title:
- Collision avoidance approach with heuristic correction policy for mobile robot navigation in dynamic environments
- Keywords:
- mobile robots; deep reinforcement learning; robot navigation; non-structural environment; dynamic collision avoidance; heuristic correction policy; self-learning; end-to-end
- 分类号:
- TP273+.2
- 摘要:
- 基于深度强化学习（deep reinforcement learning，DRL）的移动机器人无地图导航技术备受机器人和相关研究领域的关注，在非结构化环境中避免与动态障碍物的碰撞是需要解决的重要难题。为此提出一种融合专家纠偏策略的机器人自主导航DRL方法，该算法将24线激光雷达传感器信息、目标位置信息和机器人速度信息作为深度强化学习的输入，并输出控制机器人的动作指令。实验结果表明，相较于其他算法，该算法可以在保证安全的前提下以更短的距离和时间到达目标。同时将所提出的方法部署在真实机器人上，验证和评估算法的性能，为机器人动态环境避障导航提供一种技术参考。
- Abstract:
- Mapless navigation for mobile robots based on deep reinforcement learning (DRL) has received increasing attention from robotics and related research fields. The major challenge in mapless navigation is collision avoidance of dynamic obstacles in unstructured environments. Therefore, this paper proposes a DRL algorithm that incorporates a heuristic correction policy for robot autonomous navigation. The algorithm utilizes information from a 24-line laser radar sensor, target location, and robot velocity as inputs for DRL to generate action commands that regulate the robot’s motion. Experimental results demonstrate that, compared to other algorithms, the proposed approach can reach the target more efficiently in terms of distance and time while ensuring safety. Moreover, the algorithm is implemented in a real robot to verify and evaluate its performance, providing a technical reference for collision avoidance during its navigation in dynamic environments.
- 参考文献/References:
-
[1] DESOUZA G N, KAK A C. Vision for mobile robot navigation: a survey[J]. IEEE transactions on pattern analysis and machine intelligence, 2002, 24(2): 237-267.
[2] KRUSE T, PANDEY A K, ALAMI R, et al. Human-aware robot navigation: a survey[J]. Robotics and autonomous systems, 2013, 61(12): 1726-1743.
[3] ADAMKIEWICZ M, CHEN T, CACCAVALE A, et al. Vision-only robot navigation in a neural radiance world[J]. IEEE robotics and automation letters, 2022, 7(2): 4606-4613.
[4] ZHAO Haoning, WANG Chaoqun, GUO Rui, et al. Autonomous live working robot navigation with real-time detection and motion planning system on distribution line[J]. High voltage, 2022, 7(6): 1204-1216.
[5] EBADI K, BERNREITER L, BIGGIE H, et al. Present and future of SLAM in extreme environments: The DARPA SubT challenge[J]. IEEE transactions on robotics, 2024, 40: 936-959.
[6] SINGH K J, KAPOOR D S, THAKUR K, et al. Map making in social indoor environment through robot navigation using active SLAM[J]. IEEE access, 2022, 10: 134455-134465.
[7] GRANDO R B, DE JESUS J C, KICH V A, et al. Double critic deep reinforcement learning for mapless 3D navigation of unmanned aerial vehicles[J]. Journal of intelligent & robotic systems, 2022, 104(2): 29.
[8] LI Hanxiao, LUO Biao, SONG Wei, et al. Predictive hierarchical reinforcement learning for path-efficient mapless navigation with moving target[J]. Neural networks, 2023, 165: 677-688.
[9] LIKHACHEV M, FERGUSON D I, GORDON G J, et al. Anytime dynamic A*: An anytime, replanning algorithm[C]//ICAPS. Monterey, 2005, 5: 262-271.
[10] KHATIB O. Real-time obstacle avoidance for manipulators and mobile robots[M]//Cox IJ, Wilfong GT. Autonomous Robot Vehicles. New York: Springer, 1986: 396-404.
[11] NASIR J, ISLAM F, MALIK U, et al. RRT*-SMART: a rapid convergence implementation of RRT[J]. International journal of advanced robotic systems, 2013, 10(7): 299.
[12] DURRANT-WHYTE H, BAILEY T. Simultaneous localization and mapping: part I[J]. IEEE robotics & automation magazine, 2006, 13(2): 99-110.
[13] ALHMIEDAT T, MAREI A M, MESSOUDI W, et al. A SLAM-based localization and navigation system for social robots: the pepper robot case[J]. Machines, 2023, 11(2): 158.
[14] TAHERI H, XIA Zhaochun. SLAM: definition and evolution[J]. Engineering applications of artificial intelligence, 2021, 97: 104032.
[15] CHAPLOT D S, GANDHI D, GUPTA S, et al. Learning to explore using active neural SLAM[EB/OL]. (2020-04-10)[2021-01-01]. http://arxiv.org/abs/2004.05155.
[16] XUE Honghu, HEIN B, BAKR M, et al. Using deep reinforcement learning with automatic curriculum learning for mapless navigation in intralogistics[J]. Applied sciences, 2022, 12(6): 3153.
[17] HAN Yiheng, ZHAN I H, ZHAO Wang, et al. Deep reinforcement learning for robot collision avoidance with self-state-attention and sensor fusion[J]. IEEE robotics and automation letters, 2022, 7(3): 6886-6893.
[18] 欧阳勇平, 魏长赟, 蔡帛良. 动态环境下分布式异构多机器人避障方法研究[J]. 智能系统学报, 2022, 17(4): 752-763.
OUYANG Yongping, WEI Changyun, CAI Boliang. Collision avoidance approach for distributed heterogeneous multirobot systems in dynamic environments[J]. CAAI transactions on intelligent systems, 2022, 17(4): 752-763.
[19] ZHU Wei, HAYASHIBE M. A hierarchical deep reinforcement learning framework with high efficiency and generalization for fast and safe navigation[J]. IEEE transactions on industrial electronics, 2023, 70(5): 4962-4971.
[20] 李鹏, 阮晓钢, 朱晓庆, 等. 基于深度强化学习的区域化视觉导航方法[J]. 上海交通大学学报, 2021, 55(5): 575-585.
LI Peng, RUAN Xiaogang, ZHU Xiaoqing, et al. A regionalization vision navigation method based on deep reinforcement learning[J]. Journal of Shanghai Jiaotong University, 2021, 55(5): 575-585.
[21] PUTERMAN M L. Chapter 8 Markov decision processes[M]//Handbooks in Operations Research and Management Science. Amsterdam: Elsevier, 1990: 331-434.
[22] 刘克. 实用马尔可夫决策过程[M]. 北京: 清华大学出版社, 2004.
LIU Ke. Practical Markov decision processes[M]. Beijing: Tsinghua University Press, 2024.
[23] SILVER D, LEVER G, HEESS N, et al. Deterministic policy gradient algorithms[C]//International conference on machine learning. Beijing:ICML, 2014: 387-395.
[24] FUJIMOTO S, VAN HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[EB/OL]. (2018-02-26)[2021-01-01]. http://arxiv.org/abs/1802.09477.
[25] SCHAUL T, QUAN J, ANTONOGLOU I, et al. Prioritized experience replay[EB/OL]. (2015-11-18)[2021-01-01]. http://arxiv.org/abs/1511.05952.
[26] LI Peng, DING Xiangcheng, SUN Hongfang, et al. Research on dynamic path planning of mobile robot based on improved DDPG algorithm[J]. Mobile information systems, 2021, 2021: 5169460.
[27] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[EB/OL]. (2015-09-09)[2021-01-01]. http://arxiv.org/abs/1509.02971.
- 相似文献/References:
-
[1]蔡自兴,王 勇,王 璐.基于角点聚类的移动机器人自然路标检测与识别[J].智能系统学报,2006,1(1):52.
CAI Zi-xing,WANG Yong,WANG Lu.Corner clustering based detection and recognition of natural landmark for mobile robot[J].CAAI Transactions on Intelligent Systems,2006,1():52.
[2]杨甜甜,刘志远,陈 虹,等.移动机器人编队控制的现状与问题[J].智能系统学报,2007,2(4):21.
YANG Tian-tian,LIU Zhi-yuan,CHEN Hong,et al.Formation control of mobile robots: state and open prob lems[J].CAAI Transactions on Intelligent Systems,2007,2():21.
[3]李润伟,蔡自兴,童宇,等.基于ATM的提高狭窄环境探测精度的改进方法[J].智能系统学报,2008,3(4):283.
LI Run-wei,CAI Zi-xing,TONG Yu.Improving the accuracy of exploring the narrow environment by using ATM[J].CAAI Transactions on Intelligent Systems,2008,3():283.
[4]霍成立,谢 凡,秦世引.面向室内移动机器人的无迹滤波实时导航方法[J].智能系统学报,2009,4(4):295.
HUO Cheng-li,XIE Fan,QIN Shi-yin.A case study in realtime UKFbased navigation for indoor autonomous travel of mobile robots[J].CAAI Transactions on Intelligent Systems,2009,4():295.
[5]海 丹,李 勇,张 辉,等.无线传感器网络环境下基于粒子滤波的移动机器人SLAM算法[J].智能系统学报,2010,5(5):425.[doi:10.3969/j.issn.1673-4785.2010.05.008]
HAI Dan,LI Yong,ZHANG Hui,et al.Simultaneous localization and mapping of a mobile robot in wireless sensor networks based on particle filtering[J].CAAI Transactions on Intelligent Systems,2010,5():425.[doi:10.3969/j.issn.1673-4785.2010.05.008]
[6]房立金,王洪光.架空线移动机器人行走越障特点[J].智能系统学报,2010,5(6):492.
FANG Li-jin,WANG Hong-guang.Research on the characteristics of the movement and obstacleclearing processes of a wiresuspended mobile robot[J].CAAI Transactions on Intelligent Systems,2010,5():492.
[7]任立敏,王伟东,杜志江.移动机器人队形控制关键技术及其进展[J].智能系统学报,2013,8(5):381.[doi:10.3969/j.issn.1673-4785.201302011]
REN Limin,WANG Weidong,DU Zhijiang.Key technologies and development of? formation control of mobile robots[J].CAAI Transactions on Intelligent Systems,2013,8():381.[doi:10.3969/j.issn.1673-4785.201302011]
[8]贺超,刘华平,孙富春,等.采用Kinect的移动机器人目标跟踪与避障[J].智能系统学报,2013,8(5):426.[doi:10.3969/j.issn.1673-4785.201301028]
HE Chao,LIU Huaping,SUN Fuchun,et al.Target tracking and obstacle avoidance of mobile robot using Kinect[J].CAAI Transactions on Intelligent Systems,2013,8():426.[doi:10.3969/j.issn.1673-4785.201301028]
[9]阮晓钢,庞涛,张晓平,等.一种基于情感智能的机器人自主趋光行为研究[J].智能系统学报,2015,10(1):97.[doi:10.3969/j.issn.1673-4785.201312035]
RUAN Xiaogang,PANG Tao,ZHANG Xiaoping,et al.Research on the autonomous phototaxis behavior of a robot based on emotion intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():97.[doi:10.3969/j.issn.1673-4785.201312035]
[10]沈博闻,于宁波,刘景泰.仓储物流机器人集群的智能调度和路径规划[J].智能系统学报,2014,9(6):659.[doi:10.3969/j.issn.1673-4785.201312048]
SHEN Bowen,YU Ningbo,LIU Jingtai.Intelligent scheduling and path planning of warehouse mobile robots[J].CAAI Transactions on Intelligent Systems,2014,9():659.[doi:10.3969/j.issn.1673-4785.201312048]
备注/Memo
收稿日期:2023-4-27。
基金项目:国家自然科学基金项目（52371275）.
作者简介:田顺钰，硕士研究生，主要研究方向为智能自主无人系统。E-mail：shunyutian@163.com;欧阳勇平，硕士研究生，主要研究方向为多机器人协作、智能无人系统。E-mail：oy15961483506@163.com;魏长赟魏长赟，副教授，博士，主要研究方向为智能自主无人系统。发表学术论文30余篇。E-mail：c.wei@hhu.edu.cn。
通讯作者:魏长赟. E-mail：c.wei@hhu.edu.cn
更新日期/Last Update:
2024-11-05