[1]TIAN Shunyu,OUYANG Yongping,WEI Changyun.Collision avoidance approach with heuristic correction policy for mobile robot navigation in dynamic environments[J].CAAI Transactions on Intelligent Systems,2024,19(6):1492-1502.[doi:10.11992/tis.202304056]
Copy

Collision avoidance approach with heuristic correction policy for mobile robot navigation in dynamic environments

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 6 Page number: 1492-1502 Column: 学术论文—智能系统 Public date: 2024-12-05
Title:
Collision avoidance approach with heuristic correction policy for mobile robot navigation in dynamic environments
Author(s):
TIAN Shunyu;  OUYANG Yongping;  WEI Changyun
College of Mechanical and Electrical Engineering, Hohai University, Changzhou 213251, China
Keywords:
mobile robots; deep reinforcement learning; robot navigation; non-structural environment; dynamic collision avoidance; heuristic correction policy; self-learning; end-to-end
CLC:
TP273+.2
DOI:
10.11992/tis.202304056
Abstract:
Mapless navigation for mobile robots based on deep reinforcement learning (DRL) has received increasing attention from robotics and related research fields. The major challenge in mapless navigation is collision avoidance of dynamic obstacles in unstructured environments. Therefore, this paper proposes a DRL algorithm that incorporates a heuristic correction policy for robot autonomous navigation. The algorithm utilizes information from a 24-line laser radar sensor, target location, and robot velocity as inputs for DRL to generate action commands that regulate the robot’s motion. Experimental results demonstrate that, compared to other algorithms, the proposed approach can reach the target more efficiently in terms of distance and time while ensuring safety. Moreover, the algorithm is implemented in a real robot to verify and evaluate its performance, providing a technical reference for collision avoidance during its navigation in dynamic environments.
References:
[1] DESOUZA G N, KAK A C. Vision for mobile robot navigation: a survey[J]. IEEE transactions on pattern analysis and machine intelligence, 2002, 24(2): 237-267.
[2] KRUSE T, PANDEY A K, ALAMI R, et al. Human-aware robot navigation: a survey[J]. Robotics and autonomous systems, 2013, 61(12): 1726-1743.
[3] ADAMKIEWICZ M, CHEN T, CACCAVALE A, et al. Vision-only robot navigation in a neural radiance world[J]. IEEE robotics and automation letters, 2022, 7(2): 4606-4613.
[4] ZHAO Haoning, WANG Chaoqun, GUO Rui, et al. Autonomous live working robot navigation with real-time detection and motion planning system on distribution line[J]. High voltage, 2022, 7(6): 1204-1216.
[5] EBADI K, BERNREITER L, BIGGIE H, et al. Present and future of SLAM in extreme environments: The DARPA SubT challenge[J]. IEEE transactions on robotics, 2024, 40: 936-959.
[6] SINGH K J, KAPOOR D S, THAKUR K, et al. Map making in social indoor environment through robot navigation using active SLAM[J]. IEEE access, 2022, 10: 134455-134465.
[7] GRANDO R B, DE JESUS J C, KICH V A, et al. Double critic deep reinforcement learning for mapless 3D navigation of unmanned aerial vehicles[J]. Journal of intelligent & robotic systems, 2022, 104(2): 29.
[8] LI Hanxiao, LUO Biao, SONG Wei, et al. Predictive hierarchical reinforcement learning for path-efficient mapless navigation with moving target[J]. Neural networks, 2023, 165: 677-688.
[9] LIKHACHEV M, FERGUSON D I, GORDON G J, et al. Anytime dynamic A*: An anytime, replanning algorithm[C]//ICAPS. Monterey, 2005, 5: 262-271.
[10] KHATIB O. Real-time obstacle avoidance for manipulators and mobile robots[M]//Cox IJ, Wilfong GT. Autonomous Robot Vehicles. New York: Springer, 1986: 396-404.
[11] NASIR J, ISLAM F, MALIK U, et al. RRT*-SMART: a rapid convergence implementation of RRT[J]. International journal of advanced robotic systems, 2013, 10(7): 299.
[12] DURRANT-WHYTE H, BAILEY T. Simultaneous localization and mapping: part I[J]. IEEE robotics & automation magazine, 2006, 13(2): 99-110.
[13] ALHMIEDAT T, MAREI A M, MESSOUDI W, et al. A SLAM-based localization and navigation system for social robots: the pepper robot case[J]. Machines, 2023, 11(2): 158.
[14] TAHERI H, XIA Zhaochun. SLAM: definition and evolution[J]. Engineering applications of artificial intelligence, 2021, 97: 104032.
[15] CHAPLOT D S, GANDHI D, GUPTA S, et al. Learning to explore using active neural SLAM[EB/OL]. (2020-04-10)[2021-01-01]. http://arxiv.org/abs/2004.05155.
[16] XUE Honghu, HEIN B, BAKR M, et al. Using deep reinforcement learning with automatic curriculum learning for mapless navigation in intralogistics[J]. Applied sciences, 2022, 12(6): 3153.
[17] HAN Yiheng, ZHAN I H, ZHAO Wang, et al. Deep reinforcement learning for robot collision avoidance with self-state-attention and sensor fusion[J]. IEEE robotics and automation letters, 2022, 7(3): 6886-6893.
[18] 欧阳勇平, 魏长赟, 蔡帛良. 动态环境下分布式异构多机器人避障方法研究[J]. 智能系统学报, 2022, 17(4): 752-763.
OUYANG Yongping, WEI Changyun, CAI Boliang. Collision avoidance approach for distributed heterogeneous multirobot systems in dynamic environments[J]. CAAI transactions on intelligent systems, 2022, 17(4): 752-763.
[19] ZHU Wei, HAYASHIBE M. A hierarchical deep reinforcement learning framework with high efficiency and generalization for fast and safe navigation[J]. IEEE transactions on industrial electronics, 2023, 70(5): 4962-4971.
[20] 李鹏, 阮晓钢, 朱晓庆, 等. 基于深度强化学习的区域化视觉导航方法[J]. 上海交通大学学报, 2021, 55(5): 575-585.
LI Peng, RUAN Xiaogang, ZHU Xiaoqing, et al. A regionalization vision navigation method based on deep reinforcement learning[J]. Journal of Shanghai Jiaotong University, 2021, 55(5): 575-585.
[21] PUTERMAN M L. Chapter 8 Markov decision processes[M]//Handbooks in Operations Research and Management Science. Amsterdam: Elsevier, 1990: 331-434.
[22] 刘克. 实用马尔可夫决策过程[M]. 北京: 清华大学出版社, 2004.
LIU Ke. Practical Markov decision processes[M]. Beijing: Tsinghua University Press, 2024.
[23] SILVER D, LEVER G, HEESS N, et al. Deterministic policy gradient algorithms[C]//International conference on machine learning. Beijing:ICML, 2014: 387-395.
[24] FUJIMOTO S, VAN HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[EB/OL]. (2018-02-26)[2021-01-01]. http://arxiv.org/abs/1802.09477.
[25] SCHAUL T, QUAN J, ANTONOGLOU I, et al. Prioritized experience replay[EB/OL]. (2015-11-18)[2021-01-01]. http://arxiv.org/abs/1511.05952.
[26] LI Peng, DING Xiangcheng, SUN Hongfang, et al. Research on dynamic path planning of mobile robot based on improved DDPG algorithm[J]. Mobile information systems, 2021, 2021: 5169460.
[27] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[EB/OL]. (2015-09-09)[2021-01-01]. http://arxiv.org/abs/1509.02971.
Similar References:

Memo

-

Last Update: 2024-11-05

Copyright © CAAI Transactions on Intelligent Systems