[1]ZHANG Zeyu,WANG Lei,CAI Jingcao,et al.Application analysis of an enhanced Q-learning genetic algorithm in path planning[J].CAAI Transactions on Intelligent Systems,2025,20(6):1493-1504.[doi:10.11992/tis.202504016]
Copy

Application analysis of an enhanced Q-learning genetic algorithm in path planning

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 6 Page number: 1493-1504 Column: 学术论文—机器人 Public date: 2025-11-05
Title:
Application analysis of an enhanced Q-learning genetic algorithm in path planning
Author(s):
ZHANG Zeyu1;  WANG Lei1;  CAI Jingcao1;  XIA Qiangqiang2
1. School of Mechanical and Automotive Engineering, Anhui Polytechnic University, Wuhu 241000, China;
2. Yangtze River Delta Hart Robot Industry Technology Research Institute, Wuhu 241000, China
Keywords:
path planning; genetic algorithm; population initialization; simulated annealing algorithm; Q-learning algorithm; fitness function; selective crossover and mutation; elitism
CLC:
TP391
DOI:
10.11992/tis.202504016
Abstract:
An improved genetic algorithm is proposed to address the limitations of conventional genetic algorithms in path planning, such as excessive turning angles, redundant turns, and susceptibility to local optima. First, an upgraded population initialization strategy is proposed. This method selects a transitional point to generate a route from the starting point to the transition point and another from the transition point to the endpoint. The segments are then combined to establish a complete route, thereby forming a high-quality initial population and improving early search efficiency. Second, an enhanced tournament selection approach is utilized, which integrates simulated annealing with region-based population segmentation to increase population diversity and mitigate local optima entrapment. Finally, the Q-learning algorithm is incorporated into crossover and mutation operations, enabling the algorithm to interact with the environment, continuously learn, and optimize its action selection strategy. This integration enhances the algorithm’s global search capabilities, leading to the development of superior populations. As demonstrated by path planning simulations, the proposed enhanced genetic algorithm has the capacity to reduce path length and turning angles, decrease the number of turns, and ultimately identify more optimal pathways when compared to traditional genetic algorithms, enhanced adaptive genetic algorithms, and improved catastrophe genetic algorithms.
References:
[1] 林韩熙, 向丹, 欧阳剑, 等. 移动机器人路径规划算法的研究综述[J]. 计算机工程与应用, 2021, 57(18): 38-48.
LIN Hanxi, XIANG Dan, OUYANG Jian, et al. Review of path planning algorithms for mobile robots[J]. Computer engineering and applications, 2021, 57(18): 38-48.
[2] 王梓强, 胡晓光, 李晓筱, 等. 移动机器人全局路径规划算法综述[J]. 计算机科学, 2021, 48(10): 19-29.
WANG Ziqiang, HU Xiaoguang, LI Xiaoxiao, et al. Overview of global path planning algorithms for mobile robots[J]. Computer science, 2021, 48(10): 19-29.
[3] BADAMASI A M, KABIR I K, AHMED G, et al. Autonomous mobile robot path planning techniques: a review: classical and heuristic techniques[J]. IEEE access, 2025, 13: 117999-118022.
[4] 李成进, 王芳. 智能移动机器人导航控制技术综述[J]. 导航定位与授时, 2016, 3(5): 22-26.
LI Chengjin, WANG Fang. Review of navigation control technology of intelligent mobile robot[J]. Navigation positioning and timing, 2016, 3(5): 22-26.
[5] LIU Lixing, WANG Xu, YANG Xin, et al. Path planning techniques for mobile robots: review and prospect[J]. Expert systems with applications, 2023, 227: 120254
[6] TANG Yuexia, ZAKARIA M A, YOUNAS M. Path planning trends for autonomous mobile robot navigation: a review[J]. Sensors, 2025, 25(4): 1206.
[7] 赵晓, 王铮, 黄程侃, 等. 基于改进A*算法的移动机器人路径规划[J]. 机器人, 2018, 40(6): 903-910.
ZHAO Xiao, WANG Zheng, HUANG Chengkan, et al. Mobile robot path planning based on an improved A* algorithm[J]. Robot, 2018, 40(6): 903-910.
[8] SHENG Zhaokang, SONG Tingqiang, SONG Jiale, et al. Bidirectional rapidly exploring random tree path planning algorithm based on adaptive strategies and artificial potential fields[J]. Engineering applications of artificial intelligence, 2025, 148: 110393.
[9] 曲胜, 许志远, 张晓鹏, 等. 基于改进RRT算法的无人船路径规划研究[J]. 中国航海, 2024, 47(4): 175-180.
QU Sheng, XU Zhiyuan, ZHANG Xiaopeng, et al. Research on unmanned ship path planning based on improved RRT algorithm[J]. Navigation of China, 2024, 47(4): 175-180.
[10] XIAO Qianxi, CAI Jiejin. The path-planning in radioactive environment based on HIOSD-PRM method[J]. Annals of nuclear energy, 2022, 171: 109018.
[11] 王豪, 赵学军, 袁修久. 基于改进自适应遗传算法的机器人路径规划[J]. 电光与控制, 20, 29(5): 72-76.
WANG Hao, ZHAO Xuejun, YUAN Xiujiu. Robot path planning based on improved adaptive genetic algorithm [J]. Electric light & control, 20, 29(5): 72-76.
[12] MIAO Changwei, CHEN Guangzhu, YAN Chengliang, et al. Path planning optimization of indoor mobile robot based on adaptive ant colony algorithm[J]. Computers & industrial engineering, 2021, 156: 107230.
[13] 卫玉梁, 靳伍银. 基于神经网络Q-learning算法的智能车路径规划[J]. 火力与指挥控制, 2019, 44(2): 46-49.
WEI Yuliang, JIN Wuyin. Intelligent vehicle path planning based on neural network Q-learning algorithm[J]. Fire control & command control, 2019, 44(2): 46-49.
[14] 刘清云, 游雄, 张欣, 等. 移动机器人路径规划算法综述[J]. 计算机科学, 2025, 52(S1): 159-168.
LIU Qingyun, YOU Xiong, ZHANG Xin, et al. Overview of path planning algorithms for mobile robots[J]. Computer science, 2025, 52(S1): 159-168.
[15] QIN Hongwei, SHAO Shiliang, WANG Ting, et al. Review of autonomous path planning algorithms for mobile Robots[J]. Drones, 20, 7(3): 211.
[16] UGWOKE K C, NNANNA N A, ABDULLAHI S E. Simulation-based review of classical, heuristic, and metaheuristic path planning algorithms[J]. Scientific reports, 2025, 15(1): 12643.
[17] 白晓兰, 袁铮, 周文全, 等. 混合遗传算法在机器人路径规划中的应用[J]. 组合机床与自动化加工技术, 2023(11): 15-19.
BAI Xiaolan, YUAN Zheng, ZHOU Wenquan, et al. Application of hybrid genetic algorithm in robot path planning[J]. Modular machine tool & automatic manufacturing technique, 2023(11): 15-19.
[18] HAO Kun, ZHAO Jiale, YU Kaicheng, et al. Path planning of mobile robots based on a multi-population migration genetic algorithm[J]. Sensors, 2020, 20(20): 5873.
[19] 田雅琴, 胡梦辉, 刘文涛, 等. 基于跳点搜索-遗传算法的自主移动机器人路径规划[J]. 工程设计学报, 20, 30(6): 697-706.
TIAN Yaqin, HU Menghui, LIU Wentao, et al. Autonomous mobile robot path planning based on jump point search-genetic algorithm [J]. Journal of engineering design, 20, 30(6): 697-706.
[20] CHEN Ziming, YAN Jinjin, HUANG Ruen, et al. Path planning for autonomous underwater vehicles (AUVs) considering the influences and constraints of ocean currents[J]. Drones, 2024, 8(8): 348.
[21] 黄荣杰, 王亚刚. 基于可视图与改进遗传算法的机器人平滑路径规划[J]. 控制工程, 2024, 31(4): 678-686.
HUANG Rongjie, WANG Yagang. Smooth path planning for robot based on visibility graph and improved genetic algorithm[J]. Control engineering of China, 2024, 31(4): 678-686.
[22] DING Hui. Motion path planning of soccer training auxiliary robot based on genetic algorithm in fixed-point rotation environment[J]. Journal of ambient intelligence and humanized computing, 2020, 11(12): 6261-6270.
[23] 李艳生, 万勇, 张毅, 等. 基于人工蜂群-自适应遗传算法的仓储机器人路径规划[J]. 仪器仪表学报, 20, 43(4): 282-290.
LI Yansheng, WAN Yong, ZHANG Yi, et al. Warehouse robot path planning based on artificial bee colony-adaptive genetic algorithm [J]. Journal of instrumentation and measurement, 20, 43(4): 282-290.
[24] 蔡劲草, 王雷, 雷德明. 基于蛙跳算法的分布式装配混合流水车间调度[J]. 华中科技大学学报(自然科学版), 20, 51(12): 37-44.
CAI Jincao, WANG Lei, LEI Deming. Distributed assembly hybrid flow shop scheduling based on frog-leaping algorithm [J]. Journal of Huazhong University of Science and Technology (natural science edition), 20, 51(12): 37-44.
[25] 江涛, 张志安, 程志, 等. 改进遗传算法与领航跟随法的机器人编队方法[J]. 计算机工程与应用, 2020, 56(3): 240-245.
JIANG Tao, ZHANG Zhian, CHENG Zhi, et al. Robot formation method with improved genetic algorithm and leader-follower[J]. Computer engineering and applications, 2020, 56(3): 240-245.
[26] 汤云峰, 赵静, 谢非, 等. 基于改进遗传算法的机器人路径规划方法[J]. 南京师范大学学报(工程技术版), 2021, 21(3): 49-55.
TANG Yunfeng, ZHAO Jing, XIE Fei, et al. Robot path planning method based on improved genetic algorithm[J]. Journal of Nanjing Normal University (engineering and technology edition), 2021, 21(3): 49-55.
[27] FISTER I, FISTER J, YANG Xinshe, et al. A comprehensive review of firefly algorithms[J]. Swarm and evolutionary computation, 2013, 13: 34-36.
[28] ABDELAZIZ A, MEKHAMER S, BADR M, et al. The firefly metaheuristic algorithms: developments and applications[J]. International electrical engineering journal, 2015, 7(13): 1945-195.
[29] 魏书鑫, 王群京, 李国丽, 等. 萤火虫算法结合遗传算法的移动机器人路径规划[J]. 制造业自动化, 2024, 46(10): 69-82.
WEI Shuxin, WANG Qunjing, LI Guoli, et al. Firefly algorithm combined with genetic algorithm for mobile robot path planning[J]. Manufacturing automation, 2024, 46(10): 69-82.
[30] 王雷, 李明. 改进自适应遗传算法在移动机器人路径规划中的应用[J]. 南京理工大学学报, 2017, 41(5): 627-633.
WANG Lei, LI Ming. Application of improved adaptive genetic algorithm in mobile robot path planning[J]. Journal of Nanjing University of Science and Technology, 2017, 41(5): 627-633.
[31] 徐兴, 俞旭阳, 赵芸, 等. 基于改进遗传算法的移动机器人全局路径规划[J]. 计算机集成制造系统, 2022, 28(6): 1659-1672.
XU Xing, YU Xuyang, ZHAO Yun, et al. Global path planning of mobile robot based on improved genetic algorithm[J]. Computer integrated manufacturing systems, 2022, 28(6): 1659-1672.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems