[1]张泽宇,王雷,蔡劲草,等.改进Q-learning遗传算法在路径规划中的应用研究[J].智能系统学报,2025,20(6):1493-1504.[doi:10.11992/tis.202504016]
ZHANG Zeyu,WANG Lei,CAI Jingcao,et al.Application analysis of an enhanced Q-learning genetic algorithm in path planning[J].CAAI Transactions on Intelligent Systems,2025,20(6):1493-1504.[doi:10.11992/tis.202504016]
点击复制
ZHANG Zeyu,WANG Lei,CAI Jingcao,et al.Application analysis of an enhanced Q-learning genetic algorithm in path planning[J].CAAI Transactions on Intelligent Systems,2025,20(6):1493-1504.[doi:10.11992/tis.202504016]
改进Q-learning遗传算法在路径规划中的应用研究
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
20
期数:
2025年第6期
页码:
1493-1504
栏目:
学术论文—机器人
出版日期:
2025-11-05
- Title:
- Application analysis of an enhanced Q-learning genetic algorithm in path planning
- Keywords:
- path planning; genetic algorithm; population initialization; simulated annealing algorithm; Q-learning algorithm; fitness function; selective crossover and mutation; elitism
- 分类号:
- TP391
- 摘要:
- 针对传统遗传算法在路径规划中存在转向角度过大、转向次数过多、易陷入局部最优等问题，提出一种改进遗传算法。首先，提出一种改进种群初始化策略，即先确定一个过渡点，生成一条从起点到过渡点的路径和一条从过渡点到终点的路径，再将两条路径首尾相连成一条从起点到终点的路径，以生成优秀初始种群，提高前期搜索效率；其次，采用模拟退火算法与区域划分种群相结合的改进锦标赛选择策略，增加种群多样性，防止陷入局部最优；最后，设计一种Q-learning算法与交叉和变异相结合的策略，通过与环境交互，不断学习并优化动作选择策略以此提高算法的全局搜索能力，得到更优种群。路径规划仿真结果表明：相比传统遗传算法、改进自适应遗传算法和改进灾变遗传算法，本文所提改进遗传算法能减少路径长度和转向角度，降低转向次数，从而搜索到更优的路径。
- Abstract:
- An improved genetic algorithm is proposed to address the limitations of conventional genetic algorithms in path planning, such as excessive turning angles, redundant turns, and susceptibility to local optima. First, an upgraded population initialization strategy is proposed. This method selects a transitional point to generate a route from the starting point to the transition point and another from the transition point to the endpoint. The segments are then combined to establish a complete route, thereby forming a high-quality initial population and improving early search efficiency. Second, an enhanced tournament selection approach is utilized, which integrates simulated annealing with region-based population segmentation to increase population diversity and mitigate local optima entrapment. Finally, the Q-learning algorithm is incorporated into crossover and mutation operations, enabling the algorithm to interact with the environment, continuously learn, and optimize its action selection strategy. This integration enhances the algorithm’s global search capabilities, leading to the development of superior populations. As demonstrated by path planning simulations, the proposed enhanced genetic algorithm has the capacity to reduce path length and turning angles, decrease the number of turns, and ultimately identify more optimal pathways when compared to traditional genetic algorithms, enhanced adaptive genetic algorithms, and improved catastrophe genetic algorithms.
- 参考文献/References:
-
[1] 林韩熙, 向丹, 欧阳剑, 等. 移动机器人路径规划算法的研究综述[J]. 计算机工程与应用, 2021, 57(18): 38-48.
LIN Hanxi, XIANG Dan, OUYANG Jian, et al. Review of path planning algorithms for mobile robots[J]. Computer engineering and applications, 2021, 57(18): 38-48.
[2] 王梓强, 胡晓光, 李晓筱, 等. 移动机器人全局路径规划算法综述[J]. 计算机科学, 2021, 48(10): 19-29.
WANG Ziqiang, HU Xiaoguang, LI Xiaoxiao, et al. Overview of global path planning algorithms for mobile robots[J]. Computer science, 2021, 48(10): 19-29.
[3] BADAMASI A M, KABIR I K, AHMED G, et al. Autonomous mobile robot path planning techniques: a review: classical and heuristic techniques[J]. IEEE access, 2025, 13: 117999-118022.
[4] 李成进, 王芳. 智能移动机器人导航控制技术综述[J]. 导航定位与授时, 2016, 3(5): 22-26.
LI Chengjin, WANG Fang. Review of navigation control technology of intelligent mobile robot[J]. Navigation positioning and timing, 2016, 3(5): 22-26.
[5] LIU Lixing, WANG Xu, YANG Xin, et al. Path planning techniques for mobile robots: review and prospect[J]. Expert systems with applications, 2023, 227: 120254
[6] TANG Yuexia, ZAKARIA M A, YOUNAS M. Path planning trends for autonomous mobile robot navigation: a review[J]. Sensors, 2025, 25(4): 1206.
[7] 赵晓, 王铮, 黄程侃, 等. 基于改进A*算法的移动机器人路径规划[J]. 机器人, 2018, 40(6): 903-910.
ZHAO Xiao, WANG Zheng, HUANG Chengkan, et al. Mobile robot path planning based on an improved A* algorithm[J]. Robot, 2018, 40(6): 903-910.
[8] SHENG Zhaokang, SONG Tingqiang, SONG Jiale, et al. Bidirectional rapidly exploring random tree path planning algorithm based on adaptive strategies and artificial potential fields[J]. Engineering applications of artificial intelligence, 2025, 148: 110393.
[9] 曲胜, 许志远, 张晓鹏, 等. 基于改进RRT算法的无人船路径规划研究[J]. 中国航海, 2024, 47(4): 175-180.
QU Sheng, XU Zhiyuan, ZHANG Xiaopeng, et al. Research on unmanned ship path planning based on improved RRT algorithm[J]. Navigation of China, 2024, 47(4): 175-180.
[10] XIAO Qianxi, CAI Jiejin. The path-planning in radioactive environment based on HIOSD-PRM method[J]. Annals of nuclear energy, 2022, 171: 109018.
[11] 王豪, 赵学军, 袁修久. 基于改进自适应遗传算法的机器人路径规划[J]. 电光与控制, 20, 29(5): 72-76.
WANG Hao, ZHAO Xuejun, YUAN Xiujiu. Robot path planning based on improved adaptive genetic algorithm [J]. Electric light & control, 20, 29(5): 72-76.
[12] MIAO Changwei, CHEN Guangzhu, YAN Chengliang, et al. Path planning optimization of indoor mobile robot based on adaptive ant colony algorithm[J]. Computers & industrial engineering, 2021, 156: 107230.
[13] 卫玉梁, 靳伍银. 基于神经网络Q-learning算法的智能车路径规划[J]. 火力与指挥控制, 2019, 44(2): 46-49.
WEI Yuliang, JIN Wuyin. Intelligent vehicle path planning based on neural network Q-learning algorithm[J]. Fire control & command control, 2019, 44(2): 46-49.
[14] 刘清云, 游雄, 张欣, 等. 移动机器人路径规划算法综述[J]. 计算机科学, 2025, 52(S1): 159-168.
LIU Qingyun, YOU Xiong, ZHANG Xin, et al. Overview of path planning algorithms for mobile robots[J]. Computer science, 2025, 52(S1): 159-168.
[15] QIN Hongwei, SHAO Shiliang, WANG Ting, et al. Review of autonomous path planning algorithms for mobile Robots[J]. Drones, 20, 7(3): 211.
[16] UGWOKE K C, NNANNA N A, ABDULLAHI S E. Simulation-based review of classical, heuristic, and metaheuristic path planning algorithms[J]. Scientific reports, 2025, 15(1): 12643.
[17] 白晓兰, 袁铮, 周文全, 等. 混合遗传算法在机器人路径规划中的应用[J]. 组合机床与自动化加工技术, 2023(11): 15-19.
BAI Xiaolan, YUAN Zheng, ZHOU Wenquan, et al. Application of hybrid genetic algorithm in robot path planning[J]. Modular machine tool & automatic manufacturing technique, 2023(11): 15-19.
[18] HAO Kun, ZHAO Jiale, YU Kaicheng, et al. Path planning of mobile robots based on a multi-population migration genetic algorithm[J]. Sensors, 2020, 20(20): 5873.
[19] 田雅琴, 胡梦辉, 刘文涛, 等. 基于跳点搜索-遗传算法的自主移动机器人路径规划[J]. 工程设计学报, 20, 30(6): 697-706.
TIAN Yaqin, HU Menghui, LIU Wentao, et al. Autonomous mobile robot path planning based on jump point search-genetic algorithm [J]. Journal of engineering design, 20, 30(6): 697-706.
[20] CHEN Ziming, YAN Jinjin, HUANG Ruen, et al. Path planning for autonomous underwater vehicles (AUVs) considering the influences and constraints of ocean currents[J]. Drones, 2024, 8(8): 348.
[21] 黄荣杰, 王亚刚. 基于可视图与改进遗传算法的机器人平滑路径规划[J]. 控制工程, 2024, 31(4): 678-686.
HUANG Rongjie, WANG Yagang. Smooth path planning for robot based on visibility graph and improved genetic algorithm[J]. Control engineering of China, 2024, 31(4): 678-686.
[22] DING Hui. Motion path planning of soccer training auxiliary robot based on genetic algorithm in fixed-point rotation environment[J]. Journal of ambient intelligence and humanized computing, 2020, 11(12): 6261-6270.
[23] 李艳生, 万勇, 张毅, 等. 基于人工蜂群-自适应遗传算法的仓储机器人路径规划[J]. 仪器仪表学报, 20, 43(4): 282-290.
LI Yansheng, WAN Yong, ZHANG Yi, et al. Warehouse robot path planning based on artificial bee colony-adaptive genetic algorithm [J]. Journal of instrumentation and measurement, 20, 43(4): 282-290.
[24] 蔡劲草, 王雷, 雷德明. 基于蛙跳算法的分布式装配混合流水车间调度[J]. 华中科技大学学报(自然科学版), 20, 51(12): 37-44.
CAI Jincao, WANG Lei, LEI Deming. Distributed assembly hybrid flow shop scheduling based on frog-leaping algorithm [J]. Journal of Huazhong University of Science and Technology (natural science edition), 20, 51(12): 37-44.
[25] 江涛, 张志安, 程志, 等. 改进遗传算法与领航跟随法的机器人编队方法[J]. 计算机工程与应用, 2020, 56(3): 240-245.
JIANG Tao, ZHANG Zhian, CHENG Zhi, et al. Robot formation method with improved genetic algorithm and leader-follower[J]. Computer engineering and applications, 2020, 56(3): 240-245.
[26] 汤云峰, 赵静, 谢非, 等. 基于改进遗传算法的机器人路径规划方法[J]. 南京师范大学学报(工程技术版), 2021, 21(3): 49-55.
TANG Yunfeng, ZHAO Jing, XIE Fei, et al. Robot path planning method based on improved genetic algorithm[J]. Journal of Nanjing Normal University (engineering and technology edition), 2021, 21(3): 49-55.
[27] FISTER I, FISTER J, YANG Xinshe, et al. A comprehensive review of firefly algorithms[J]. Swarm and evolutionary computation, 2013, 13: 34-36.
[28] ABDELAZIZ A, MEKHAMER S, BADR M, et al. The firefly metaheuristic algorithms: developments and applications[J]. International electrical engineering journal, 2015, 7(13): 1945-195.
[29] 魏书鑫, 王群京, 李国丽, 等. 萤火虫算法结合遗传算法的移动机器人路径规划[J]. 制造业自动化, 2024, 46(10): 69-82.
WEI Shuxin, WANG Qunjing, LI Guoli, et al. Firefly algorithm combined with genetic algorithm for mobile robot path planning[J]. Manufacturing automation, 2024, 46(10): 69-82.
[30] 王雷, 李明. 改进自适应遗传算法在移动机器人路径规划中的应用[J]. 南京理工大学学报, 2017, 41(5): 627-633.
WANG Lei, LI Ming. Application of improved adaptive genetic algorithm in mobile robot path planning[J]. Journal of Nanjing University of Science and Technology, 2017, 41(5): 627-633.
[31] 徐兴, 俞旭阳, 赵芸, 等. 基于改进遗传算法的移动机器人全局路径规划[J]. 计算机集成制造系统, 2022, 28(6): 1659-1672.
XU Xing, YU Xuyang, ZHAO Yun, et al. Global path planning of mobile robot based on improved genetic algorithm[J]. Computer integrated manufacturing systems, 2022, 28(6): 1659-1672.
- 相似文献/References:
-
[1]周本达,陈明华.随机化均匀设计混合遗传算法求解图的二划分问题[J].智能系统学报,2009,4(1):91.
ZHOU Ben-da,CHEN Ming-hua.Solving the 2-way graph partitioning problem using a genetic algorithm based on randomized uniform design[J].CAAI Transactions on Intelligent Systems,2009,4():91.
[2]黄彦文,曹其新.RoboCup比赛环境下足球机器人路径规划研究[J].智能系统学报,2007,2(4):52.
HUANG Yan-wen,CAO Qin-xin.Path planning for robot soccer in the RoboCup environment[J].CAAI Transactions on Intelligent Systems,2007,2():52.
[3]康 琦,汪 镭,刘小莉,等.基于群体智能框架理念的遗传算法总体模式描述[J].智能系统学报,2007,2(5):42.
KANG Qi,WANG Lei,LIU Xiao-li,et al.General mode description genetic algorithms based on a framework of swarm intelligence[J].CAAI Transactions on Intelligent Systems,2007,2():42.
[4]马 炫,张亚龙.基于遗传算法的大规模矩形件优化排样[J].智能系统学报,2007,2(5):48.
MA Xuan,ZHANG Ya-long.A genetic algorithm for the layout of large scale rectang ular parts[J].CAAI Transactions on Intelligent Systems,2007,2():48.
[5]徐 雄.人工情感的进化控制系统实现[J].智能系统学报,2008,3(2):135.
XU Xiong.Implementation of an evolutionary control system based on artificial emotion[J].CAAI Transactions on Intelligent Systems,2008,3():135.
[6]刘 胜,李高云,孙天英.一种基于种群多样度的实数编码并行遗传算法[J].智能系统学报,2008,3(5):423.
L IU Sheng,L I Gao-yun,SUN Tian-ying.A real coding parallel genetic algorithm based on diversity of population[J].CAAI Transactions on Intelligent Systems,2008,3():423.
[7]张 涛,费树岷,李晓东.基于GARBF神经网络及边界不变特征的车辆识别[J].智能系统学报,2009,4(3):278.
ZHANG Tao,FEI Shu-min,LI Xiao-dong.Vehicle recognition using boundary invariants and a genetic algorithm trained radial basis function neural network[J].CAAI Transactions on Intelligent Systems,2009,4():278.
[8]周树德,孙增圻.遗传算法中的联结关系[J].智能系统学报,2009,4(6):483.[doi:10.3969/j.issn.1673-4785.2009.06.003]
ZHOU Shu-de,SUN Zeng-qi.Linkage in genetic algorithms[J].CAAI Transactions on Intelligent Systems,2009,4():483.[doi:10.3969/j.issn.1673-4785.2009.06.003]
[9]程显毅,巩向普.改进的模糊C-均值算法在医学图像分割中的应用[J].智能系统学报,2010,5(1):80.
CHENG Xian-yi,GONG Xiang-pu.An improved fuzzy Cmeans algorithm for segmentation of medical images[J].CAAI Transactions on Intelligent Systems,2010,5():80.
[10]曹卫华,吴净斌,吴 敏,等.无路标环境下遥操作机器人SLAM系统[J].智能系统学报,2010,5(3):240.
CAO Wei-hua,WU Jing-bin,WU Min,et al.A system for telerobotics in environments without landmarks[J].CAAI Transactions on Intelligent Systems,2010,5():240.
[11]秦世引,高书征.面向救援任务的地面移动机器人路径规划[J].智能系统学报,2009,4(5):414.[doi:10.3969/j.issn.1673-4785.2009.05.005]
QIN Shi-yin,GAO Shu-zhen.Path planning for mobile rescue robots in disaster areas with complex environments[J].CAAI Transactions on Intelligent Systems,2009,4():414.[doi:10.3969/j.issn.1673-4785.2009.05.005]
[12]夏琳琳,张健沛,初妍.计算智能在移动机器人路径规划中的应用综述[J].智能系统学报,2011,6(2):160.
XIA Linlin,ZHANG Jianpei,CHU Yan.An application survey on computational intelligence for path planning of mobile robots[J].CAAI Transactions on Intelligent Systems,2011,6():160.
[13]王玉卓,闵华松.基于毛笔建模的机器人书法系统[J].智能系统学报,2021,16(4):707.[doi:10.11992/tis.202006033]
WANG Yuzhuo,MIN Huasong.Robot calligraphy system based on brush modeling[J].CAAI Transactions on Intelligent Systems,2021,16():707.[doi:10.11992/tis.202006033]
[14]蔡军,钟志远.改进蚁群算法的送餐机器人路径规划[J].智能系统学报,2024,19(2):370.[doi:10.11992/tis.202205056]
CAI Jun,ZHONG Zhiyuan.Path planning of a meal delivery robot based on an improved ant colony algorithm[J].CAAI Transactions on Intelligent Systems,2024,19():370.[doi:10.11992/tis.202205056]
备注/Memo
收稿日期:2025-4-16。
基金项目:安徽省高校优秀拔尖人才培育项目（gxbjZD2022023）；安徽省高校自然科学研究重点项目（2023AH050935）；安徽省机器视觉检测与感知技术重点实验室开放基金项目（KLMVI-2024-HIT-15）.
作者简介:张泽宇，硕士研究生，主要研究方向为路径规划。E-mail：3473182516@qq.com。;王雷，教授，博士，主要研究方向为智能优化算法在制造系统中的应用。发表学术论文50余篇，获发明专利授权30余项。E-mail：wangdalei2000@126.com。;蔡劲草，讲师，博士，主要研究方向为智能优化算法在车间调度等中的应用。发表学术论文10余篇。E-mail：caijingcao@foxmail.com。
通讯作者:王雷. E-mail：wangdalei2000@126.com
更新日期/Last Update:
1900-01-01