[1]文家燕,王怡博,辛华健,等.基于改进深度Q网络的智能网联汽车路径规划[J].智能系统学报,2026,21(1):226-235.[doi:10.11992/tis.202502010]
WEN Jiayan,WANG Yibo,XIN Huajian,et al.Intelligent connected vehicle path planning based on optimized deep Q-network[J].CAAI Transactions on Intelligent Systems,2026,21(1):226-235.[doi:10.11992/tis.202502010]
点击复制
WEN Jiayan,WANG Yibo,XIN Huajian,et al.Intelligent connected vehicle path planning based on optimized deep Q-network[J].CAAI Transactions on Intelligent Systems,2026,21(1):226-235.[doi:10.11992/tis.202502010]
基于改进深度Q网络的智能网联汽车路径规划
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第1期
页码:
226-235
栏目:
吴文俊人工智能科学技术奖论坛
出版日期:
2026-03-05
- Title:
- Intelligent connected vehicle path planning based on optimized deep Q-network
- Keywords:
- intelligent connected vehicles; path planning; unstructured environment; attention mechanism; experience replay; obstacle avoidance; deep Q-network; deep reinforcement learning
- 分类号:
- TP183；TP2
- 摘要:
- 针对非结构环境中的智能网联汽车路径规划问题，传统的深度Q网络(deep Q-network，DQN)算法存在规划效率低、收敛速度慢、泛化性差等问题，本文提出了一种结合注意力机制和经验分类的DQN规划方法。通过结合注意力机制设计经验回放池，通过动态权重分配解决多目标优化冲突，提升相似环境中的经验利用率，降低规划时间，加快收敛；构建非稀疏奖励约束，结合交通环境特性优化状态空间，以便适应多目标场景和实现多场景泛化。仿真表明，优化后的算法平均规划速度提升了28.6%，行进路程较优化前缩短了25.2%，且在不同场景下通过载入训练数据，首次规划成功的耗时缩短了32.8%。
- Abstract:
- Aiming at the path planning problem of intelligent connected vehicles in unstructured environment, the traditional deep Q-network (DQN) algorithm has problems such as low planning efficiency, slow convergence speed, poor generalization, etc. This paper proposes a DQN planning method combining attention mechanism and empirical classification. The experience playback pool is designed by combining the attention mechanism, and the multi-objective optimization conflict is solved by dynamic weight allocation, so as to improve the experience utilization rate in similar environments, reduce the planning time, and accelerate the convergence; Build non sparse reward constraints, and optimize the state space in combination with the characteristics of the traffic environment, so as to adapt to multi-objective scenarios and achieve multi scenario generalization. The simulation shows that the average planning speed of the optimized algorithm is increased by 28.6%, and the travel distance is shortened by 25.2% compared with that before optimization. In addition, the time for the first successful planning is shortened by 32.8% by loading training data in different scenarios.
- 参考文献/References:
-
[1] 杨龙海 , 车婷婷 , 熊月程 , 等. 考虑智能网联车队要素的交通震荡特性研究[J]. 北京交通大学学报, 2024, 48(4): 104-114 YANG Longhai, CHE Tingting, XIONG Yuecheng, et al. Research on the characteristics of traffic oscillations considering the elements of connected and automated vehicle platoon[J]. Journal of Beijing Jiaotong University, 2024, 48(4): 104-114
[2] ZHANG E, MASOUD N. V2XSim: A V2X simulator for connected and automated vehicle environment simulation[C]//2020 IEEE 23rd International Conference on Intelligent Transportation Systems. Rhodes: IEEE, 2020: 1-6.
[3] 马庆禄, 李美强, 黄光浩, 等. 智能网联汽车超车路径规划方法[J]. 控制理论与应用, 2024, 41(10): 1882-1898 MA Qinglu, LI Meiqiang, HUANG Ghuanghao, et al. Overtaking path planning method for intelligent connected vehicle[J]. Control theory and technology, 2024, 41(10): 1882-1898
[4] 虞立斌, 张亿, 黄磊, 等. 双向A*路径规划算法的邻域改进方法研究[J]. 小型微型计算机系统, 2025, 46(6): 1312-1318 YU Libin, ZHANG Yi, HUANG Lei, et al. Research on neighborhood improvement based on two-way A* path planning algorithm[J]. Journal of Chinese computer systems, 2025, 46(6): 1312-1318
[5] 梅艺林, 崔立堃, 胡雪岩. 基于人工势场法的无人车路径规划与避障研究[J]. 兵器装备工程学报, 2024, 45(9): 300-306 MEI Yilin, CUI Likun, HU Xueyan. Research on path planning and obstacle avoidance of unmanned vehicle based on artificial potential field method[J]. Journal of ordnance equipment engineering, 2024, 45(9): 300-306
[6] 谢春丽, 陶天艺. 基于混合A*算法的移动机器人路径规划研究[J]. 南京信息工程大学学报, 2025, 17(3): 340-351 XIE Chunli, TAO Tianyi. Research on path planning of mobile robots based on hybrid A* algorithm[J]. Journal of Nanjing University of Information Science and Technology, 2025, 17(3): 340-351
[7] 于逸然, 赖惠成, 高古学, 等. 基于遗传算法和A*算法的多农机协同作业优化方法[J]. 系统仿真学报, 2025, 37(9): 2397-2408 YU Yiran, LAI Huicheng, GAO Guxue, et al. Optimization method for multi agricultural machinery collaborative operation based on genetic algorithm and A~(*) algorithm[J]. Journal of system simulation, 2025, 37(9): 2397-2408
[8] 杨国, 吴晓, 肖如奇, 等. 改进A*算法的安全高效室内全局路径规划[J]. 电子测量与仪器学报, 2024, 38(7): 131-142 YANG Guo, WU Xiao, XIAO Ruqi, et al. Improved A* algorithm for secure and efficient indoor global path planning[J]. Journal of electronic measurement and instrumentation, 2024, 38(7): 131-142
[9] LIU Chenguang, MAO Qingzhou, CHU Xiumin, et al. An improved A-star algorithm considering water current, traffic separation and berthing for vessel path planning[J]. Applied sciences, 2019, 9(6): 1057
[10] 赵晓, 王铮, 黄程侃, 等. 基于改进A*算法的移动机器 人路径规划[J]. 机器人, 2018, 40(6): 903-910 ZHAO Xiao, WANG Zheng, HUANG Chengkan, et al. Mobile robot path planning based on an improved A* algorithm[J]. Robot, 2018, 40(6): 903-910
[11] WANG Zhongshan, LI Peiqing, WANG Zhiwei, et al. APG-RRT: sampling-based path planning method for small autonomous vehicle in closed scenarios[J]. IEEE access, 2024, 12: 25731-25739
[12] SHI Yangyang, LI Qiongqiong, BU Shengqiang, et al. Research on intelligent vehicle path planning based on rapidly-exploring random tree[J]. Mathematical problems in engineering, 2020, 2020(1): 5910503
[13] 郭利进, 李强. 基于改进RRT*算法的移动机器人路径规划[J]. 智能系统学报, 2024, 19(5): 1209-1217 GUO Lijin, LI Qiang. Path planning of mobile robots based on improved RRT* algorithm[J]. CAAI transactions on intelligent systems, 2024, 19(5): 1209-1217
[14] 陈旭飞, 胡耀炜, 丛培龙, 等. 面向路径规划的双向交互多步蚁群算法研究[J]. 计算机工程与应用, 2025, 61(3): 166-176 CHEN Xufei, HU Yaowei, CONG Peilong, et al. Research on bidirectional interactive multi step ant colony algorithm for path planning[J]. Computer engineering and applications, 2025, 61(3): 166-176
[15] 郑琰, 席宽, 巴文婷, 等. 基于蚁群-动态窗口法的无人驾驶汽车动态路径规划[J]. 南京信息工程大学学报, 2025(2): 256-264 ZHENG Yan, XI Kuan, BA Wenting, et al. Dynamic path planning for autonomous vehicles based on ant colony dynamic window method[J]. Journal of Nanjing University of Information Science and Technology, 2025(2): 256-264
[16] 蒲兴成, 冼文杰, 聂壮. 基于改进蚁群优化算法的AUV三维路径规划[J]. 智能系统学报, 2024, 19(3): 627-634 PU Xingcheng, XIAN Wenjie, NIE Zhuang. Three-dimensional path planning of AUV based on improved ant colony optimization algorithm[J]. CAAI transactions on intelligent systems, 2024, 19(3): 627-634
[17] 张志文, 刘伯威, 张继园, 等. 麻雀搜索算法-粒子群算法与快速扩展随机树算法协同优化的智能车辆路径规划[J]. 中国机械工程, 2024, 35(6): 993-999,1009 ZHANG Zhiwen, LIU Baiwei, ZHANG Jiyuan, et al. Cooperative optimization of intelligent vehicle path planning based on PSO-SSA and RRT[J]. China mechanical engineering, 2024, 35(6): 993-999,1009
[18] 谢金燕, 刘丽星, 杨欣, 等. 改进粒子群优化算法的果园割草机作业路径规划[J]. 中国农业大学学报, 2023, 28(11): 182-191 XIE Jinyan, LIU Lixing, YANG Xin, et al. Orchard lawn mower operation path planning based on improved particle swarm optimization algorithm[J]. Journal of China Agricultural University, 2023, 28(11): 182-191
[19] 王飞, 杨清平. 基于改进粒子群算法的城市物流无人机路径规划[J]. 科学技术与工程, 2023, 23(30): 13187-13194 WANG Fei, YANG Qingping. Route planning of urban logistics UAV based on improved particle swarm optimization algorithm[J]. Science technology and engineering, 2023, 23(30): 13187-13194
[20] WATKINS C J C H, DAYAN P. Q-learning[J]. Machine learning, 1992, 8: 279-292
[21] SUTTON R S, BARTO A G. Reinforcement Learning: An Introduction[M]. 2nd ed. Cambridge: MIT Press, 2018.
[22] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Playing atari with deep reinforcement learning[EB/OL]. (2013-12-19)[2025-02-24]. https://arxiv.org/abs/1312.5602.
[23] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533
[24] 夏雨奇, 黄炎焱, 陈恰. 基于深度Q网络的无人车侦察路径规划[J]. 系统工程与电子技术, 2024, 46(9): 3070-3081 XIA Yuqi, HUANG Yanyan, CHEN Qia. Path planning for unmanned vehicle reconnaissance based on deep Q-network[J]. Systems engineering and electronics, 2024, 46(9): 3070-3081
[25] 李宗刚, 韩森, 陈引娟, 等. 基于角度搜索和深度Q网络的移动机器人路径规划算法[J]. 兵工学报, 2025, 46(2): 30-44 LI Zonggang, HAN Sen, CHEN Yinjuan, et al. Mobile robots path planning algorithm based on angle searching and deep Q-network[J]. Acta armamentarii, 2025, 46(2): 30-44
[26] SNIDER J M. Automatic steering methods for autonomous automobile path tracking[EB/OL]. (2009-12-30)[2025-02-24]. https://api.semanticscholar.org.
- 相似文献/References:
-
[1]黄彦文,曹其新.RoboCup比赛环境下足球机器人路径规划研究[J].智能系统学报,2007,2(4):52.
HUANG Yan-wen,CAO Qin-xin.Path planning for robot soccer in the RoboCup environment[J].CAAI Transactions on Intelligent Systems,2007,2():52.
[2]秦世引,高书征.面向救援任务的地面移动机器人路径规划[J].智能系统学报,2009,4(5):414.[doi:10.3969/j.issn.1673-4785.2009.05.005]
QIN Shi-yin,GAO Shu-zhen.Path planning for mobile rescue robots in disaster areas with complex environments[J].CAAI Transactions on Intelligent Systems,2009,4():414.[doi:10.3969/j.issn.1673-4785.2009.05.005]
[3]曹卫华,吴净斌,吴 敏,等.无路标环境下遥操作机器人SLAM系统[J].智能系统学报,2010,5(3):240.
CAO Wei-hua,WU Jing-bin,WU Min,et al.A system for telerobotics in environments without landmarks[J].CAAI Transactions on Intelligent Systems,2010,5():240.
[4]薛英花,田国会,吴 皓,等.智能空间中的服务机器人路径规划[J].智能系统学报,2010,5(3):260.
XUE Ying-hua,TIAN Guo-hui,WU Hao,et al.Path planning for service robots in an intelligent space[J].CAAI Transactions on Intelligent Systems,2010,5():260.
[5]黄晓丹,王粉花,王志良.情感决策的智能家居虚拟人路径规划[J].智能系统学报,2010,5(4):292.
HUANG Xiao-dan,WANG Fen-hua,WANG Zhi-liang.Using affective decisionmaking for the path planning of virtual humans in a smart home[J].CAAI Transactions on Intelligent Systems,2010,5():292.
[6]唐小勇,于 飞,潘洪悦.改进粒子群算法的潜器导航规划[J].智能系统学报,2010,5(5):443.[doi:10.3969/j.issn.1673-4785.2010.05.011]
TANG Xiao-yong,YU Fei,PAN Hong-yue.Submersible path-planning based on an improved PSO[J].CAAI Transactions on Intelligent Systems,2010,5():443.[doi:10.3969/j.issn.1673-4785.2010.05.011]
[7]夏琳琳,张健沛,初妍.计算智能在移动机器人路径规划中的应用综述[J].智能系统学报,2011,6(2):160.
XIA Linlin,ZHANG Jianpei,CHU Yan.An application survey on computational intelligence for path planning of mobile robots[J].CAAI Transactions on Intelligent Systems,2011,6():160.
[8]蒲兴成,张军,张毅.基于神经网络的改进行为协调控制及其在智能轮椅路径规划中的应用[J].智能系统学报,2011,6(5):456.
PU Xingcheng,ZHANG Jun,ZHANG Yi.Modified behavior coordination for intelligent wheelchair path planning based on a neural network[J].CAAI Transactions on Intelligent Systems,2011,6():456.
[9]肖国宝,严宣辉.一种基于改进Theta *的机器人路径规划算法[J].智能系统学报,2013,8(1):58.[doi:10.3969/j.issn.1673-4785.201208032]
XIAO Guobao,YAN Xuanhui.A path planning algorithm based on improved Theta * for mobile robot[J].CAAI Transactions on Intelligent Systems,2013,8():58.[doi:10.3969/j.issn.1673-4785.201208032]
[10]杨茂,田彦涛.复杂环境下多机器人觅食路径规划与控制[J].智能系统学报,2013,8(2):162.[doi:10.3969/j.issn.1673-4785.201208022]
YANG Mao,TIAN Yantao.Foraging path planning and control for multi-robot in complex environment[J].CAAI Transactions on Intelligent Systems,2013,8():162.[doi:10.3969/j.issn.1673-4785.201208022]
备注/Memo
收稿日期:2025-2-24。
基金项目:国家自然科学基金(62541306，619630060); 广西科技重大专项 (桂科 AA24206054).
作者简介:文家燕，教授，博士生导师，中国自动化学会青年工作委员会委员。主要研究方向为多智能体系统协同控制、智能网联汽车队列控制。现主持国家自然科学基金及省部级基金项目 8 项，获专利授权 10 项，发表学术论文35篇。E-mail：wenjiayan2012@126.com。;辛华健，副教授，中国仿真学会机器人专委会委员，主持完成了广西职业教学改革重点项目1项，广西教育科学规划课题重点项目1项，广西中青年教师科研项目2项。发表学术论文20余篇，主编教材2部。E-mail：13659619535@163.com。;谢广明，教授，博士生导师，主要研究方向为智能仿生机器人、复杂系统与多机器人控制和水下特种机器人技术，作为核心负责人主持多项国家自然科学基金重点项目、面上项目等国家级科研课题，获发明专利授权10余项，获国家自然科学奖二等奖、教育部自然科学奖一等奖、吴文俊人工智能科学技术创新奖二等奖，发表学术论文200余篇。E-mail：xiegming@pku.edu.cn。
通讯作者:辛华健. E-mail：13659619535@163.com
更新日期/Last Update:
2026-01-05