[1]WEN Jiayan,WANG Yibo,XIN Huajian,et al.Intelligent connected vehicle path planning based on optimized deep Q-network[J].CAAI Transactions on Intelligent Systems,2026,21(1):226-235.[doi:10.11992/tis.202502010]
Copy

Intelligent connected vehicle path planning based on optimized deep Q-network

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 21 Number of periods: 2026 1 Page number: 226-235 Column: 吴文俊人工智能科学技术奖论坛 Public date: 2026-03-05
Title:
Intelligent connected vehicle path planning based on optimized deep Q-network
Author(s):
WEN Jiayan1; 2;  WANG Yibo1; 2;  XIN Huajian3;  XIE Guangming4
1. School of Automation, Guangxi University of Science and Technology, Liuzhou 545616, China;
2. The Research Center for Intelligent Cooperation and Cross-application, Guangxi University of Science and Technology, Liuzhou 545616, China;
3. Guangxi Vocational and Technical College of Industry, Nanning 530001, China;
4. College of Engineering, Peking University, Beijing 100871, China
Keywords:
intelligent connected vehicles; path planning; unstructured environment; attention mechanism; experience replay; obstacle avoidance; deep Q-network; deep reinforcement learning
CLC:
TP183；TP2
DOI:
10.11992/tis.202502010
Abstract:
Aiming at the path planning problem of intelligent connected vehicles in unstructured environment, the traditional deep Q-network (DQN) algorithm has problems such as low planning efficiency, slow convergence speed, poor generalization, etc. This paper proposes a DQN planning method combining attention mechanism and empirical classification. The experience playback pool is designed by combining the attention mechanism, and the multi-objective optimization conflict is solved by dynamic weight allocation, so as to improve the experience utilization rate in similar environments, reduce the planning time, and accelerate the convergence; Build non sparse reward constraints, and optimize the state space in combination with the characteristics of the traffic environment, so as to adapt to multi-objective scenarios and achieve multi scenario generalization. The simulation shows that the average planning speed of the optimized algorithm is increased by 28.6%, and the travel distance is shortened by 25.2% compared with that before optimization. In addition, the time for the first successful planning is shortened by 32.8% by loading training data in different scenarios.
References:
[1] 杨龙海 , 车婷婷 , 熊月程 , 等. 考虑智能网联车队要素的交通震荡特性研究[J]. 北京交通大学学报, 2024, 48(4): 104-114 YANG Longhai, CHE Tingting, XIONG Yuecheng, et al. Research on the characteristics of traffic oscillations considering the elements of connected and automated vehicle platoon[J]. Journal of Beijing Jiaotong University, 2024, 48(4): 104-114
[2] ZHANG E, MASOUD N. V2XSim: A V2X simulator for connected and automated vehicle environment simulation[C]//2020 IEEE 23rd International Conference on Intelligent Transportation Systems. Rhodes: IEEE, 2020: 1-6.
[3] 马庆禄, 李美强, 黄光浩, 等. 智能网联汽车超车路径规划方法[J]. 控制理论与应用, 2024, 41(10): 1882-1898 MA Qinglu, LI Meiqiang, HUANG Ghuanghao, et al. Overtaking path planning method for intelligent connected vehicle[J]. Control theory and technology, 2024, 41(10): 1882-1898
[4] 虞立斌, 张亿, 黄磊, 等. 双向A*路径规划算法的邻域改进方法研究[J]. 小型微型计算机系统, 2025, 46(6): 1312-1318 YU Libin, ZHANG Yi, HUANG Lei, et al. Research on neighborhood improvement based on two-way A* path planning algorithm[J]. Journal of Chinese computer systems, 2025, 46(6): 1312-1318
[5] 梅艺林, 崔立堃, 胡雪岩. 基于人工势场法的无人车路径规划与避障研究[J]. 兵器装备工程学报, 2024, 45(9): 300-306 MEI Yilin, CUI Likun, HU Xueyan. Research on path planning and obstacle avoidance of unmanned vehicle based on artificial potential field method[J]. Journal of ordnance equipment engineering, 2024, 45(9): 300-306
[6] 谢春丽, 陶天艺. 基于混合A*算法的移动机器人路径规划研究[J]. 南京信息工程大学学报, 2025, 17(3): 340-351 XIE Chunli, TAO Tianyi. Research on path planning of mobile robots based on hybrid A* algorithm[J]. Journal of Nanjing University of Information Science and Technology, 2025, 17(3): 340-351
[7] 于逸然, 赖惠成, 高古学, 等. 基于遗传算法和A*算法的多农机协同作业优化方法[J]. 系统仿真学报, 2025, 37(9): 2397-2408 YU Yiran, LAI Huicheng, GAO Guxue, et al. Optimization method for multi agricultural machinery collaborative operation based on genetic algorithm and A~(*) algorithm[J]. Journal of system simulation, 2025, 37(9): 2397-2408
[8] 杨国, 吴晓, 肖如奇, 等. 改进A*算法的安全高效室内全局路径规划[J]. 电子测量与仪器学报, 2024, 38(7): 131-142 YANG Guo, WU Xiao, XIAO Ruqi, et al. Improved A* algorithm for secure and efficient indoor global path planning[J]. Journal of electronic measurement and instrumentation, 2024, 38(7): 131-142
[9] LIU Chenguang, MAO Qingzhou, CHU Xiumin, et al. An improved A-star algorithm considering water current, traffic separation and berthing for vessel path planning[J]. Applied sciences, 2019, 9(6): 1057
[10] 赵晓, 王铮, 黄程侃, 等. 基于改进A*算法的移动机器 人路径规划[J]. 机器人, 2018, 40(6): 903-910 ZHAO Xiao, WANG Zheng, HUANG Chengkan, et al. Mobile robot path planning based on an improved A* algorithm[J]. Robot, 2018, 40(6): 903-910
[11] WANG Zhongshan, LI Peiqing, WANG Zhiwei, et al. APG-RRT: sampling-based path planning method for small autonomous vehicle in closed scenarios[J]. IEEE access, 2024, 12: 25731-25739
[12] SHI Yangyang, LI Qiongqiong, BU Shengqiang, et al. Research on intelligent vehicle path planning based on rapidly-exploring random tree[J]. Mathematical problems in engineering, 2020, 2020(1): 5910503
[13] 郭利进, 李强. 基于改进RRT*算法的移动机器人路径规划[J]. 智能系统学报, 2024, 19(5): 1209-1217 GUO Lijin, LI Qiang. Path planning of mobile robots based on improved RRT* algorithm[J]. CAAI transactions on intelligent systems, 2024, 19(5): 1209-1217
[14] 陈旭飞, 胡耀炜, 丛培龙, 等. 面向路径规划的双向交互多步蚁群算法研究[J]. 计算机工程与应用, 2025, 61(3): 166-176 CHEN Xufei, HU Yaowei, CONG Peilong, et al. Research on bidirectional interactive multi step ant colony algorithm for path planning[J]. Computer engineering and applications, 2025, 61(3): 166-176
[15] 郑琰, 席宽, 巴文婷, 等. 基于蚁群-动态窗口法的无人驾驶汽车动态路径规划[J]. 南京信息工程大学学报, 2025(2): 256-264 ZHENG Yan, XI Kuan, BA Wenting, et al. Dynamic path planning for autonomous vehicles based on ant colony dynamic window method[J]. Journal of Nanjing University of Information Science and Technology, 2025(2): 256-264
[16] 蒲兴成, 冼文杰, 聂壮. 基于改进蚁群优化算法的AUV三维路径规划[J]. 智能系统学报, 2024, 19(3): 627-634 PU Xingcheng, XIAN Wenjie, NIE Zhuang. Three-dimensional path planning of AUV based on improved ant colony optimization algorithm[J]. CAAI transactions on intelligent systems, 2024, 19(3): 627-634
[17] 张志文, 刘伯威, 张继园, 等. 麻雀搜索算法-粒子群算法与快速扩展随机树算法协同优化的智能车辆路径规划[J]. 中国机械工程, 2024, 35(6): 993-999,1009 ZHANG Zhiwen, LIU Baiwei, ZHANG Jiyuan, et al. Cooperative optimization of intelligent vehicle path planning based on PSO-SSA and RRT[J]. China mechanical engineering, 2024, 35(6): 993-999,1009
[18] 谢金燕, 刘丽星, 杨欣, 等. 改进粒子群优化算法的果园割草机作业路径规划[J]. 中国农业大学学报, 2023, 28(11): 182-191 XIE Jinyan, LIU Lixing, YANG Xin, et al. Orchard lawn mower operation path planning based on improved particle swarm optimization algorithm[J]. Journal of China Agricultural University, 2023, 28(11): 182-191
[19] 王飞, 杨清平. 基于改进粒子群算法的城市物流无人机路径规划[J]. 科学技术与工程, 2023, 23(30): 13187-13194 WANG Fei, YANG Qingping. Route planning of urban logistics UAV based on improved particle swarm optimization algorithm[J]. Science technology and engineering, 2023, 23(30): 13187-13194
[20] WATKINS C J C H, DAYAN P. Q-learning[J]. Machine learning, 1992, 8: 279-292
[21] SUTTON R S, BARTO A G. Reinforcement Learning: An Introduction[M]. 2nd ed. Cambridge: MIT Press, 2018.
[22] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Playing atari with deep reinforcement learning[EB/OL]. (2013-12-19)[2025-02-24]. https://arxiv.org/abs/1312.5602.
[23] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533
[24] 夏雨奇, 黄炎焱, 陈恰. 基于深度Q网络的无人车侦察路径规划[J]. 系统工程与电子技术, 2024, 46(9): 3070-3081 XIA Yuqi, HUANG Yanyan, CHEN Qia. Path planning for unmanned vehicle reconnaissance based on deep Q-network[J]. Systems engineering and electronics, 2024, 46(9): 3070-3081
[25] 李宗刚, 韩森, 陈引娟, 等. 基于角度搜索和深度Q网络的移动机器人路径规划算法[J]. 兵工学报, 2025, 46(2): 30-44 LI Zonggang, HAN Sen, CHEN Yinjuan, et al. Mobile robots path planning algorithm based on angle searching and deep Q-network[J]. Acta armamentarii, 2025, 46(2): 30-44
[26] SNIDER J M. Automatic steering methods for autonomous automobile path tracking[EB/OL]. (2009-12-30)[2025-02-24]. https://api.semanticscholar.org.
Similar References:

Memo

-

Last Update: 2026-01-05

Copyright © CAAI Transactions on Intelligent Systems