[1]陈智雄,詹学滋,左嘉烁.基于深度强化学习的电力线与无线双模通信MAC层接入算法[J].智能系统学报,2025,20(2):344-354.[doi:10.11992/tis.202312023]
CHEN Zhixiong,ZHAN Xuezi,ZUO Jiashuo.Adaptive MAC layer access algorithm for power line and wireless dual-mode communication based on deep reinforcement learning[J].CAAI Transactions on Intelligent Systems,2025,20(2):344-354.[doi:10.11992/tis.202312023]
点击复制
CHEN Zhixiong,ZHAN Xuezi,ZUO Jiashuo.Adaptive MAC layer access algorithm for power line and wireless dual-mode communication based on deep reinforcement learning[J].CAAI Transactions on Intelligent Systems,2025,20(2):344-354.[doi:10.11992/tis.202312023]
基于深度强化学习的电力线与无线双模通信MAC层接入算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
20
期数:
2025年第2期
页码:
344-354
栏目:
学术论文—机器学习
出版日期:
2025-03-05
- Title:
- Adaptive MAC layer access algorithm for power line and wireless dual-mode communication based on deep reinforcement learning
- Keywords:
- power line communication; wireless communication; dual-mode nodes; deep reinforcement learning; double deep Q-network; MAC layer access; fairness utility function; P-persistent access
- 分类号:
- TM721
- 摘要:
- 针对无线和电力线通信混合组网的信道竞争接入问题，提出了一种基于深度强化学习的电力线与无线双模通信的MAC接入算法。双模节点根据网络广播信息和信道使用等数据自适应接入双媒质信道。首先建立了基于双模通信网络交互和统计信息的双模通信节点数据采集模型；接着定义了基于协作信息的深度强化学习(deep reinforcement learning, DRL)状态空间、动作空间和奖励，设计了联合α-公平效用函数和P坚持接入机制的节点决策流程，实现基于双深度Q网络(double deep Q-network, DDQN)的双模节点自适应接入算法；最后进行算法性能仿真和对比分析。仿真结果表明，提出的接入算法能够在保证双模网络和信道接入公平性的条件下，有效提高双模通信节点的接入性能。
- Abstract:
- Aiming to address the issue of channel competition in hybrid networks of PLC and WC, this study proposes a MAC access algorithm based on deep reinforcement learning for dual-mode communication over power lines and wireless channels. Dual-mode nodes adaptively access the dual-medium channel based on data such as network broadcast information and channel usage. First, a dual-mode node data collection model is established based on interactions and statistical information from dual-mode communication networks. Then, the DRL state space, action space, and rewards are defined based on collaborative information, and an adaptive access algorithm is developed using a dual deep Q-network. This algorithm incorporates a node decision-making process that combines the α-fairness utility function with the P-persistent access mechanism. Finally, simulations and comparative analyses of the algorithm’s performance are performed. Simulation results show that the proposed access algorithm effectively improves the access performance of dual-mode communication nodes while ensuring fairness in dual-mode network and channel access.
- 参考文献/References:
-
[1] 谢可, 王剑锋, 金尧, 等. 电力物联网关键技术研究综述[J]. 电力信息与通信技术, 2022, 20(1): 1-12.
XIE Ke, WANG Jianfeng, JIN Yao, et al. Review of key technologies for power Internet of things[J]. Electric power information and communication technology, 2022, 20(1): 1-12.
[2] 张丽, 郝佳恺. 5G网络切片电力通信业务与测试技术研究[J]. 电力信息与通信技术, 2022, 20(5): 74-79.
ZHANG Li, HAO Jiakai. Research on 5G network slicing technology in power communication service and testing[J]. Electric power information and communication technology, 2022, 20(5): 74-79.
[3] 杨璐, 吴清亮. 无线接入网络自适应公平调度算法[J]. 通信学报, 2012, 33(1): 102-106,113.
YANG Lu, WU Qingliang. Self-adaptive fair scheduling algorithm in wireless network[J]. Journal on communications, 2012, 33(1): 102-106,113.
[4] YOON S G, JANG S, KIM Y H, et al. Opportunistic routing for smart grid with power line communication access networks[J]. IEEE transactions on smart grid, 2014, 5(1): 303-311.
[5] PAPADOPOULOS T A, KALOUDAS C G, CHRYSOCHOS A I, et al. Application of narrowband power-line communication in medium-voltage smart distribution grids[J]. IEEE transactions on power delivery, 2013, 28(2): 981-988.
[6] 蒲红红, 刘晓胜, 韩铭, 等. 电力线通信信道下协作非正交多址接入系统的分布式机会中继选择[J]. 电工技术学报, 2020, 35(11): 2306-2318.
PU Honghong, LIU Xiaosheng, HAN Ming, et al. Distributed opportunistic relay selection for cooperative non-orthogonal multiple access power line communication networks[J]. Transactions of China electrotechnical society, 2020, 35(11): 2306-2318.
[7] MATHUR A, BHATNAGAR M R, AI Yun, et al. Performance analysis of a dual-hop wireless-power line mixed cooperative system[J]. IEEE access, 2018, 6: 34380-34392.
[8] 陈智雄, 韩东升, 邱丽君. 室内无线和电力线双媒质协作通信系统性能研究[J]. 中国电机工程学报, 2017, 37(9): 2589-2599.
CHEN Zhixiong, HAN Dongsheng, QIU Lijun. Research on the performance of cooperation communication system based on indoor double media with wireless channel and power line[J]. Proceedings of the CSEE, 2017, 37(9): 2589-2599.
[9] ZHU Yihua, XU Mengying. Enhancing network throughput via the equal interval frame aggregation scheme for IEEE 802.11ax WLANs[J]. Chinese journal of electronics, 2023, 32(4): 747-759.
[10] VLACHOU C, BANCHS A, SALVADOR P, et al. Analysis and enhancement of CSMA/CA with deferral in power-line communications[J]. IEEE journal on selected areas in communications, 2016, 34(7): 1978-1991.
[11] LIU Ran, YUAN Jinsha, CHEN Zhixiong, et al. Research on the MAC layer performance of wireless and power line parallel communication[C]//The 8th International Conference on Computer Engineering and Networks. Shanghai: Springer, 2018: 729-737.
[12] CHEN Zhixiong, ZHI Leixin, ZHAO Xiongwen, et al. Parallel access scheme for wireless and power line dual-connection communication[J]. IEEE transactions on vehicular technology, 2022, 71(7): 7997-8001.
[13] CHEN Zhixiong, ZHI Leixin, CHEN Peiru, et al. An MAC layer algorithm based on power line-wireless dual media channels and multiplexing[J]. China communications, 2022, 19(10): 78-91.
[14] 孙志军, 薛磊, 许阳明, 等. 深度学习研究综述[J]. 计算机应用研究, 2012, 29(8): 2806-2810.
SUN Zhijun, XUE Lei, XU Yangming, et al. Overview of deep learning[J]. Application research of computers, 2012, 29(8): 2806-2810.
[15] 周文吉, 俞扬. 分层强化学习综述[J]. 智能系统学报, 2017, 12(5): 590-594.
ZHOU Wenji, YU Yang. Summarize of hierarchical reinforcement learning[J]. CAAI transactions on intelligent systems, 2017, 12(5): 590-594.
[16] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.
[17] NGUYEN T T, NGUYEN N D, NAHAVANDI S. Deep reinforcement learning for multiagent systems: a review of challenges, solutions, and applications[J]. IEEE transactions on cybernetics, 2020, 50(9): 3826-3839.
[18] 赵晓荣. 基于深度强化学习的异构网络资源分配[D]. 济南: 山东大学, 2023.
ZHAO Xiaorong. Resource allocation of heterogeneous networks based on deep reinforcement learning[D]. Jinan: Shandong University, 2023.
[19] 姜华, 杨家伟, 黄巍, 等. 基于深度强化学习的D2D辅助MEC网络资源分配算法[J]. 电力信息与通信技术, 2023, 21(7): 51-58.
JIANG Hua, YANG Jiawei, HUANG Wei, et al. A D2D-assisted MEC network resource allocation algorithm based on deep reinforcement learning[J]. Electric power information and communication technology, 2023, 21(7): 51-58.
[20] 刘乔寿, 周雄, 刘爽, 等. 基于深度强化学习的OFDM自适应导频设计[J]. 通信学报, 2023, 44(9): 104-114.
LIU Qiaoshou, ZHOU Xiong, LIU Shuang, et al. Adaptive pilot design for OFDM based on deep reinforcement learning[J]. Journal on communications, 2023, 44(9): 104-114.
[21] 赵玉新, 杜登辉, 成小会, 等. 基于强化学习的海洋移动观测网络观测路径规划方法[J]. 智能系统学报, 2022, 17(1): 192-200.
ZHAO Yuxin, DU Denghui, CHENG Xiaohui, et al. Path planning for mobile ocean observation network based on reinforcement learning[J]. CAAI transactions on intelligent systems, 2022, 17(1): 192-200.
[22] 何云, 申敏, 王蕊, 等. 无蜂窝毫米波大规模MIMO系统基于深度强化学习的节能睡眠策略[J]. 电子学报, 2023, 51(10): 2831-2843.
HE Yun, SHEN Min, WANG Rui, et al. Energy-efficient sleep-mode based on deep reinforcement learning for cell-free mmwave massive MIMO systems[J]. Acta electronica sinica, 2023, 51(10): 2831-2843.
[23] PEI Zhonghui, CHEN Wei, DU Luyao, et al. MAC contention protocol based on reinforcement learning for IoV communication environments[C]//2021 IEEE 6th International Conference on Computer and Communication Systems. Chengdu: IEEE, 2021: 546-552.
[24] WU C M, KAO Y C, CHANG Kaifu, et al. A Q-learning-based adaptive MAC protocol for Internet of Things networks[J]. IEEE access, 2021, 9: 128905-128918.
[25] NAPARSTEK O, COHEN K. Deep multi-user reinforcement learning for distributed dynamic spectrum access[J]. IEEE transactions on wireless communications, 2019, 18(1): 310-323.
[26] YU Yiding, LIEW S C, WANG Taotao. Carrier-sense multiple access for heterogeneous wireless networks using deep reinforcement learning[C]//2019 IEEE Wireless Communications and Networking Conference Workshop. Marrakech: IEEE, 2019: 1-7.
[27] YU Yiding, LIEW S C, WANG Taotao. Non-uniform time-step deep Q-network for carrier-sense multiple access in heterogeneous wireless networks[J]. IEEE transactions on mobile computing, 2021, 20(9): 2848-2861.
[28] YE Xiaowen, YU Yiding, FU Liqun. Deep reinforcement learning based MAC protocol for underwater acoustic networks[J]. IEEE transactions on mobile computing, 2022, 21(5): 1625-1638.
[29] TONELLO A M, LETIZIA N A, RIGHINI D, et al. Machine learning tips and tricks for power line communications[J]. IEEE access, 2019, 7: 82434-82452.
[30] BIANCHI G. Performance analysis of the IEEE 802.11 distributed coordination function[J]. IEEE journal on selected areas in communications, 2000, 18(3): 535-547.
备注/Memo
收稿日期:2023-12-16。
基金项目:国家自然科学基金青年基金项目(61601182)；中央高校科研业务费专项资金项目(2023MS113).
作者简介:陈智雄，副教授，主要研究方向为电力物联网、电力线通信。主持国家自然科学基金项目、河北省自然科学基金项目等10余项，获得国家发明专利授权6项。E-mail：zxchen@ncepu.edu.cn;詹学滋，硕士研究生，主要研究方向为电力线通信和无线通信。E-mail：15659630390@163.com;左嘉烁，硕士研究生,主要研究方向为电力线通信和无线通信。E-mail：1032888158@qq.com。
通讯作者:陈智雄. E-mail：zxchen@ncepu.edu.cn
更新日期/Last Update:
2025-03-05