[1]赵泽华,梁美玉,薛哲,等.基于数据质量评估的高效强化联邦学习节点动态采样优化[J].智能系统学报,2024,19(6):1552-1561.[doi:10.11992/tis.202305054]
ZHAO Zehua,LIANG Meiyu,XUE Zhe,et al.Client dynamic sampling optimization of efficient reinforcement federated learning based on data quality assessment[J].CAAI Transactions on Intelligent Systems,2024,19(6):1552-1561.[doi:10.11992/tis.202305054]
点击复制
ZHAO Zehua,LIANG Meiyu,XUE Zhe,et al.Client dynamic sampling optimization of efficient reinforcement federated learning based on data quality assessment[J].CAAI Transactions on Intelligent Systems,2024,19(6):1552-1561.[doi:10.11992/tis.202305054]
基于数据质量评估的高效强化联邦学习节点动态采样优化
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第6期
页码:
1552-1561
栏目:
学术论文—人工智能基础
出版日期:
2024-12-05
- Title:
- Client dynamic sampling optimization of efficient reinforcement federated learning based on data quality assessment
- Keywords:
- federated learning; deep reinforcement learning; client dynamic sampling; contribution index; data quality; communication efficiency; Shapley value; model accuracy
- 分类号:
- TP295
- 摘要:
- 系统异构性和统计异构性的存在使得通信开销和通信效率成为联邦学习的关键瓶颈之一，在众多参与方中只选取一部分客户端执行模型更新和聚合可以有效降低通信开销，但是选择偏差和客户端上的数据质量分布不平衡对客户端采样方法提出了额外的挑战。为此，提出数据质量评估的高效强化联邦学习节点动态采样优化方法（client dynamic sampling optimization of efficient reinforcement federated learning based on data quality assessment, RQCS），该方法采用沙普利值的贡献指数评估客户端上的数据质量，基于深度强化学习模型，智能的动态选择具有高数据质量且能提高最终模型精度的客户端参与每一轮的联邦学习，以抵消数据质量分布不平衡引入的偏差，加速模型收敛并提高模型精度。在MNIST及CIFAR-10数据集上的实验表明，所提出算法与其他算法相比，在减少通信开销的同时进一步加快了收敛速度，同时在模型最终准确性上也有较好的性能。
- Abstract:
- Communication cost and efficiency are the key bottlenecks of federated learning due to the existence of system and statistical heterogeneities. Selecting only a subset of clients to perform model updates and aggregation can effectively reduce communication costs among numerous participants. However, biased selection and uneven distribution of data quality across clients pose additional challenges to client sampling methods. Therefore, this paper proposes a method for client dynamic sampling optimization in efficient reinforcement federated learning based on data quality assessment (RQCS) to address the aforementioned issues. This method evaluates data quality on clients using a contribution index based on the Shapley value and intelligently selects clients with high data quality for each round of federated learning. By leveraging reinforcement learning, the method aims to offset the bias introduced by uneven data quality distribution, accelerate model convergence, and improve model accuracy. Experiments on the MNIST and CIFAR-10 datasets show that the proposed algorithm not only reduces communication costs but also further accelerates convergence speed and achieves better performance in model accuracy compared to other algorithms.
- 参考文献/References:
-
[1] 林伟伟, 石方, 曾岚, 等. 联邦学习开源框架综述[J]. 计算机研究与发展, 2023, 60(7): 1551-1580.
LIN Weiwei, SHI Fang, ZENG Lan, et al. Survey of federated learning open-source frameworks[J]. Journal of computer research and development, 2023, 60(7): 1551-1580.
[2] 田家会, 吕锡香, 邹仁朋, 等. 一种联邦学习中的公平资源分配方案[J]. 计算机研究与发展, 2022, 59(6): 1240-1254.
TIAN Jiahui, LYU Xixiang, ZOU Renpeng, et al. A fair resource allocation scheme in federated learning[J]. Journal of computer research and development, 2022, 59(6): 1240-1254.
[3] LUO Bing, XIAO Wenli, WANG Shiqiang, et al. Tackling system and statistical heterogeneity for federated learning with adaptive client sampling[C]//IEEE INFOCOM 2022-IEEE Conference on Computer Communications. London: IEEE, 2022: 1739-1748.
[4] 李少波, 杨磊, 李传江, 等. 联邦学习概述: 技术、应用及未来[J]. 计算机集成制造系统, 2022, 28(7): 2119-2138.
LI Shaobo , YANG Lei , LI Chuangjiang, et al. Overview of federated learning: technology, applications and future[J]. Computer integrated manufacturing systems, 2022, 28(7): 2119-2138.
[5] 邱鑫源, 叶泽聪, 崔翛龙, 等. 联邦学习通信开销研究综述[J]. 计算机应用, 2022, 42(2): 333-342.
QIU Xinyuan, YE Zecong, CUI Xiaolong, et al. Survey of communication overhead of federated learning[J]. Journal of computer applications, 2022, 42(2): 333-342.
[6] KARIMIREDDY S P, KALE S, MOHRI M, et al. SCAFFOLD: stochastic controlled averaging for federated learning[EB/OL]. (2019-10-14)[2021-01-01]. http://arxiv.org/abs/1910.06378.
[7] SONG Tianshu, TONG Yongxin, WEI Shuyue. Profit allocation for federated learning[C]//2019 IEEE International Conference on Big Data. Los Angeles: IEEE, 2019: 2577-2586.
[8] MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[EB/OL]. (2016-02-17)[2021-01-01]. http://arxiv.org/abs/1602.05629.
[9] LI Tian, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[EB/OL]. (2018-12-14)[2021-01-01]. http://arxiv.org/abs/1812.06127.
[10] NISHIO T, YONETANI R. Client selection for federated learning with heterogeneous resources in mobile edge[C]//ICC 2019—2019 IEEE International Conference on Communications. Shanghai: IEEE, 2019: 1-7.
[11] CHO Y J, WANG Jianyu, JOSHI G. Client selection in federated learning: convergence analysis and power-of-choice selection strategies[EB/OL]. (2020-12-03)[2021-01-01]. http://arxiv.org/abs/2010.01243.
[12] FRABONI Y, VIDAL R, KAMENI L, et al. Clustered sampling: low-variance and improved representativity for clients selection in federated learning[EB/OL]. (2021-05-12)[2021-12-01]. http://arxiv.org/abs/2105.05883.
[13] ZHAO Yue, LI Meng, LAI Liangzhen, et al. Federated learning with non-IID data[EB/OL]. (2018-06-02)[2021-01-01]. http://arxiv.org/abs/1806.00582.
[14] KOLLIAS G, SALONIDIS T, WANG Shiqiang. Sketch to Skip and Select: communication efficient federated learning using locality sensitive hashing[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2023: 72-83.
[15] WANG Hao, KAPLAN Z, NIU Di, et al. Optimizing federated learning on non-IID data with reinforcement learning[C]//IEEE INFOCOM 2020-IEEE Conference on Computer Communications. Toronto: IEEE, 2020: 1698-1707.
[16] HUANG Tiansheng, LIN Weiwei, SHEN Li, et al. Stochastic client selection for federated learning with volatile clients[J]. IEEE internet of things journal, 2022, 9(20): 20055-20070.
[17] JI Shaoxiong, JIANG Wenqi, WALID A, et al. Dynamic sampling and selective masking for communication-efficient federated learning[EB/OL]. (2020-03-21)[2021-01-01]. http://arxiv.org/abs/2003.09603.
[18] RAI S, KUMARI A, PRASAD D K. Client selection in federated learning under imperfections in environment[J]. AI, 2022, 3(1): 124-145.
[19] LI Qinbin, HE Bingsheng, SONG D. Model-contrastive federated learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 10708-10717.
[20] WACHI A, SUI Yanan. Safe reinforcement learning in constrained Markov decision processes[EB/OL]. (2008-08-15)[2021-01-01]. http://arxiv.org/abs/2008.06626.
[21] HENDERSON P, ISLAM R, BACHMAN P, et al. Deep reinforcement learning that matters[C]//Proceedings of the AAAI Conference on Artificial Intelligence. AAAI , 2018.
[22] FENG Y, LI L, LIU Q. A kernel loss for solving the bellman equation[J]. Advances in neural information processing systems, 2019, 32.
[23] ZHANG Jie, GUO Song, QU Zhihao, et al. Adaptive federated learning on non-IID data with resource constraint[J]. IEEE transactions on computers, 2022, 71(7): 1655-1667.
[24] NICOLSON A, PALIWAL K K. Deep learning for minimum mean-square error approaches to speech enhancement[J]. Speech communication, 2019, 111: 44-55.
- 相似文献/References:
-
[1]周文吉,俞扬.分层强化学习综述[J].智能系统学报,2017,12(5):590.[doi:10.11992/tis.201706031]
ZHOU Wenji,YU Yang.Summarize of hierarchical reinforcement learning[J].CAAI Transactions on Intelligent Systems,2017,12():590.[doi:10.11992/tis.201706031]
[2]王作为,徐征,张汝波,等.记忆神经网络在机器人导航领域的应用与研究进展[J].智能系统学报,2020,15(5):835.[doi:10.11992/tis.202002020]
WANG Zuowei,XU Zheng,ZHANG Rubo,et al.Research progress and application of memory neural network in robot navigation[J].CAAI Transactions on Intelligent Systems,2020,15():835.[doi:10.11992/tis.202002020]
[3]杨瑞,严江鹏,李秀.强化学习稀疏奖励算法研究——理论与实验[J].智能系统学报,2020,15(5):888.[doi:10.11992/tis.202003031]
YANG Rui,YAN Jiangpeng,LI Xiu.Survey of sparse reward algorithms in reinforcement learning — theory and experiment[J].CAAI Transactions on Intelligent Systems,2020,15():888.[doi:10.11992/tis.202003031]
[4]王健宗,肖京,朱星华,等.联邦推荐系统的协同过滤冷启动解决方法[J].智能系统学报,2021,16(1):178.[doi:10.11992/tis.202009032]
WANG Jianzong,XIAO Jing,ZHU Xinghua,et al.Cold starts in collaborative filtering for federated recommender systems[J].CAAI Transactions on Intelligent Systems,2021,16():178.[doi:10.11992/tis.202009032]
[5]赵玉新,杜登辉,成小会,等.基于强化学习的海洋移动观测网络观测路径规划方法[J].智能系统学报,2022,17(1):192.[doi:10.11992/tis.202106004]
ZHAO Yuxin,DU Denghui,CHENG Xiaohui,et al.Path planning for mobile ocean observation network based on reinforcement learning[J].CAAI Transactions on Intelligent Systems,2022,17():192.[doi:10.11992/tis.202106004]
[6]窦勇敢,袁晓彤.基于隐式随机梯度下降优化的联邦学习[J].智能系统学报,2022,17(3):488.[doi:10.11992/tis.202106029]
DOU Yonggan,YUAN Xiaotong.Federated learning with implicit stochastic gradient descent optimization[J].CAAI Transactions on Intelligent Systems,2022,17():488.[doi:10.11992/tis.202106029]
[7]欧阳勇平,魏长赟,蔡帛良.动态环境下分布式异构多机器人避障方法研究[J].智能系统学报,2022,17(4):752.[doi:10.11992/tis.202106044]
OUYANG Yongping,WEI Changyun,CAI Boliang.Collision avoidance approach for distributed heterogeneous multirobot systems in dynamic environments[J].CAAI Transactions on Intelligent Systems,2022,17():752.[doi:10.11992/tis.202106044]
[8]王竣禾,姜勇.基于深度强化学习的动态装配算法[J].智能系统学报,2023,18(1):2.[doi:10.11992/tis.202201006]
WANG Junhe,JIANG Yong.Dynamic assembly algorithm based on deep reinforcement learning[J].CAAI Transactions on Intelligent Systems,2023,18():2.[doi:10.11992/tis.202201006]
[9]陶鑫钰,王艳,纪志成.基于深度强化学习的节能工艺路线发现方法[J].智能系统学报,2023,18(1):23.[doi:10.11992/tis.202112030]
TAO Xinyu,WANG Yan,JI Zhicheng.Energy-saving process route discovery method based on deep reinforcement learning[J].CAAI Transactions on Intelligent Systems,2023,18():23.[doi:10.11992/tis.202112030]
[10]张钰欣,赵恩娇,赵玉新.规则耦合下的多异构子网络MADDPG博弈对抗算法[J].智能系统学报,2024,19(1):190.[doi:10.11992/tis.202303037]
ZHANG Yuxin,ZHAO Enjiao,ZHAO Yuxin.MADDPG game confrontation algorithm of polyisomer network based on rule coupling based on rule coupling[J].CAAI Transactions on Intelligent Systems,2024,19():190.[doi:10.11992/tis.202303037]
备注/Memo
收稿日期:2023-5-31。
基金项目:国家自然科学基金项目（62192784，U22B2038，62172056，62272058）；中国人工智能学会–华为MindSpore学术奖励基金项目(CAAIXSJLJJ-2021-007B).
作者简介:赵泽华，硕士，主要研究方向为联邦学习中的高效通信。E-mail：zehuazhao@bupt.edu.cn;梁美玉，教授，博士生导师，主要研究方向为人工智能、数据挖掘、多媒体信息处理、计算机视觉。主持和参与国家自然科学基金重大项目、国家重点研发计划项目、973计划课题、国家自然科学基金重点项目/重大国际合作项目/面上项目/青年科学基金等科研项目。发表学术论文100余篇，出版学术专著3部，申请和授权发明专利40余项。E-mail：meiyu1210@bupt.edu.cn;薛哲，副教授。主要研究方向为人工智能、机器学习、数据挖掘、多媒体信息处理。E-mail：xuezhe@bupt.edu.cn。
通讯作者:梁美玉. E-mail：meiyu1210@bupt.edu.cn
更新日期/Last Update:
2024-11-05