[1]ZHAO Zehua,LIANG Meiyu,XUE Zhe,et al.Client dynamic sampling optimization of efficient reinforcement federated learning based on data quality assessment[J].CAAI Transactions on Intelligent Systems,2024,19(6):1552-1561.[doi:10.11992/tis.202305054]
Copy

Client dynamic sampling optimization of efficient reinforcement federated learning based on data quality assessment

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 6 Page number: 1552-1561 Column: 学术论文—人工智能基础 Public date: 2024-12-05
Title:
Client dynamic sampling optimization of efficient reinforcement federated learning based on data quality assessment
Author(s):
ZHAO Zehua;  LIANG Meiyu;  XUE Zhe;  LI Ang;  ZHANG Min
Beijing Key Laboratory of Intelligent Communication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing 100876, China
Keywords:
federated learning; deep reinforcement learning; client dynamic sampling; contribution index; data quality; communication efficiency; Shapley value; model accuracy
CLC:
TP295
DOI:
10.11992/tis.202305054
Abstract:
Communication cost and efficiency are the key bottlenecks of federated learning due to the existence of system and statistical heterogeneities. Selecting only a subset of clients to perform model updates and aggregation can effectively reduce communication costs among numerous participants. However, biased selection and uneven distribution of data quality across clients pose additional challenges to client sampling methods. Therefore, this paper proposes a method for client dynamic sampling optimization in efficient reinforcement federated learning based on data quality assessment (RQCS) to address the aforementioned issues. This method evaluates data quality on clients using a contribution index based on the Shapley value and intelligently selects clients with high data quality for each round of federated learning. By leveraging reinforcement learning, the method aims to offset the bias introduced by uneven data quality distribution, accelerate model convergence, and improve model accuracy. Experiments on the MNIST and CIFAR-10 datasets show that the proposed algorithm not only reduces communication costs but also further accelerates convergence speed and achieves better performance in model accuracy compared to other algorithms.
References:
[1] 林伟伟, 石方, 曾岚, 等. 联邦学习开源框架综述[J]. 计算机研究与发展, 2023, 60(7): 1551-1580.
LIN Weiwei, SHI Fang, ZENG Lan, et al. Survey of federated learning open-source frameworks[J]. Journal of computer research and development, 2023, 60(7): 1551-1580.
[2] 田家会, 吕锡香, 邹仁朋, 等. 一种联邦学习中的公平资源分配方案[J]. 计算机研究与发展, 2022, 59(6): 1240-1254.
TIAN Jiahui, LYU Xixiang, ZOU Renpeng, et al. A fair resource allocation scheme in federated learning[J]. Journal of computer research and development, 2022, 59(6): 1240-1254.
[3] LUO Bing, XIAO Wenli, WANG Shiqiang, et al. Tackling system and statistical heterogeneity for federated learning with adaptive client sampling[C]//IEEE INFOCOM 2022-IEEE Conference on Computer Communications. London: IEEE, 2022: 1739-1748.
[4] 李少波, 杨磊, 李传江, 等. 联邦学习概述: 技术、应用及未来[J]. 计算机集成制造系统, 2022, 28(7): 2119-2138.
LI Shaobo , YANG Lei , LI Chuangjiang, et al. Overview of federated learning: technology, applications and future[J]. Computer integrated manufacturing systems, 2022, 28(7): 2119-2138.
[5] 邱鑫源, 叶泽聪, 崔翛龙, 等. 联邦学习通信开销研究综述[J]. 计算机应用, 2022, 42(2): 333-342.
QIU Xinyuan, YE Zecong, CUI Xiaolong, et al. Survey of communication overhead of federated learning[J]. Journal of computer applications, 2022, 42(2): 333-342.
[6] KARIMIREDDY S P, KALE S, MOHRI M, et al. SCAFFOLD: stochastic controlled averaging for federated learning[EB/OL]. (2019-10-14)[2021-01-01]. http://arxiv.org/abs/1910.06378.
[7] SONG Tianshu, TONG Yongxin, WEI Shuyue. Profit allocation for federated learning[C]//2019 IEEE International Conference on Big Data. Los Angeles: IEEE, 2019: 2577-2586.
[8] MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[EB/OL]. (2016-02-17)[2021-01-01]. http://arxiv.org/abs/1602.05629.
[9] LI Tian, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[EB/OL]. (2018-12-14)[2021-01-01]. http://arxiv.org/abs/1812.06127.
[10] NISHIO T, YONETANI R. Client selection for federated learning with heterogeneous resources in mobile edge[C]//ICC 2019—2019 IEEE International Conference on Communications. Shanghai: IEEE, 2019: 1-7.
[11] CHO Y J, WANG Jianyu, JOSHI G. Client selection in federated learning: convergence analysis and power-of-choice selection strategies[EB/OL]. (2020-12-03)[2021-01-01]. http://arxiv.org/abs/2010.01243.
[12] FRABONI Y, VIDAL R, KAMENI L, et al. Clustered sampling: low-variance and improved representativity for clients selection in federated learning[EB/OL]. (2021-05-12)[2021-12-01]. http://arxiv.org/abs/2105.05883.
[13] ZHAO Yue, LI Meng, LAI Liangzhen, et al. Federated learning with non-IID data[EB/OL]. (2018-06-02)[2021-01-01]. http://arxiv.org/abs/1806.00582.
[14] KOLLIAS G, SALONIDIS T, WANG Shiqiang. Sketch to Skip and Select: communication efficient federated learning using locality sensitive hashing[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2023: 72-83.
[15] WANG Hao, KAPLAN Z, NIU Di, et al. Optimizing federated learning on non-IID data with reinforcement learning[C]//IEEE INFOCOM 2020-IEEE Conference on Computer Communications. Toronto: IEEE, 2020: 1698-1707.
[16] HUANG Tiansheng, LIN Weiwei, SHEN Li, et al. Stochastic client selection for federated learning with volatile clients[J]. IEEE internet of things journal, 2022, 9(20): 20055-20070.
[17] JI Shaoxiong, JIANG Wenqi, WALID A, et al. Dynamic sampling and selective masking for communication-efficient federated learning[EB/OL]. (2020-03-21)[2021-01-01]. http://arxiv.org/abs/2003.09603.
[18] RAI S, KUMARI A, PRASAD D K. Client selection in federated learning under imperfections in environment[J]. AI, 2022, 3(1): 124-145.
[19] LI Qinbin, HE Bingsheng, SONG D. Model-contrastive federated learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 10708-10717.
[20] WACHI A, SUI Yanan. Safe reinforcement learning in constrained Markov decision processes[EB/OL]. (2008-08-15)[2021-01-01]. http://arxiv.org/abs/2008.06626.
[21] HENDERSON P, ISLAM R, BACHMAN P, et al. Deep reinforcement learning that matters[C]//Proceedings of the AAAI Conference on Artificial Intelligence. AAAI , 2018.
[22] FENG Y, LI L, LIU Q. A kernel loss for solving the bellman equation[J]. Advances in neural information processing systems, 2019, 32.
[23] ZHANG Jie, GUO Song, QU Zhihao, et al. Adaptive federated learning on non-IID data with resource constraint[J]. IEEE transactions on computers, 2022, 71(7): 1655-1667.
[24] NICOLSON A, PALIWAL K K. Deep learning for minimum mean-square error approaches to speech enhancement[J]. Speech communication, 2019, 111: 44-55.
Similar References:

Memo

-

Last Update: 2024-11-05

Copyright © CAAI Transactions on Intelligent Systems