[1]GAO Yuan,SHI Runhua,LIU Changjie.Federated learning scheme with adaptive differential privacy[J].CAAI Transactions on Intelligent Systems,2024,19(6):1395-1406.[doi:10.11992/tis.202306052]
Copy

Federated learning scheme with adaptive differential privacy

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 6 Page number: 1395-1406 Column: 学术论文—机器学习 Public date: 2024-12-05
Title:
Federated learning scheme with adaptive differential privacy
Author(s):
GAO Yuan;  SHI Runhua;  LIU Changjie
School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China
Keywords:
federated learning; differential privacy; adaptive; gradient descent; convolutional neural network; learning rate; gradient; privacy budget
CLC:
TP181
DOI:
10.11992/tis.202306052
Abstract:
Differential privacy is widely used in federated learning to ensure the security of model parameters. However, inappropriate methods for adding noise can limit the further improvement of model accuracy. A federated learning method with adaptive allocation of the privacy budget and calculation of the learning rate (DP–FLAGD) is proposed to address this problem. Through the adaptive allocation of the privacy budget, the right descending direction of the gradient can be identified, and the appropriate learning rate can be calculated to achieve minimal loss. Simultaneously, DP–FLAGD provides different privacy budgets for users with various privacy requirements. Experiments were conducted on two widely used datasets, namely MNIST and CIFAR-10, to evaluate the validity of DP–FLAGD. Experimental results show that the DP–FLAGD scheme can further improve model accuracy while ensuring the safety of model parameters.
References:
[1] 徐树良, 王俊红. 结合无监督学习的数据流分类算法[J]. 模式识别与人工智能, 2016, 29(7): 665-672.
XU Shuliang, WANG Junhong. Classification algorithm combined with unsupervised learning for data stream[J]. Pattern recognition and artificial intelligence, 2016, 29(7): 665-672.
[2] ALWARAFY A, AL-THELAYA K A, ABDALLAH M, et al. A survey on security and privacy issues in edge-computing-assisted Internet of Things[J]. IEEE Internet of Things journal, 2021, 8(6): 4004-4022.
[3] VOIGT P, VON DEM BUSSCHE A. The EU general data protection regulation (GDPR)[M]. Cham: Springer International Publishing, 2017.
[4] PHONG L T, AONO Y, HAYASHI T, et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE transactions on information forensics and security, 2018, 13(5): 1333-1345.
[5] FREDRIKSON M, JHA S, RISTENPART T. Model inversion attacks that exploit confidence information and basic countermeasures[C]//Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. Denver: ACM, 2015: 1322-1333.
[6] NASR M, SHOKRI R, HOUMANSADR A. Comprehensive privacy analysis of deep learning: passive and active white-box inference attacks against centralized and federated learning[C]//2019 IEEE Symposium on Security and Privacy. San Francisco: IEEE, 2019: 739-753.
[7] GEYER R C, KLEIN T, NABI M. Differentially private federated learning: a client level perspective[EB/OL]. (2017-12-20) [2021-01-01]. http://arxiv.org/abs/1712.07557.
[8] WEI Kang, LI Jun, DING Ming, et al. Federated learning with differential privacy: algorithms and performance analysis[J]. IEEE transactions on information forensics and security, 2020, 15: 3454-3469.
[9] LI Yiwei, CHANG T H, CHI C Y. Secure federated averaging algorithm with differential privacy[C]//2020 IEEE 30th International Workshop on Machine Learning for Signal Processing. Espoo: IEEE, 2020: 1-6.
[10] MCMAHAN H B, RAMAGE D, TALWAR K, et al. Learning differentially private recurrent language models[EB/OL]. (2017-10-18) [2021-01-01]. http://arxiv.org/abs/1710.06963.
[11] ZHAO Yang, ZHAO Jun, YANG Mengmeng, et al. Local differential privacy-based federated learning for Internet of Things[J]. IEEE internet of things journal, 2021, 8(11): 8836-8853.
[12] YANG Jia, FU Cai, LIU Xiaoyang, et al. Recommendations in smart devices using federated tensor learning[J]. IEEE internet of things journal, 2022, 9(11): 8425-8437.
[13] DWORK C, ROTH A. The algorithmic foundations of differential privacy[J]. Foundations and trends? in theoretical computer science, 2013, 9(3/4): 211-407.
[14] 梁文雅, 刘波, 林伟伟, 等. 联邦学习激励机制研究综述[J]. 计算机科学, 2022, 49(12): 46-52.
LIANG Wenya, LIU Bo, LIN Weiwei, et al. Survey of incentive mechanism for federated learning[J]. Computer science, 2022, 49(12): 46-52.
[15] SHOKRI R, SHMATIKOV V. Privacy-preserving deep learning[C]//2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton). Monticello: IEEE, 2015: 909-910.
[16] DWORK C, LEI Jing. Differential privacy and robust statistics[C]//Proceedings of the forty-first annual ACM symposium on Theory of computing. Bethesda: ACM, 2009: 371-380.
[17] ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. Vienna: ACM, 2016: 308-318.
[18] PHAN N, WANG Yue, WU Xintao, et al. Differential privacy preservation for deep auto-encoders: an application of human behavior prediction[J]. Proceedings of the AAAI conference on artificial intelligence, 2016, 30(1): 3175-3183.
[19] BENGIO Y. Learning deep architectures for AI[J]. Foundations and trends? in machine learning, 2009, 2(1): 1-127.
[20] PHAN N, WU Xintao, HU Han, et al. Adaptive Laplace mechanism: differential privacy preservation in deep learning[C]//2017 IEEE International Conference on Data Mining. New Orleans: IEEE, 2017: 385-394.
[21] LEE J, KIFER D. Concentrated differentially private gradient descent with adaptive per-iteration privacy budget[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. London: ACM, 2018: 1656-1665.
[22] MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[EB/OL]. (2016-02-17)[2021-01-01] http://arxiv.org/abs/1602.05629.
[23] 张兴, 陈昊. 差分隐私的高维数据发布研究综述[J]. 智能系统学报, 2021, 16(6): 989-998.
ZHANG Xing, CHEN Hao. A research review of high-dimensional data publishing based on a differential privacy model[J]. CAAI transactions on intelligent systems, 2021, 16(6): 989-998.
[24] ZHANG Jun, XIAO Xiaokui, YANG Yin, et al. PrivGene: differentially private model fitting using genetic algorithms[C]//Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data. New York: ACM, 2013: 665-676.
[25] TALWAR K, THAKURTA A, ZHANG Li. Nearly-optimal private LASSO[J]. Advances in neural information processing systems, 2015: 3025-3033.
[26] SUN Peng, CHE Haoxuan, WANG Zhibo, et al. Pain-FL: personalized privacy-preserving incentive for federated learning[J]. IEEE journal on selected areas in communications, 2021, 39(12): 3805-3820.
[27] BUN M, STEINKE T. Concentrated differential privacy: simplifications, extensions, and lower bounds[M]//Lecture Notes in Computer Science. Berlin: Springer Berlin Heidelberg, 2016: 635-658.
[28] MAHAWAGA ARACHCHIGE P C, BERTOK P, et al. Local differential privacy for deep learning[J]. IEEE internet of things journal, 2020, 7(7): 5827-5842.
Similar References:

Memo

-

Last Update: 2024-11-05

Copyright © CAAI Transactions on Intelligent Systems