自适应差分隐私的联邦学习方案

PDF下载HTML

Title:

Federated learning scheme with adaptive differential privacy

作者:

高媛, 石润华, 刘长杰

华北电力大学 控制与计算机工程学院, 北京 102206

Author(s):

GAO Yuan, SHI Runhua, LIU Changjie

School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China

关键词:

联邦学习; 差分隐私; 自适应; 梯度下降; 卷积神经网络; 学习率; 梯度; 隐私预算

Keywords:

federated learning; differential privacy; adaptive; gradient descent; convolutional neural network; learning rate; gradient; privacy budget

分类号:

TP181

DOI:

10.11992/tis.202306052

摘要:

差分隐私被广泛应用于联邦学习中，以保障模型参数的安全，但不够合理的加噪方式会限制模型准确度进一步提高。为此，提出一种能够自适应分配隐私预算和计算学习率的联邦学习方案（differential privacy-federated learning adaptive gradient descent，DP-FLAGD），通过自适应分配隐私预算找到梯度的正确下降方向，并计算合适的学习率以达到最小的损失。同时，DP-FLAGD方案能够为不同隐私需求的用户提供不同的隐私预算，以满足其需求。为评估DP-FLAGD的有效性，在广泛使用的2个数据集MNIST（modiffe national institute of standard and technology）和CIFAR-10上进行相关实验，实验结果表明，DP-FLAGD方案在保证模型参数安全的同时，能够进一步提高模型的准确率。

Abstract:

Differential privacy is widely used in federated learning to ensure the security of model parameters. However, inappropriate methods for adding noise can limit the further improvement of model accuracy. A federated learning method with adaptive allocation of the privacy budget and calculation of the learning rate (DP–FLAGD) is proposed to address this problem. Through the adaptive allocation of the privacy budget, the right descending direction of the gradient can be identified, and the appropriate learning rate can be calculated to achieve minimal loss. Simultaneously, DP–FLAGD provides different privacy budgets for users with various privacy requirements. Experiments were conducted on two widely used datasets, namely MNIST and CIFAR-10, to evaluate the validity of DP–FLAGD. Experimental results show that the DP–FLAGD scheme can further improve model accuracy while ensuring the safety of model parameters.