Shensheng Zheng | Searcharxiv

arXiv2024

Adaptive Federated Learning via New Entropy Approach

Federated Learning (FL) has emerged as a prominent distributed machine learning framework that enables geographically discrete clients to train a global model collaboratively while preserving their privacy-sensitive data. However, due to the non-independent-and-identically-distributed (Non-IID) data generated by heterogeneous clients, the performances of the conventional federated optimization schemes such as FedAvg and its variants deteriorate, requiring the design to adaptively adjust specific model parameters to alleviate the negative influence of heterogeneity. In this paper, by leveraging entropy as a new metric for assessing the degree of system disorder, we propose an adaptive FEDerated learning algorithm based on ENTropy theory (FedEnt) to alleviate the parameter deviation among heterogeneous clients and achieve fast convergence. Nevertheless, given the data disparity and parameter deviation of heterogeneous clients, determining the optimal dynamic learning rate for each client becomes a challenging task as there is no communication among participating clients during the local training epochs. To enable a decentralized learning rate for each participating client, we first introduce the mean-field terms to estimate the components associated with other clients' local parameters. Furthermore, we provide rigorous theoretical analysis on the existence and determination of the mean-field estimators. Based on the mean-field estimators, the closed-form adaptive learning rate for each client is derived by constructing the Hamilton equation. Moreover, the convergence rate of our proposed FedEnt is proved. The extensive experimental results on the real-world datasets (i.e., MNIST, EMNIST-L, CIFAR10, and CIFAR100) show that our FedEnt algorithm surpasses FedAvg and its variants (i.e., FedAdam, FedProx, and FedDyn) under Non-IID settings and achieves a faster convergence rate.

cs.DC↗

arXiv2022

Dynamic Pricing for Client Recruitment in Federated Learning

Though federated learning (FL) well preserves clients' data privacy, many clients are still reluctant to join FL given the communication cost and energy consumption in their mobile devices. It is important to design pricing compensations to motivate enough clients to join FL and distributively train the global model. Prior pricing mechanisms for FL are static and cannot adapt to clients' random arrival pattern over time. We propose a new dynamic pricing solution in closed-form by constructing the Hamiltonian function to optimally balance the client recruitment time and the model training time, without knowing clients' actual arrivals or training costs. During the client recruitment phase, we offer time-dependent monetary rewards per client arrival to trade-off between the total payment and the FL model's accuracy loss. Such reward gradually increases when we approach to the recruitment deadline or have greater data aging, and we also extend the deadline if the clients' training time per iteration becomes shorter. Further, we extend to consider heterogeneous client types in training data size and training time per iteration. We successfully extend our dynamic pricing solution and develop an optimal algorithm of linear complexity to monotonically select client types for FL. Finally, we also show the robustness of our solution against estimation error of clients' data sizes and run numerical experiments to validate our conclusion.

cs.GT↗