Group Personalized Federated Learning

Federated learning (FL) can help promote data privacy by training a shared model in a de-centralized manner on the physical devices of clients. In the presence of heterogeneous distributions of local data, personalized FL strategy is introduced to mitigate the potential client drift. In this paper, we present the group personalization approach for applications of FL in which there exist inherent partitions over clients that are significantly distinct. In our approach, the global FL model is fine-tuned through another FL training process over each homogeneous group of clients, after which each group-specific FL model is further adapted and personalized per client. The proposed method can be well interpreted from a Bayesian hierarchical modeling perspective. With experiments on two real-world datasets for language modeling task, we demonstrate this approach can achieve superior personalization performance than other FL counterparts.