FedBalancer

Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients



Abstract


Federated Learning (FL) trains a machine learning model on distributed clients without exposing individual data. Unlike centralized training that is usually based on carefully-organized data, FL deals with on-device data that are often unfiltered and imbalanced. As a result, conventional FL training protocol that treats all data equally leads to a waste of local computational resources and slows down the global learning process. To this end, we propose FedBalancer, a systematic FL framework that actively selects clients' training samples. Our sample selection strategy prioritizes more "informative" data while respecting privacy and computational capabilities of clients. To better utilize the sample selection to speed up global training, we further introduce an adaptive deadline control scheme that predicts the optimal deadline for each round with varying client training data. Compared with existing FL algorithms with deadline configuration methods, our evaluation on five datasets from three different domains shows that FedBalancer improves the time-to-accuracy performance by 1.20~4.48x while improving the model accuracy by 1.1~5.0%. We also show that FedBalancer is readily applicable to other FL approaches by demonstrating that FedBalancer improves the convergence speed and accuracy when operating jointly with three different FL algorithms.


Video




Conference Presentation




Publications


FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients
Jaemin Shin, Yuanchun Li, Yunxin Liu, and Sung-Ju Lee
ACM International Conference on Mobile Systems, Applications, and Services (MobiSys) 2022.
PDF


People


Jaemin Shin

KAIST

Yuanchun Li

Tsinghua University

Yunxin Liu

Tsinghua University

Sung-Ju Lee

KAIST