Giulia Fanti: Communication Complexity of Federated Learning

Title: Reducing the Communication Complexity of Federated Learning through Multistage Optimization
Speaker: Giulia Fanti, Carnegie Mellon University
Abstract: Federated learning (FL) aims to minimize the communication cost of training a model over heterogeneous data distributed across many clients. A common approach is local update methods, where clients take multiple optimization steps over local data before communicating with the server (e.g., FedAvg). When clients’ data is homogeneous, local methods converge fast with a small number of communications. On the other hand, when clients are highly heterogeneous, global update methods, where clients take one optimization step per communication round (e.g., SGD), significantly outperform local methods. A fundamental question in FL is how to exploit the benefits of both local and global approaches to achieve near-optimal convergence rates for all heterogeneity regimes. To this end, we propose a federated chaining framework; we first run a local-update method up to a heterogeneity-induced error floor and then switch to a global-update method. Federated chaining is the first approach to achieve near-optimal convergence rates for strongly convex functions and non-convex PL functions. Empirical results support this theoretical gain over existing methods.
About the speaker: Giulia Fanti is an Assistant Professor of Electrical and Computer Engineering at Carnegie Mellon University. Her research interests span the security, privacy, and efficiency of distributed systems. She is a two-time fellow of the World Economic Forum’s Global Future Council on Cybersecurity and a member of NIST’s Information Security and Privacy Advisory Board. Her work has been recognized with several awards, including best paper awards, a Sloan Fellowship, an Intel Rising Star Faculty Award, and a SIGMETRICS Rising Star Award. She obtained her Ph.D. in EECS from U.C. Berkeley and her B.S. in ECE from Olin College of Engineering.
Upcoming seminars:
2023-01-26: Lena Voita, FAIR
2023-02-02: Vincent Szolnoky, Chalmers University of Technology
2023-02-23: Alexander Ilin, Aalto University
2023-03-09: Shay Cohen, University of Edinburgh

[@ahmedal-saedi4672]

Thanks for the inspiring seminar