Robust Semi-Decentralized Federated Learning via Collaborative Relaying

Michal Yemini, Rajarshi Saha, Emre Ozfatura, Deniz Gunduz, Andrea J. Goldsmith

Research output: Contribution to journalArticlepeer-review

Abstract

Intermittent connectivity of clients to the parameter server (PS) is a major bottleneck in federated edge learning frameworks. The lack of constant connectivity induces a large generalization gap, especially when the local data distribution amongst clients exhibits heterogeneity. To overcome intermittent communication outages between clients and the central PS, we introduce the concept of collaborative relaying wherein the participating clients relay their neighbors’ local updates to the PS in order to boost the participation of clients with poor connectivity to the PS. We propose a semi-decentralized federated learning framework in which at every communication round, each client initially computes a local averaging of a subset of its neighboring clients’ updates, and eventually transmits to the PS a weighted average of its own update and those of its neighbors’. We appropriately optimize these local averaging weights to ensure that the global update at the PS is unbiased with minimal variance – consequently improving the convergence rate. Numerical evaluations on the CIFAR-10 dataset demonstrate that our collaborative relaying approach outperforms federated averaging-based benchmarks for learning over intermittently-connected networks such as when the clients communicate over millimeter wave channels with intermittent blockages.

Original languageEnglish (US)
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Wireless Communications
DOIs
StateAccepted/In press - 2023

ASJC Scopus subject areas

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics

Keywords

  • Collaboration
  • collaborative relaying
  • convergence
  • Convergence
  • Federated learning
  • Federated learning
  • intermittent connectivity
  • Optimization
  • Servers
  • Topology
  • weight optimization
  • Wireless communication

Fingerprint

Dive into the research topics of 'Robust Semi-Decentralized Federated Learning via Collaborative Relaying'. Together they form a unique fingerprint.

Cite this