SignSGD With Federated Voting

TY - JOUR

T1 - SignSGD With Federated Voting

AU - Park, Chanho

AU - Poor, H. Vincent

AU - Lee, Namyoon

N1 - Publisher Copyright: © 2026 IEEE.

PY - 2026

Y1 - 2026

N2 - Distributed learning is commonly used for accelerating model training by harnessing the computational capabilities of multiple-edge devices. However, in practical applications, communication delay emerges as a bottleneck due to substantial information exchange required between workers and a central parameter server. SignSGD with majority voting (signSGD-MV) is an effective distributed learning algorithm that can significantly reduce communication costs by one-bit quantization. However, due to heterogeneous computational capabilities, it fails to converge when mini-batch sizes differ among workers. To overcome this, we propose a novel signSGD optimizer with federated voting (signSGD-FV). The idea of federated voting is to exploit learnable weights to perform weighted majority voting. The server learns the weights assigned to the edge devices in an online fashion based on their computational capabilities. Subsequently, these weights are employed to decode signs of aggregated local gradients to minimize the sign decoding error probability. We provide a unified convergence rate analysis applicable to scenarios where the estimated weights are known to the parameter server either perfectly or imperfectly. We demonstrate that the proposed signSGD-FV algorithm has a theoretical convergence guarantee even when edge devices use heterogeneous mini-batch sizes. Experimental results show that signSGD-FV outperforms signSGD-MV, exhibiting a faster convergence rate, especially in heterogeneous mini-batch sizes. Furthermore, we demonstrate that variants of the proposed signSGD-FV framework can address additional challenges in distributed learning systems, such as data heterogeneity and partial participation, highlighting the extensibility of signSGD-FV.

AB - Distributed learning is commonly used for accelerating model training by harnessing the computational capabilities of multiple-edge devices. However, in practical applications, communication delay emerges as a bottleneck due to substantial information exchange required between workers and a central parameter server. SignSGD with majority voting (signSGD-MV) is an effective distributed learning algorithm that can significantly reduce communication costs by one-bit quantization. However, due to heterogeneous computational capabilities, it fails to converge when mini-batch sizes differ among workers. To overcome this, we propose a novel signSGD optimizer with federated voting (signSGD-FV). The idea of federated voting is to exploit learnable weights to perform weighted majority voting. The server learns the weights assigned to the edge devices in an online fashion based on their computational capabilities. Subsequently, these weights are employed to decode signs of aggregated local gradients to minimize the sign decoding error probability. We provide a unified convergence rate analysis applicable to scenarios where the estimated weights are known to the parameter server either perfectly or imperfectly. We demonstrate that the proposed signSGD-FV algorithm has a theoretical convergence guarantee even when edge devices use heterogeneous mini-batch sizes. Experimental results show that signSGD-FV outperforms signSGD-MV, exhibiting a faster convergence rate, especially in heterogeneous mini-batch sizes. Furthermore, we demonstrate that variants of the proposed signSGD-FV framework can address additional challenges in distributed learning systems, such as data heterogeneity and partial participation, highlighting the extensibility of signSGD-FV.

KW - communication-efficient learning

KW - Federated learning

KW - majority voting

UR - https://www.scopus.com/pages/publications/105040407722

UR - https://www.scopus.com/pages/publications/105040407722#tab=citedBy

U2 - 10.1109/TSP.2026.3697805

DO - 10.1109/TSP.2026.3697805

M3 - Article

SN - 1053-587X

VL - 74

SP - 2740

EP - 2755

JO - IEEE Transactions on Signal Processing

JF - IEEE Transactions on Signal Processing

ER -