Decentralized SGD with Over-the-Air Computation

E. Ozfatura, Stefano Rini, D. Gunduz

研究成果: Conference contribution同行評審

摘要

We consider multiple devices with local datasets collaboratively learning a global model through device-to-device (D2D) communications. The conventional decentralized stochastic gradient descent (DSGD) solution for this problem assumes error-free orthogonal links among the devices. This is based on the assumption of an underlying communication protocol that takes care of the noise, fading, and interference in the wireless medium. In this work, we show the suboptimality of this approach by designing the communication and learning protocols jointly. We first consider a point-to-point (P2P) communication scheme by scheduling D2D transmissions in an orthogonal fashion to minimize interference. Then, we propose a novel over-the-air consensus scheme by exploiting the signal superposition property of wireless transmission, rather than avoiding interference. In the proposed OAC-MAC scheme, multiple nodes align their transmissions toward a single receiver node. For both schemes, we cast the scheduling problem as a graph coloring problem. We then numerically compare the two approaches for the distributed MNIST image classification task under various network conditions. We show that the OAC-MAC scheme attains better convergence speed and final accuracy thanks to the improved robustness against channel fading and noise. We also introduce a noise-aware version of the OAC-MAC scheme with further improvements in the convergence speed and accuracy.

原文English
主出版物標題2020 IEEE Global Communications Conference, GLOBECOM 2020 - Proceedings
發行者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子)9781728182988
DOIs
出版狀態Published - 十二月 2020
事件2020 IEEE Global Communications Conference, GLOBECOM 2020 - Virtual, Taipei, Taiwan
持續時間: 7 十二月 202011 十二月 2020

出版系列

名字2020 IEEE Global Communications Conference, GLOBECOM 2020 - Proceedings

Conference

Conference2020 IEEE Global Communications Conference, GLOBECOM 2020
國家Taiwan
城市Virtual, Taipei
期間7/12/2011/12/20

指紋 深入研究「Decentralized SGD with Over-the-Air Computation」主題。共同形成了獨特的指紋。

引用此