Accelerated Gossip via Stochastic Heavy Ball Method
Accelerated Gossip via Stochastic Heavy Ball Method
In this paper we show how the stochastic heavy ball method (SHB) -- a popular method for solving stochastic convex and non-convex optimization problems --operates as a randomized gossip algorithm. In particular, we focus on two special cases of SHB: the Randomized Kaczmarz method with momentum and its block variant. Building upon a recent framework for the design and analysis of randomized gossip algorithms, [Loizou Richtarik, 2016] we interpret the distributed nature of the proposed methods. We present novel protocols for solving the average consensus problem where in each step all nodes of the network update their values but only a subset of them exchange their private values. Numerical experiments on popular wireless sensor networks showing the benefits of our protocols are also presented.
8 pages, 5 Figures, 56th Annual Allerton Conference on Communication, Control, and Computing, 2018
- University of Edinburgh United Kingdom
- King Abdullah University of Science and Technology Saudi Arabia
FOS: Computer and information sciences, Numerical Analysis, Information Theory (cs.IT), Information Theory, Numerical Analysis (math.NA), Systems and Control (eess.SY), Machine Learning (cs.LG), Machine Learning, Optimization and Control (math.OC), Optimization and Control, FOS: Mathematics, FOS: Electrical engineering, electronic engineering, information engineering, Systems and Control
FOS: Computer and information sciences, Numerical Analysis, Information Theory (cs.IT), Information Theory, Numerical Analysis (math.NA), Systems and Control (eess.SY), Machine Learning (cs.LG), Machine Learning, Optimization and Control (math.OC), Optimization and Control, FOS: Mathematics, FOS: Electrical engineering, electronic engineering, information engineering, Systems and Control
selected citations These citations are derived from selected sources.This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).10 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Top 10% influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Top 10%
Citations provided by BIP!Popularity provided by BIP!