Statistically Preconditioned Accelerated Gradient Method for Distributed Optimization

25 Feb 2020 Hendrikx Hadrien Xiao Lin Bubeck Sebastien Bach Francis Massoulie Laurent

We consider the setting of distributed empirical risk minimization where multiple machines compute the gradients in parallel and a centralized server updates the model parameters. In order to reduce the number of communications required to reach a given accuracy, we propose a \emph{preconditioned} accelerated gradient method where the preconditioning is done by solving a local optimization problem over a subsampled dataset at the server... (read more)

PDF Abstract
No code implementations yet. Submit your code now

Categories


  • OPTIMIZATION AND CONTROL
  • DISTRIBUTED, PARALLEL, AND CLUSTER COMPUTING