A Simple Nearly-Optimal Restart Scheme For Speeding-Up First Order Methods

21 Oct 2020  ·  Renegar James, Grimmer Benjamin ·

We present a simple scheme for restarting first-order methods for convex optimization problems. Restarts are made based only on achieving specified decreases in objective values, the specified amounts being the same for all optimization problems... Unlike existing restart schemes, the scheme makes no attempt to learn parameter values characterizing the structure of an optimization problem, nor does it require any special information that would not be available in practice (unless the first-order method chosen to be employed in the scheme itself requires special information). As immediate corollaries to the main theorems, we show that when some well-known first-order methods are employed in the scheme, the resulting complexity bounds are nearly optimal for particular -- yet quite general -- classes of problems. read more

PDF Abstract
No code implementations yet. Submit your code now


Optimization and Control