Get our free extension to see links to code for papers anywhere online!Free add-on: code for papers everywhere!Free add-on: See code for papers anywhere!
Add to Chrome
Add to Firefox
Add to EdgeDepth creates no more spurious local minima
Jan 28, 2019
We show that for any convex differentiable loss function, a deep linear network has no spurious local minima as long as it is true for the two layer case. When applied to the quadratic loss, our result immediately implies the powerful result in [Kawaguchi 2016] that there is no spurious local minima in deep linear networks. Further, with the recent work [Zhou and Liang 2018], we can remove all the assumptions in [Kawaguchi 2016]. Our proof is short and elementary. It builds on the recent work of [Laurent and von Brecht 2018] and uses a new rank one perturbation argument.
View paper on