On the loss landscape of a class of deep neural networks with no bad local valleys

Add code
Sep 27, 2018
Quynh Nguyen, Mahesh Chandra Mukkamala, Matthias Hein

Figure 1 for On the loss landscape of a class of deep neural networks with no bad local valleys
Figure 2 for On the loss landscape of a class of deep neural networks with no bad local valleys
Figure 3 for On the loss landscape of a class of deep neural networks with no bad local valleys
Figure 4 for On the loss landscape of a class of deep neural networks with no bad local valleys

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily close to zero. This implies that these networks have no sub-optimal strict local minima.

View paper onarxiv iconopen_review iconOpenReview

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon