An Inertial Newton Algorithm for Deep Learning - IRT Saint Exupéry - Institut de Recherche Technologique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2019

An Inertial Newton Algorithm for Deep Learning

Résumé

We devise a learning algorithm for possibly nonsmooth deep neural networks featuring inertia and Newtonian directional intelligence only by means of a back-propagation oracle. Our algorithm, called INDIAN, has an appealing mechanical interpretation, making the role of its two hyperparameters transparent. An elementary phase space lifting allows both for its implementation and its theoretical study under very general assumptions. We handle in particular a stochastic version of our method (which encompasses usual mini-batch approaches) for nonsmooth activation functions (such as ReLU). Our algorithm shows high efficiency and reaches state of the art on image classification problems.
Fichier principal
Vignette du fichier
indian.pdf (591.04 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02140748 , version 1 (27-05-2019)
hal-02140748 , version 2 (06-06-2019)
hal-02140748 , version 3 (12-12-2019)
hal-02140748 , version 4 (12-10-2020)
hal-02140748 , version 5 (02-07-2021)
hal-02140748 , version 6 (20-08-2021)

Identifiants

Citer

Camille Castera, Jérôme Bolte, Cédric Févotte, Edouard Pauwels. An Inertial Newton Algorithm for Deep Learning. 2019. ⟨hal-02140748v2⟩
694 Consultations
346 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More