Shallow Univariate ReLU Networks as Splines: Initialization, Loss Surface, Hessian, and Gradient Flow Dynamics

Understanding the learning dynamics and inductive bias of neural networks (NNs) is hindered by the opacity of the relationship between NN parameters and the function represented. Partially, this is due to symmetries inherent within the NN parameterization, allowing multiple different parameter setti...

Descrizione completa

Dettagli Bibliografici

Autori principali:	Justin Sahs, Ryan Pyle, Aneel Damaraju, Josue Ortega Caro, Onur Tavaslioglu, Andy Lu, Fabio Anselmi, Ankit B. Patel
Natura:	Artigo
Lingua:	Inglês
Pubblicazione:	Frontiers Media S.A. 2022-05-01
Serie:	Frontiers in Artificial Intelligence
Soggetti:	neural networks symmetry implicit bias splines learning dynamics
Accesso online:	https://www.frontiersin.org/articles/10.3389/frai.2022.889981/full
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!