QR-Code

Anti-correlated noise in epoch-based stochastic gradient descent: implications for weight variances in flat directions

Stochastic gradient descent (SGD) has become a cornerstone of neural network optimization due to its computational efficiency and generalization capabilities. However, the gradient noise introduced by SGD is often assumed to be uncorrelated over time, despite the common practice of epoch-based train...

Ausführliche Beschreibung

Gespeichert in:
Hauptverfasser: Marcel Kühn, Bernd Rosenow
Format: Artigo
Sprache:Inglês
Veröffentlicht: IOP Publishing 2025-01-01
Schriftenreihe:Machine Learning: Science and Technology
Schlagworte:
stochastic gradient descent
asymptotic analysis
discrete time
hessian
Online-Zugang:https://doi.org/10.1088/2632-2153/ae0ab2
Tags: Tag hinzufügen
Keine Tags, Fügen Sie das erste Tag hinzu!

Ähnliche Einträge