Anti-correlated noise in epoch-based stochastic gradient descent: implications for weight variances in flat directions
Stochastic gradient descent (SGD) has become a cornerstone of neural network optimization due to its computational efficiency and generalization capabilities. However, the gradient noise introduced by SGD is often assumed to be uncorrelated over time, despite the common practice of epoch-based train...
में बचाया:
| मुख्य लेखकों: | , |
|---|---|
| स्वरूप: | Artigo |
| भाषा: | Inglês |
| प्रकाशित: |
IOP Publishing
2025-01-01
|
| श्रृंखला: | Machine Learning: Science and Technology |
| विषय: | |
| ऑनलाइन पहुंच: | https://doi.org/10.1088/2632-2153/ae0ab2 |
| टैग: |
कोई टैग नहीं, इस रिकॉर्ड को टैग करने वाले पहले व्यक्ति बनें!
|