Mutual Information Based Learning Rate Decay for Stochastic Gradient Descent Training of Deep Neural Networks

This paper demonstrates a novel approach to training deep neural networks using a Mutual Information (MI)-driven, decaying Learning Rate (LR), Stochastic Gradient Descent (SGD) algorithm. MI between the output of the neural network and true outcomes is used to adaptively set the LR for the network,...

Descripción completa

Detalles Bibliográficos

Autor principal:	Shrihari Vasudevan
Formato:	Artigo
Lenguaje:	Inglês
Publicado:	MDPI AG 2020-05-01
Colección:	Entropy
Materias:	deep neural networks stochastic gradient descent mutual information adaptive learning rate
Acceso en línea:	https://www.mdpi.com/1099-4300/22/5/560
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!