Learning Rate
The learning rate is a hyperparameter that sets how big a step a model takes when updating its parameters during training.
What is Learning Rate?
Too high a learning rate makes training unstable and may overshoot good solutions; too low makes it slow or stuck. Modern training often changes the learning rate over time using a schedule, for example warming up and then decaying.
How it works
In each gradient descent update, the gradient is multiplied by the learning rate to decide how much each parameter changes.
← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions