Gradient Descent
Gradient descent is the optimisation method used to train most machine learning models, repeatedly adjusting parameters in the direction that most reduces the loss.
What is Gradient Descent?
The gradient tells the algorithm which way is "downhill" on the error landscape. Practical variants such as stochastic gradient descent and Adam work on small batches of data, which makes training on huge datasets feasible.
How it works
Compute the gradient of the loss, move each parameter a small step (set by the learning rate) in the opposite direction, and repeat until the loss stops improving.
← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions