Gradient descent
In mathematics, gradient descent (also often called steepest descent) is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent.
Gradient descent - Wikipedia Jump to content From Wikipedia, the free encyclopedia Optimization algorithm For the analytical method called "steepest descent", see Method of steepest descent . Gradient descent in 2D Gradient descent is a method for unconstrained mathematical optimization . It is a first-order iterative algorithm for minimizing a differentiable multivariate function . Part of a series on Machine learning and data mining Paradigms Supervised learning Unsupervised learning Semi-supervised learning Self-supervised learning Reinforcement learning Meta-learning Online learning Batch
Explore this link on the map →related reading
- Stochastic gradient descent - Wikipediaen.m.wikipedia.org
- Why Momentum Really Worksdistill.pub
- Gradient Descent With Momentum from Scratch - MachineLearningMastery.commachinelearningmastery.com
- A Visual Explanation of Gradient Descent Methods (Momentum, AdaGrad, RMSProp, Adam) | Towards Data Sciencetowardsdatascience.com
- Linear regression: Gradient descent | Machine Learning | Google for Developersdevelopers.google.com
- Effortless optimization through gradient flows – Machine Learning Research Blogfrancisbach.com
- Gradient descent, how neural networks learn | 3Blue1Brown3blue1brown.com
- AdaGrad - Cornell University Computational Optimization Open Textbook - Optimization Wikioptimization.cbe.cornell.edu
- The Little Book of Deep Learningfleuret.org
- An overview of gradient descent optimization algorithmsruder.io
- Modular Manifolds - Thinking Machines Labthinkingmachines.ai
- Neural networks and deep learningneuralnetworksanddeeplearning.com