learning rate
Gradient step size
Learning rate is a parameter in machine learning that controls how much a model adjusts its internal values during training.
A high learning rate makes large adjustments and learns quickly but may overshoot optimal solutions.
A low learning rate makes smaller, more precise adjustments but takes longer to converge.
Finding the right learning rate is crucial—too high and the model never stabilizes, too low and training becomes impractically slow.
- gradientRate of change across space
- descentDownward slope or lineage
- optimizerThing that improves efficiency
- epochDistinct historical period
- trainingDeveloping abilities through practice
convergence · step · decay · backpropagation · hyperparameter · tuning · fine tuning · batch · loss · weights · neural · neural network · Adam · schedule · warmup · annealing · plateau · adjustment
- machine learningneural networks, backpropagation, underfitting, CNNs, weights
- optimizationoptimization, caching, optimum, indexing, prefetching
- neural networksneural network, neural networks, hidden layers, backpropagation, layers
- ML settingshyperparameters, epochs, dropout, regularization, momentum
- Reinforcement Learningagent, reward, state, action, environment
- Bandit Algorithmsarm, exploration, reward, regret, pull