overfitting
Model memorizes noise
Overfitting is a problem in machine learning where a model learns its training data too precisely, including noise and quirks, and then performs poorly on new data.
An overfitted model essentially memorizes rather than generalizes: The model achieved 99% accuracy in training but failed miserably in production—classic overfitting.
The concept extends loosely to any situation where excessive tailoring to past conditions leaves you unprepared for new ones.
Deriving principles from examples
- regularizationPenalty preventing model complexity
- cross-validationResampling method for model assessment
- validationConfirming truth or accuracy
- trainingDeveloping abilities through practice
memorization · overtraining · dropout · pruning · ensemble · bagging · boosting · hyperparameters
Committing information to recall
- outliersExtreme data points
- artifactsHuman-made historical objects
- idiosyncrasiesIndividual behavioral peculiarities
- anomaliesDepartures from expected patterns
Excessive structural intricacy
- overengineeringExcessive unnecessary complexity
- flexibilityAbility to adapt easily
- parametersDefined boundaries or constraints
- polynomialExpression with variable terms
nonlinearity · depth · layers · neurons · architecture
generalization
Deriving principles from examples
- regularizationPenalty preventing model complexity
- cross-validationResampling method for model assessment
- validationConfirming truth or accuracy
training · memorization · overtraining · dropout · pruning · ensemble · bagging · boosting · hyperparameters · ridge
memorization
Committing information to recall
- outliersExtreme data points
- artifactsHuman-made historical objects
- idiosyncrasiesIndividual behavioral peculiarities
anomalies · signal-to-noise · interpolation · wiggly · jagged
overcomplexity
Excessive structural intricacy
- overengineeringExcessive unnecessary complexity
- flexibilityAbility to adapt easily
- parametersDefined boundaries or constraints
polynomial · nonlinearity · depth · layers · neurons · architecture
underfitting
Model too simple
overinterpretation
Excessive meaning extraction
- harkingListening closely, archaically
- data sciencealgorithm, data analytics, data, machine learning, dataset
- machine learningneural networks, deep learning, backpropagation, underfitting, CNNs
- neural networksneural network, neural networks, hidden layers, backpropagation, layers
- analysis errorsunderfitting, wrong assumptions, extrapolation, cherry picking, selection bias
- ML errorsunderfitting, misclassification, type I error, type II error, false positive
- model evaluationaccuracy, AUC, cross-validation, precision, recall