Underfitting
Underfitting is like trying to draw a complex, curvy shape using only a straight ruler; because your tool is too simple, you can't capture the true pattern, resulting in poor performance everywhere.
Underfitting occurs when a model lacks the capacity or training duration to learn the underlying data distribution, leading to high bias and poor predictive performance on both the training and validation sets.
Underfitting is a failure mode characterized by high bias where the model's hypothesis space is insufficiently expressive to represent the true underlying function, resulting in systematic error and an inability to minimize the loss function on both training and generalization datasets.
evolution
- 1950 · historyFoundations of Learning Theory
Alan Turing's 'Computing Machinery and Intelligence' established the conceptual basis for machine learning, implicitly defining the limits of simple models.
- 1969 · historyPerceptrons and Limitations
Minsky and Papert demonstrated that single-layer perceptrons could not solve non-linearly separable problems, formally identifying the phenomenon of underfitting.
- 1992 · historyBias-Variance Tradeoff Formalization
Geman et al. provided a rigorous mathematical framework for the bias-variance tradeoff, explicitly categorizing underfitting as high-bias error.
- 2016 · historyDeep Learning Standardization
The widespread adoption of deep learning architectures shifted the primary challenge from underfitting (simple models) to overfitting (complex models), cementing underfitting as a fundamental diagnostic baseline.