Parameters
Think of these as the billions of tiny settings or 'knobs' inside an AI that it adjusts while learning, which store everything the model knows and allow it to make decisions.
The internal numerical weights and biases within a neural network that are optimized during training; these billions of values represent the model's learned knowledge and determine its capacity to process and generate information.
The learned numerical weights inside a model; their count (often billions) roughly indexes model capacity, though architecture and data quality matter as much.
evolution
- 1958 · historyPerceptron Weights
Frank Rosenblatt introduced the Perceptron, where adjustable numerical weights were first used to represent learned associations.
- 1986 · historyBackpropagation
Rumelhart, Hinton, and Williams popularized the backpropagation algorithm, enabling the systematic optimization of millions of internal parameters.
- 2012 · historyAlexNet Scaling
The success of AlexNet demonstrated that deep neural networks with 60 million parameters could significantly outperform traditional computer vision methods.
- 2017 · historyTransformer Architecture
The introduction of the Transformer model enabled massive parallelization, allowing parameter counts to scale into the billions.
- 2020 · historyGPT-3 Era
OpenAI released GPT-3, marking the transition to Large Language Models (LLMs) with 175 billion parameters.