Weights
The billions of numbers learned during training — the 'soul' of the model on disk. These weights are what make the model a probability machine; they are static until the model is re-trained or fine-tuned.
Weights are the learned parameters of a neural network—the billions of numerical values stored on disk that define the model's internal state. They function as the coefficients that dictate how signals propagate through the layers of the architecture. Because they are frozen post-training, they represent the static probability distribution that governs the model's inference behavior until a new training or fine-tuning cycle occurs.
Weights are the high-dimensional tensor parameters optimized via backpropagation and gradient descent during the training phase. These billions of scalar values constitute the persistent state of the model, defining the transformation matrices applied to input embeddings. As static artifacts, they represent the crystallized probability distribution of the model's learned manifold; they remain immutable during inference and are only modified through subsequent weight updates in retraining or fine-tuning regimes.
evolution
- 1943 · historyMcCulloch-Pitts Neuron
Introduced the concept of adjustable numerical weights representing synaptic strength in a mathematical model of a neuron.
- 1958 · historyThe Perceptron
Frank Rosenblatt implemented the first algorithm to automatically learn weights through iterative error correction.
- 1986 · historyBackpropagation Popularization
Rumelhart, Hinton, and Williams demonstrated how to efficiently calculate weight updates across multi-layer networks using the chain rule.
- 2012 · historyAlexNet Breakthrough
Showcased the power of deep learning by training millions of weights on GPUs, setting the standard for modern large-scale parameter optimization.
- 2020 · historyGPT-3 Scale
Demonstrated the emergence of complex capabilities in models with 175 billion static weights, shifting focus to massive-scale parameter storage.