Stochasticity
Built-in randomness. The technical word for 'not deterministic'. This means you cannot expect the exact same output twice, even with the same prompt.
In AI, stochasticity refers to the non-deterministic nature of model inference. Because the model samples from a probability distribution of potential next tokens rather than always picking the highest-probability one, the output varies across runs even when the input prompt and parameters remain constant.
Stochasticity denotes the inherent randomness in the token-sampling process, where the model's output is a stochastic variable derived from a probability distribution over the vocabulary. Given identical input prompts, the system produces divergent outputs due to the sampling mechanism—governed by hyperparameters like temperature, top-p, and top-k—which prevents deterministic mapping between input and output.
evolution
- 1950 · historyTuring Test Proposal
Alan Turing introduced the concept of a 'random element' in machine behavior to simulate human-like unpredictability.
- 1986 · historyBoltzmann Machines
Hinton and Sejnowski introduced stochastic neural networks that use probabilistic units to learn internal representations.
- 2014 · historyGenerative Adversarial Networks (GANs)
Goodfellow et al. popularized the use of latent noise vectors to inject stochasticity into generative model outputs.
- 2017 · historyTransformer Architecture
The introduction of 'temperature' parameters in softmax layers allowed for controlled stochastic sampling in large language models.