Cost-per-token (property)
AI inference has a per-use cost. Different from owning software outright. Plan for it. Every interaction consumes resources that scale with the length of your prompts and the model's responses.
Cost-per-token is a variable usage model where expenses scale linearly with the volume of input and output data processed. Unlike traditional perpetual software licenses, this requires active financial planning because every interaction consumes compute resources proportional to the total token count.
Cost-per-token represents the granular economic unit of inference, where operational expenditure is directly coupled to the aggregate input and output token sequence length. This model necessitates a dynamic cost-management strategy, as the marginal cost of inference is non-zero and scales deterministically with the computational overhead of the prompt-response cycle.
evolution
- 2020-06 · historyGPT-3 API Launch
OpenAI introduced the first commercial API model, establishing the paradigm of charging users based on token consumption.
- 2022-11 · historyChatGPT Public Release
The massive scale of consumer adoption solidified the 'cost-per-token' model as the standard economic unit for LLM inference.
- 2023-03 · historyGPT-4 API Introduction
OpenAI introduced tiered pricing for input and output tokens, formalizing the distinction between prompt costs and generation costs.
- 2024-05 · historyGPT-4o Price Reduction
The release of GPT-4o marked a significant shift toward aggressive price competition, with costs dropping to $5 per million input tokens.