SIGNAL//DESK
AI/MLsrc: Roost FA glossary v2

Cost-per-token

What each token of input or output costs you — measured in fractions of a cent. High-volume automated tasks like parsing hundreds of client emails can accumulate costs if the prompt is inefficiently written.

Cost-per-token is the unit pricing model for LLM inference, where providers bill based on the total volume of input and output tokens. Because high-volume automation—such as processing thousands of emails—scales linearly with token count, optimizing prompt length and output tokens is essential to maintaining cost efficiency.

Cost-per-token is the granular billing metric for LLM API consumption, defined as the monetary value assigned to each discrete token processed during the inference lifecycle. Providers typically quote rates per million tokens (MTP). In high-throughput production environments, the cumulative cost is a function of the total token volume; therefore, prompt engineering and output constraint strategies are required to minimize the financial impact of inefficient token utilization.

evolution

  1. 2020-06 · history
    GPT-3 API Launch

    OpenAI introduces the first commercial API for LLMs, establishing the paradigm of charging per 1,000 tokens.

  2. 2023-03 · history
    GPT-4 Release

    OpenAI introduces tiered pricing for input and output tokens, significantly increasing the cost-per-token compared to previous models.

  3. 2023-11 · history
    GPT-4 Turbo

    OpenAI drastically reduces the cost-per-token for both input and output, setting a new industry standard for model affordability.

  4. 2024-05 · history
    GPT-4o Launch

    OpenAI introduces a new flagship model with a 50% price reduction compared to GPT-4 Turbo, further commoditizing token-based billing.


← all terms