Test-time compute
Giving an AI more time to 'think' before it gives you an answer, similar to how a student might take extra time to double-check their math or outline an essay to avoid mistakes.
The practice of allocating additional computational resources during the inference phase—such as through chain-of-thought prompting, iterative sampling, or tree-of-thought search—to improve output accuracy at the expense of increased latency and operational cost.
Computation spent at inference (e.g. extended reasoning, sampling, or search) to improve answer quality, trading latency and cost for accuracy; central to reasoning models.
evolution
- 2017-06 · historyAttention Is All You Need
The Transformer architecture introduced the mechanism that allows for iterative processing and self-attention, the foundation for later test-time compute methods.
- 2022-06 · historyChain-of-Thought Prompting
Researchers demonstrated that prompting models to generate intermediate reasoning steps significantly improves performance on complex tasks.
- 2023-05 · historyTree of Thoughts
A framework was introduced allowing models to explore multiple reasoning paths and backtrack, formalizing the concept of deliberate test-time search.
- 2024-09 · historyOpenAI o1 Release
The first major commercial model explicitly optimized to perform extensive internal 'thinking' processes before outputting a final response.