SIGNAL//DESK
AI/MLsrc: Roost FA glossary v2

Fine-tuning

Retraining a model on your own examples to bias it toward a domain. This is rarely necessary for FAs; usually, a well-crafted system prompt with few-shot examples achieves the same result without the cost and complexity.

Fine-tuning involves updating a pre-trained model's weights on a domain-specific dataset to shift its output distribution. However, it introduces significant operational overhead and data management costs. For most functional applications, prompt engineering—specifically utilizing robust system instructions and few-shot prompting—is the preferred approach to achieve domain alignment without the complexity of model retraining.

Fine-tuning is the process of continuing the gradient-based optimization of a pre-trained model's parameters on a curated, task-specific corpus to minimize loss on a target domain. While effective for deep behavioral shifts, it is often an over-engineered solution for functional applications (FAs). In practice, in-context learning via well-crafted system prompts and few-shot exemplars typically yields equivalent performance gains while avoiding the computational costs, latency, and maintenance burdens associated with weight-level adaptation.

evolution

  1. 2014 · history
    Transfer Learning Emergence

    Researchers began demonstrating that pre-trained neural networks could be repurposed for new tasks by updating weights on smaller, domain-specific datasets.

  2. 2018 · history
    ULMFiT and BERT

    The introduction of Universal Language Model Fine-tuning (ULMFiT) and BERT popularized the paradigm of pre-training on massive corpora followed by task-specific fine-tuning.

  3. 2020 · history
    GPT-3 and In-Context Learning

    The release of GPT-3 shifted the focus toward few-shot prompting, demonstrating that massive models could perform tasks without weight updates.

  4. 2022 · history
    RLHF Integration

    Reinforcement Learning from Human Feedback (RLHF) became the standard fine-tuning method for aligning large language models with human intent.

  5. 2023 · history
    PEFT and LoRA

    Parameter-Efficient Fine-Tuning (PEFT) techniques like LoRA emerged, allowing users to adapt large models with minimal computational overhead.

← all terms