Transfer learning
Transfer learning is like learning to play the guitar after already knowing how to play the piano; instead of starting from scratch, you use the musical skills you already have to learn the new instrument much faster.
Transfer learning involves taking a pre-trained neural network model and repurposing it for a new, related task by fine-tuning its weights, which significantly reduces the amount of training data and computational power required compared to training from scratch.
Transfer learning is a machine learning paradigm where a model developed for a source task is reused as the starting point for a model on a target task, leveraging learned feature representations to improve generalization performance and accelerate convergence in data-constrained environments.
evolution
- 1993 · historyMulti-task Learning
Rich Caruana formalizes the concept of learning multiple tasks simultaneously to improve generalization.
- 2005 · historyInductive Transfer
Thrun and Pratt publish 'Learning to Learn,' establishing the theoretical framework for transferring knowledge between domains.
- 2014 · historyCNN Feature Transfer
Donahue et al. demonstrate that features learned by deep convolutional neural networks on ImageNet can be effectively transferred to other visual tasks.
- 2018 · historyTransformer Pre-training
The release of BERT and GPT models establishes the paradigm of large-scale unsupervised pre-training followed by task-specific fine-tuning.