What is fine-tuning?

A language model finishes pre-training knowing a great deal and doing very little of it consistently. Fine-tuning is the step that addresses the second problem without restarting the first.

What fine-tuning changes

Pre-training exposes a model to a massive general corpus, teaching it statistical patterns of language across an enormous range of topics. The result is a model that produces fluent text but has no particular reason to follow instructions or maintain a consistent output format. Fine-tuning continues training from those learned weights on a much smaller, curated dataset of labeled examples.

The mechanism is unchanged: gradient descent adjusts weights in response to a loss signal. What differs is scale and purpose. Fine-tuning datasets measure in thousands of examples; pre-training data measures in billions of tokens. The aim is not to teach the model what language is but to shift its default behavior toward a specific task or style.

Behavior and knowledge are different problems

Fine-tuning shapes how a model responds. It does not reliably add or update facts.

InstructGPT is the clearest demonstration. Ouyang et al. fine-tuned a GPT-3 base model on human demonstrations and then applied reinforcement learning from human feedback. The result was a model that followed user intent more reliably, with “improvements in truthfulness and reductions in toxic output generation.” In human evaluations, outputs from the 1.3-billion-parameter tuned model were preferred over outputs from the raw 175-billion-parameter GPT-3 (Ouyang et al., 2022). The tuned model knew nothing new. It answered better.

Facts learned through fine-tuning are baked into weights, frozen at training time and without a source trail. They cannot be updated without another training run. For factual accuracy on current or proprietary data, RAG vs fine-tuning covers the decision axis in full.

Pre-training vs fine-tuning

Pre-training builds the foundation from scratch, training on billions of tokens and typically requiring thousands of GPU-hours. Fine-tuning assumes that foundation exists and starts from the learned weights.

Starting from pre-trained weights is what makes fine-tuning tractable. The model already understands grammar and context across a vast range of topics. Fine-tuning adjusts how those existing capabilities get expressed, consistently, on a narrower target. Training from scratch would mean re-learning everything; fine-tuning moves only the weights that need to move.

Parameter-efficient fine-tuning

Full fine-tuning updates every parameter in the model. At the scale of GPT-3, the authors of LoRA called this “prohibitively expensive” (Hu et al., 2021). Running a separate full fine-tune for each application, and storing each resulting 175-billion-parameter model, is not feasible for most organizations.

Parameter-efficient fine-tuning (PEFT) addresses this by training only a fraction of parameters while leaving the rest frozen. LoRA (Low-Rank Adaptation) freezes the pre-trained weights and injects trainable low-rank matrices into each transformer layer. The original LoRA paper reports reducing trainable parameters by up to 10,000x compared to full fine-tuning, cutting GPU memory use by roughly 3x, while matching fine-tuning quality on standard benchmarks (Hu et al., 2021).

PEFT brought fine-tuning within reach for teams without large-scale compute. A task that previously required the budget of a large lab now runs on a single GPU workstation.

When fine-tuning is the right tool

Prompt engineering should come first. Giving the model clear instructions at inference requires no training data and often closes most of the gap between base model behavior and what a specific application needs. Fine-tuning earns its cost when the task is stable, the target behavior is well-defined, and you have a corpus of labeled examples of the right output.

Fine-tuning’s practical edge is consistency and inference cost. A smaller fine-tuned model can match a larger prompted model on a narrow task and serve queries at a fraction of the cost. One training run now, cheaper inference from that point forward. That trade-off is the clearest case for it: a stable, well-scoped task, run at scale.

The capability must already exist

This is the point most explainers skip: fine-tuning cannot create a capability the base model does not already have. It shifts which capabilities surface by default when a particular kind of input arrives and makes those responses more consistent. If the base model cannot perform the underlying task, fine-tuning on task examples will not build that ability from the weights up.

That matters practically. When fine-tuning fails to deliver, the most common root cause is not insufficient training examples but a gap in the base model. Debugging starts there.

Sources: Ouyang et al., Training language models to follow instructions with human feedback (arXiv:2203.02155, 2022); Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685, 2021).