Fine-tuning (model adaptation)
At a glance
Definition
Fine-tuning is the process of adapting a pretrained neural network — usually a foundation model — to perform better on a specific task, domain, style, or safety requirement. The base model has already learned general patterns from massive pretraining data; fine-tuning updates some or all of its parameters using a smaller, more targeted dataset.
The term covers a wide spectrum of techniques. Full fine-tuning updates every parameter — highest quality, highest cost. Parameter-efficient fine-tuning (PEFT) methods like LoRA and QLoRA update only a small fraction of parameters — achieving comparable quality at a fraction of the cost. Preference-based methods like RLHF and DPO use human or AI feedback to shape behavior. And reasoning-focused methods like GRPO train models to think through problems.
Fine-tuning is one of the two dominant patterns for specializing LLMs. The other is RAG. They solve different problems: fine-tuning teaches consistent style, format, or reasoning patterns; RAG injects up-to-date knowledge. In production, teams often use both — see our RAG vs Fine-tuning comparison.
In 2026, fine-tuning is more accessible than it has ever been. Tools like Axolotl, TRL, LlamaFactory, and Unsloth make it possible to fine-tune a Llama-3 8B model on a consumer GPU in an afternoon. Providers like OpenAI and Anthropic offer managed fine-tuning APIs. Open-weight releases from Meta, Mistral, DeepSeek, and Alibaba mean you have strong base models to start from.
Real-world example
Fine-tuning a customer support agent
Say your support team responds in a specific voice (empathetic, structured, always ending with next steps). You've collected 500 great past support responses. A generic LLM will help, but its tone will vary and it won't consistently produce your format. Fine-tune a small open-weight model (Llama-4 8B) on your 500 examples using LoRA. Training takes a few hours on a single H100 GPU. The result: a model that consistently responds in your voice, formatted your way, at a fraction of the API cost of a frontier model.
# Fine-tuning with LoRA using HuggingFace TRL
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig
from peft import LoraConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
# 1. Load base model and your dataset
model_id = "meta-llama/Llama-4-8B"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16")
tokenizer = AutoTokenizer.from_pretrained(model_id)
dataset = load_dataset("json", data_files="support_examples.jsonl")
# 2. Configure LoRA — trains ~0.5% of params
lora_config = LoraConfig(
r=16, # rank
lora_alpha=32, # scaling
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
# 3. Train
trainer = SFTTrainer(
model=model,
args=SFTConfig(
output_dir="./out",
num_train_epochs=3,
per_device_train_batch_size=4,
learning_rate=2e-4,
),
train_dataset=dataset["train"],
peft_config=lora_config,
tokenizer=tokenizer,
)
trainer.train()
trainer.save_model("./support-agent-lora")When you'll encounter this
- You need consistent style, tone, or format that prompting can't reliably produce.
- You have a well-defined narrow task (classification, extraction, transformation) where you want speed and accuracy.
- You want to reduce cost or latency by using a smaller fine-tuned model instead of a large general one.
- You need to encode domain knowledge that's specific to your industry (legal terminology, medical protocols, internal processes).
- You want to build a specialist that outperforms generalists on your specific task — often achievable with quality data.
How it works
Choose a base model
Start with a strong pretrained foundation model. Open-weight options: Llama-4, Mistral, Qwen, DeepSeek. Managed options: OpenAI GPT fine-tuning, Anthropic partner fine-tuning.
Prepare training data
Collect high-quality examples of the task you want the model to do. For chat models, this is (instruction, ideal_response) pairs. Quality matters far more than quantity — 500 great examples beat 10,000 mediocre ones.
Configure hyperparameters
Learning rate (usually 1e-5 to 2e-4 depending on method), batch size, number of epochs (often 2-3 for SFT), LoRA rank (typically 8-64). Sensible defaults exist for most tools.
Run training
Actual training. Depending on model size and method: minutes to hours on a single GPU (LoRA on 7B model) to days on multi-GPU clusters (full fine-tuning on 70B+).
Evaluate
Test on a held-out set. Compare against your baseline (usually the base model with prompting alone). Measure the metrics that matter for your use case, not just perplexity.
Iterate
Fine-tuning is rarely one-and-done. You'll iterate on data quality (usually the biggest lever), hyperparameters, and method choice. Track experiments carefully.
Deploy
Fine-tuned models can be deployed via provider APIs (managed) or self-hosted. LoRA weights can be swapped in and out without loading a new base model.
Common misconceptions
Not usually. Well-designed prompts on a strong base model often match or beat naïve fine-tuning. Fine-tuning wins when you need consistency, want to encode complex behavior, or need cost/speed improvements. See our comparison.
No. LoRA can produce meaningful adaptation with 100-1,000 high-quality examples. Full fine-tuning wants more, but quality still dominates quantity. One great example beats ten mediocre ones.
Not since 2023. QLoRA + Unsloth + a $2/hour cloud GPU means anyone can fine-tune a 7-8B model on a small dataset in an afternoon. Frontier fine-tuning is still capital-intensive, but everyday fine-tuning is accessible.
Related terms
Where you'll see this in practice
Learn more
- Read the deep-dive: How Fine-tuning Works
- Take the Master Fine-Tuning in 12 Lessons path
- Compare RAG vs Fine-tuning
- Browse Fine-tuning Methods category
Sources & further reading
- LoRA: Low-Rank Adaptation of Large Language Models — Hu et al., 2021. Read →
- QLoRA: Efficient Finetuning of Quantized LLMs — Dettmers et al., 2023. Read →
- Direct Preference Optimization — Rafailov et al., 2023. Read →
- InstructGPT: Training language models to follow instructions with human feedback — Ouyang et al., 2022 (RLHF). Read →
Frequently asked about Fine-tuning
Usually RAG first. Fine-tune when you need consistent style, a specific format, or knowledge that's hard to retrieve cleanly. Often the answer is both. See our detailed comparison.
LoRA on a 7-8B model: $10-100 in cloud GPU time. Full fine-tuning on a 70B model: hundreds to thousands. Managed fine-tuning APIs (OpenAI, Anthropic) charge based on tokens processed. Small experiments are affordable; frontier fine-tuning is not.
For LoRA on a strong base model: often 100-1,000 high-quality examples produces useful results. For full fine-tuning: more data helps. Quality matters more than quantity — one great example beats ten mediocre ones.
OpenAI offers fine-tuning for some GPT models. Anthropic offers Claude fine-tuning through partners. Most cutting-edge experimental fine-tuning happens on open-weight models (Llama, Mistral, Qwen, DeepSeek) because you have full control.
Under-investing in data. Teams often obsess over hyperparameters and method choice when the biggest lever is dataset quality. Spend most of your time curating clean, diverse, representative examples of the behavior you want. Everything else is smaller.