Every way to fine-tune an LLM
SFT, LoRA, QLoRA, DPO, GRPO, RLHF, and everything in between — every technique for adapting a pretrained model to your use case, with when to use each.
Fine-tuning is how you take a general-purpose foundation model and specialize it for your task, domain, style, or safety requirements. The last three years have seen an explosion of fine-tuning methods — LoRA and QLoRA made it affordable, RLHF and DPO made it aligned, and GRPO and reasoning-focused methods pushed the frontier.
These 50 terms are grouped into four sub-topics: parameter-efficient methods (LoRA family), instruction and preference training (SFT, RLHF, DPO), reinforcement-learning methods for reasoning (PPO, GRPO), and the data-preparation vocabulary you need to run any of it. For the big picture, see our How Fine-Tuning Works tutorial and the RAG vs Fine-tuning comparison.
The most important terms in Fine-tuning Methods
Start here if you're new. These entries explain the foundational vocabulary in depth.
Fine-tuning
Adapting a pretrained model to a specific task, domain, or behavior. Umbrella term covering everything from full fine-tuning to LoRA to RLHF.
Read the full entry →PEFTLoRA (Low-Rank Adaptation)
A parameter-efficient method that freezes the base model and inserts small trainable low-rank matrices. Cuts memory and compute cost by orders of magnitude while achieving comparable quality.
Read the full entry →PEFTQLoRA
LoRA combined with 4-bit quantization of the base model. Lets you fine-tune 65B+ parameter models on a single GPU.
Read the full entry →AlignmentRLHF
Reinforcement Learning from Human Feedback — training a reward model from human preferences, then using PPO to optimize the LLM against it. Turned base models into helpful chat models.
Read the full entry →AlignmentDirect Preference Optimization (DPO)
A simpler alternative to RLHF that skips the reward model entirely — optimizes directly on preference pairs. Cheaper and often works as well.
Read the full entry →ReasoningGRPO
Group Relative Policy Optimization — a PPO variant used by DeepSeek to train reasoning models. Enables strong math and coding capabilities without a value function.
Read the full entry →All Fine-tuning Methods terms, organized
Grouped into sub-topics so you can find neighbors and prerequisites, not just alphabetical entries.
Parameter-efficient fine-tuning (PEFT)
Instruction & supervised training
Preference & RL methods
Data & operations
Related term categories
These categories connect naturally to Fine-tuning Methods — many terms cross-reference between them.
Go deeper on Fine-tuning Methods
📖 Related concept tutorials
Long-form guides that walk through how these concepts actually work.
- How Fine-Tuning Works
SFT, LoRA, RLHF, DPO in one walkthrough.
- How LoRA Works
Low-rank matrices explained with math.
- How RLHF Works
Reward model, PPO, and everything in between.
🎯 Related models & tools
Real products and models where you'll encounter these terms.
- RAG vs Fine-tuning
When each is the right choice.
- Best open-weight models
The models you can actually fine-tune.
- Fine-tuning tools
Axolotl, TRL, LlamaFactory, Unsloth.
Questions about Fine-tuning Methods
Usually RAG first. Fine-tune when you need consistent style, a specific format, or knowledge that's hard to retrieve. Often the answer is both — see our RAG vs Fine-tuning comparison.
QLoRA on a small open-weight model. For 7B-8B parameter models, you can fine-tune on a single consumer GPU. For 70B+, still one GPU with QLoRA — training just takes longer.
No. LoRA can produce meaningful adaptation with 100-1,000 high-quality examples. Full fine-tuning usually wants more. Quality matters more than quantity — one great example beats ten mediocre ones.
Not always, but often. DPO is much simpler and cheaper. RLHF has more knobs to tune and can achieve slightly better quality on complex preferences. For most teams, DPO is the practical choice.
OpenAI offers fine-tuning for some GPT models. Anthropic offers fine-tuning through partners. Most cutting-edge fine-tuning research happens on open-weight models — Llama, Mistral, Qwen — because you have full control.