Retrieval-Augmented Generation (RAG)

Definition

Retrieval-Augmented Generation (RAG) is a pattern for building AI applications where relevant documents are retrieved from a knowledge source and inserted into an LLM's prompt at query time. The model then generates its response based on the retrieved context — grounding its answer in your data rather than relying only on what it learned during pretraining.

The pattern was introduced by Patrick Lewis, Ethan Perez, and colleagues in their 2020 paper Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. The insight was elegant: LLMs are excellent at synthesizing information but limited by what's in their weights. Retrieval systems are excellent at finding relevant documents but can't synthesize. Combine them, and you get grounded generation.

By 2026, RAG has become the most widely deployed pattern for LLM applications. Every customer-support bot backed by a knowledge base, every 'chat with your documents' feature, every research assistant that cites sources — they're all variations of RAG. The pattern has evolved: modern RAG systems use embeddings and vector databases for semantic retrieval, rerankers for quality, hybrid search for robustness, and agentic RAG patterns where the model decides what to retrieve.

RAG is often compared with fine-tuning, but they solve different problems. RAG is best for injecting up-to-date knowledge that changes frequently and for citation. Fine-tuning is best for consistent style, format, or reasoning patterns that don't change. In practice, most production systems use both — see our RAG vs Fine-tuning comparison.

Real-world example

Building a "chat with your docs" feature

You have 500 support articles and want users to ask questions in natural language. Without RAG, you'd need to fine-tune a model (expensive, brittle when docs change) or paste all 500 articles into every prompt (way over context window). With RAG: at indexing time, split each article into chunks, embed each chunk into a vector, store in a vector database. At query time, embed the user's question, find the 5 most similar chunks by cosine similarity, feed them to the LLM as context, and ask it to answer based only on those chunks. Result: grounded answers that cite specific articles, with your knowledge base staying easy to update.

# Minimal RAG in Python
from anthropic import Anthropic
from openai import OpenAI  # for embeddings

client = Anthropic()
embedder = OpenAI()

# 1. Retrieve — find relevant chunks
def retrieve(question, k=5):
    q_embedding = embedder.embeddings.create(
        input=question, model="text-embedding-3-large"
    ).data[0].embedding
    # Assume `vector_db.search()` returns k most similar chunks
    chunks = vector_db.search(q_embedding, k=k)
    return chunks

# 2. Augment — build the prompt
def build_prompt(question, chunks):
    context = "\n\n".join([f"[Doc {i+1}]\n{c.text}" for i, c in enumerate(chunks)])
    return f"""Use only the documents below to answer.
Cite sources by [Doc N].

DOCUMENTS:
{context}

QUESTION: {question}"""

# 3. Generate — pass to LLM
def rag_answer(question):
    chunks = retrieve(question)
    prompt = build_prompt(question, chunks)
    response = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

When you'll encounter this

• You need the model to answer from a specific document set — company docs, product manuals, legal contracts, research papers.
• You need citations and sources — RAG naturally produces answers you can trace back to specific documents.
• Your knowledge base changes frequently — retrieval means updates are as simple as re-indexing; no retraining required.
• You want to reduce hallucination — grounding the model in retrieved context dramatically cuts made-up answers.
• Your data is too big for the context window — even with long contexts, retrieval is often cheaper and faster than stuffing everything.

How it works

Common misconceptions

Related terms

Where you'll see this in practice

Learn more

• Read the deep-dive: How RAG Works (End-to-End)
• Take the Master RAG in 10 Lessons path
• Compare RAG vs Fine-tuning
• Browse Vector Databases & RAG category