Beginner path
- Read the definition of RAG.
- Study the pipeline from documents to citations.
- Learn the difference between RAG, fine-tuning, and long context.
- Use the glossary whenever a term is unfamiliar.
Learn RAG
Retrieval-Augmented Generation, from first principles to practice
Understand why RAG emerged, how it differs from fine-tuning and long-context models, and how modern RAG systems are designed.
Expert path
- Compare naive, advanced, modular, graph, agentic, and multimodal RAG.
- Design an evaluation set before changing models.
- Measure retrieval separately from generation.
- Track source freshness and access-control behavior.
What is RAG?
Retrieval-Augmented Generation connects a language model to external knowledge sources. The retriever finds relevant evidence, and the model uses that evidence to answer with better grounding, accuracy, traceability, and domain relevance.
Why RAG emerged
RAG emerged because model weights cannot reliably contain every private, current, or specialized fact. Retrieval lets knowledge be updated independently from the model.
RAG vs fine-tuning
RAG is generally suitable when information changes or must be cited. Fine-tuning is generally suitable for behavior, tone, formatting, or domain patterns rather than constantly changing facts.
RAG vs long-context models
Long context can hold more material, but retrieval still helps select relevant evidence, reduce cost, improve traceability, and manage very large collections.
Naive, advanced, modular, graph, agentic, and multimodal RAG
Naive RAG retrieves chunks directly. Advanced RAG adds query rewriting, hybrid search, reranking, and evaluation. Modular RAG separates components. Graph, agentic, and multimodal RAG add relationships, tool use, and multiple content types.
Core concepts
Important concepts include chunking, embeddings, vector databases, hybrid search, metadata, reranking, query rewriting, retrieval evaluation, hallucination control, and citations.
Knowledge-base design
A RAG system is only as good as the knowledge it can retrieve. Good knowledge-base design defines source authority, metadata, ownership, update cadence, permissions, document structure, and deprecation rules before indexing starts.
Retrieval quality
Retrieval quality depends on chunking, embeddings, keyword coverage, metadata filters, reranking, and query transformation. Inspect failed questions directly; dashboards alone rarely explain why a source was missed.
Citations and traceability
Citations should help users verify claims, not merely decorate answers. Strong systems preserve source IDs, page numbers, section titles, timestamps, permissions, and snippets throughout the pipeline.
Security and governance
RAG systems must handle access control, prompt injection, sensitive data, logging, retention, source licensing, and human escalation. Retrieved text should be treated as untrusted content that cannot override system policy.
Production RAG
Production RAG requires monitoring, test sets, trace review, feedback loops, source refresh workflows, incident handling, and clear ownership. A prototype can answer questions; a production system must be maintained.
Strengths and limitations
RAG improves grounding and updateability, but it can still fail through bad ingestion, weak retrieval, stale sources, prompt injection, poor evaluation, or unsupported generation.
Common mistakes
Common mistakes include chunking everything the same way, ignoring metadata, skipping evaluation, trusting top-k retrieval blindly, failing to manage permissions, and not showing sources to users.
Best practices
Start with a narrow use case, curate sources, keep metadata, evaluate retrieval and answers separately, use reranking for noisy corpora, add citations, monitor failures, and define human escalation.
Practical examples
A simple RAG example
A university policy assistant receives the question: 'Can graduate students borrow interlibrary loan books?' The retriever searches approved library policy pages, finds the relevant borrowing rule, and passes that passage to the model. The answer cites the policy page instead of relying on model memory.
- User asks a question
- Retriever searches approved sources
- Relevant passages are selected
- Model answers only from those passages
- UI shows citations
When RAG is the wrong tool
RAG is not a cure for every AI problem. If the task is pure classification, style transfer, translation, or extraction from a single provided document, retrieval may add unnecessary complexity.
- Check whether external knowledge is needed
- Check whether sources must be updated
- Check whether citations matter
- Choose simpler patterns when retrieval adds no value
A strong first project
The best first RAG project is narrow, source-rich, and easy to evaluate: an internal policy assistant, course-material Q&A bot, support documentation assistant, or research-paper explorer.
- Limit the domain
- Start with 20-100 trusted documents
- Write test questions before launch
- Review failures with subject experts
Prompting, long context, RAG, or fine-tuning?
These approaches solve different problems and can be combined. Choose according to the knowledge, behavior, traceability, and maintenance requirements of the task.
| Strategy | Choose when | Do not rely on it when | How it combines |
|---|---|---|---|
| Prompting | The model already has the needed knowledge and the main problem is instructions, format, tone, or reasoning scaffolding. | Answers require private, rapidly changing, or citable sources. | RAG prompts still define source use, uncertainty, answer format, and citations. |
| Long context | The relevant material is already known, fits comfortably, and must be considered together. | The corpus is much larger than the context window, changes frequently, or contains substantial irrelevant material. | Use retrieval to select a compact evidence set, then use long context for cross-document synthesis. |
| RAG | Knowledge is external, private, changing, too large for a single prompt, or must be traced to sources. | The task needs no external knowledge or retrieval cannot access an authoritative corpus. | Use prompting for behavior and fine-tuning for stable task patterns where justified. |
| Fine-tuning | The goal is stable behavior, style, output structure, domain language, or learned task patterns. | The main requirement is updating factual knowledge or producing verifiable citations. | A fine-tuned model can still retrieve current evidence through RAG. |
Chunking strategy guide
There is no universally correct chunk size. Treat chunking as a retrieval design decision and evaluate it on representative questions from the target corpus.
Fixed-size
- Best for:
- Fast baselines, uniform prose, and early experiments.
- Method:
- Split by a target token or character count, optionally with overlap.
- Watch for:
- Broken headings, tables, procedures, and arguments that cross chunk boundaries.
- Evaluate with:
- Expected-passage recall, duplicate retrieval, and answer completeness.
Structure-aware
- Best for:
- Policies, manuals, books, web pages, and documentation with meaningful headings.
- Method:
- Respect sections, paragraphs, lists, tables, functions, or other document-native boundaries.
- Watch for:
- Very long sections and parser errors that erase layout or hierarchy.
- Evaluate with:
- Section-level provenance, exact-reference queries, and human inspection.
Semantic
- Best for:
- Narrative or mixed-length content where topic boundaries matter more than layout.
- Method:
- Group adjacent sentences according to semantic similarity or model-assisted topic shifts.
- Watch for:
- Higher processing cost, unstable boundaries, and difficult reproducibility.
- Evaluate with:
- Intrachunk coherence, retrieval relevance, latency, and indexing cost.
Parent-child
- Best for:
- Cases that need precise matching but broader context for answer generation.
- Method:
- Retrieve small child chunks, then return a larger parent section to the model.
- Watch for:
- Repeated parent context, metadata linkage errors, and oversized prompts.
- Evaluate with:
- Child retrieval recall, parent usefulness, context redundancy, and token usage.
Entity or graph-based
- Best for:
- Relationship-heavy corpora involving people, organizations, events, concepts, or dependencies.
- Method:
- Create retrievable units around entities, relations, graph neighborhoods, or linked passages.
- Watch for:
- Extraction errors, graph maintenance cost, and false or missing relationships.
- Evaluate with:
- Entity coverage, relation accuracy, multi-hop questions, and provenance.
Where a RAG answer can fail
“The model hallucinated” is often too vague to be actionable. Diagnose the full chain, starting with whether the right source was available at all.
Source
Failure: The authoritative answer is absent, stale, duplicated, contradictory, or inaccessible.
Inspect: Inspect corpus coverage, ownership, dates, permissions, and superseded documents.
Improve: Curate sources before changing models or retrieval settings.
Parsing and chunking
Failure: The answer exists but tables, headings, pages, or relationships were damaged during ingestion.
Inspect: Compare parsed chunks with the original document visually.
Improve: Use format-aware parsing, structure-aware chunks, and preserved provenance.
Retrieval
Failure: Relevant evidence is not returned or is buried under irrelevant passages.
Inspect: Evaluate expected-source recall and relevance of each top result.
Improve: Tune chunking, hybrid search, metadata filters, query rewriting, and reranking.
Context assembly
Failure: Good evidence is truncated, duplicated, ordered poorly, or mixed with conflicting sources.
Inspect: Log the exact context sent to the model.
Improve: Deduplicate, prioritize authority and freshness, compress carefully, and expose conflicts.
Generation and reasoning
Failure: The model ignores, misreads, overgeneralizes, or invents beyond the supplied evidence.
Inspect: Break the answer into claims and map each claim to supporting context.
Improve: Improve instructions, require claim support, reduce noise, or use a better-suited model.
Citation and presentation
Failure: The answer may be supported, but users cannot verify which source supports which claim.
Inspect: Test citation correctness, granularity, links, page numbers, and source snippets.
Improve: Preserve provenance end to end and design citations as a verification workflow.
On this page
What is RAG?Why RAG emergedRAG vs fine-tuningRAG vs long-context modelsNaive, advanced, modular, graph, agentic, and multimodal RAGCore conceptsKnowledge-base designRetrieval qualityCitations and traceabilitySecurity and governanceProduction RAGStrengths and limitationsCommon mistakesBest practicesStrategy decision guideChunking strategy guideRAG failure chain
Trusted starting sources
Original RAG paperFoundational research paper for Retrieval-Augmented Generation.LangChain RAG tutorialOfficial implementation tutorial for indexing, retrieval, generation, and agents.LlamaIndex RAG guideOfficial conceptual and framework guide for data-connected LLM applications.Dify documentationOfficial source for Dify apps, knowledge bases, workflows, models, and deployment.Haystack documentationOfficial docs for component-based search, QA, and RAG pipelines.