Advanced RAG Masterclass

Take Retrieval-Augmented Generation from a demo to a production system: chunking and embeddings, vector indexes at scale, hybrid search, reranking and query transformation, rigorous evaluation, keeping knowledge fresh, agentic and graph-based RAG, memory, and multimodal documents.

Post Grad and Professional · English · 27 chapters

• Course Overview
• Why Retrieval? The Problem RAG Solves — Language models are fluent but closed-book: they cannot see private data, they do not know recent events, and they guess when unsure. RAG turns the exam into an open-book one. This chapter sets out the pipeline the rest of the course improves.
• Few-Shot Prompting and In-Context Learning — Before you retrieve anything or train anything, there is a cheaper lever: show the model a few worked examples inside the prompt. Nothing is learned permanently, but behaviour changes immediately — and understanding why explains what RAG is really doing.
• Chain-of-Thought Prompting — Some problems do not get easier with more examples — they need working space. Asking a model to reason step by step before answering turns a single compressed guess into a sequence of small, tractable steps, and it costs nothing but tokens.
• RAG, Fine-Tuning or Long Context? — There are three ways to give a model knowledge it lacks: retrieve it, train it in, or paste it all into a huge context window. They solve different problems. Choosing well depends on whether you need knowledge or behaviour, how fast the data changes, and how much of it there is.
• Chunking: How You Split Decides What You Find — Retrieval can only return what chunking created. Fixed-size, recursive, structure-aware and semantic splitting each make a different trade-off between precision and context. This chapter covers each method, the role of overlap, and how to choose a chunk size.
• Embeddings: Turning Meaning into Geometry — An embedding model maps text to a point in a high-dimensional space where similar meanings land close together. This chapter covers static versus contextual embeddings, the bi-encoder setup that makes retrieval fast, and what embeddings inevitably lose — which motivates reranking later.
• Vector Databases: Where Embeddings Live — A vector database stores each chunk as a record of ID, vector, text and metadata, and answers 'what is nearest to this vector?' quickly. This chapter covers the anatomy of a record, indexes and namespaces, upserts, and how to choose between managed and self-hosted stores.
• Measuring Similarity: Cosine, Euclidean and Dot Product — 'Nearest' needs a definition. Cosine similarity compares direction, Euclidean distance compares position, and the dot product mixes direction with magnitude. For normalised text embeddings all three rank results identically — and knowing why tells you when the choice actually matters.
• Vector Indexes: Flat, IVF and HNSW — Comparing a query with every stored vector is exact but cannot scale to millions. Approximate nearest-neighbour indexes trade a little recall for a lot of speed: IVF searches only the closest clusters, HNSW walks a layered graph. This chapter explains how each works and which settings control the speed–recall trade-off.
• Hybrid Search: Meaning Plus Keywords — Dense embeddings understand paraphrase but stumble on product codes, error IDs and jargon. Sparse keyword search is the opposite. Hybrid search runs both and fuses the rankings. This chapter covers TF-IDF, BM25 and reciprocal rank fusion, with a worked example.
• Metadata Filtering: Searching the Right Shelf — Similarity alone cannot know that a question is about this year's policy, the medical guide, or one customer's documents. Metadata filters constrain the search to the right subset before similarity ranks it. This chapter covers what to store, how pre- and post-filtering differ, and three production uses: scoping, versioning and citations.
• Reranking: A Second, Sharper Look — Vector similarity is a fast but lossy first pass, so the best chunk is often ranked fifth instead of first. A cross-encoder reranker reads the query and each candidate together and re-scores them precisely. This chapter covers the two-stage retrieve-then-rerank pattern, bi- versus cross-encoders, LLM rerankers, and the latency cost.
• Query Transformation: Fixing the Question Before the Search — Users ask vague, compound and oddly worded questions, and a single embedding of their exact words often retrieves poorly. Using an LLM to rewrite the query first — multi-query, RAG-Fusion, decomposition, step-back, HyDE and routing — makes retrieval far more robust.
• Measuring Retrieval: Recall, Precision, MRR, MAP and NDCG — If retrieval fails, nothing downstream can save the answer, so measure it first. This chapter defines the standard retrieval metrics, works through each by hand, explains what each one rewards, and shows how to build the labelled dataset they need.
• Evaluating the Whole Pipeline: Faithfulness, Relevancy and LLM Judges — Good retrieval is necessary but not sufficient: the model can still ignore the evidence, invent details or answer a different question. This chapter covers the core end-to-end metrics, how LLM-as-judge frameworks such as DeepEval and RAGAS compute them, and how to build and use a golden dataset.
• Tracing and Debugging a RAG Pipeline — Evaluation scores tell you that something is wrong; a trace tells you where. This chapter covers how observability tools such as LangSmith record every step of a request as a tree of runs, and gives a systematic method for turning one bad answer into a located, fixable cause.
• Keeping the Index Fresh: Updates Without Duplicates — Freshness is a key reason to choose RAG over fine-tuning, yet many pipelines can only append. This chapter covers deterministic chunk IDs and upserts, the orphan-chunk trap when documents shrink, content hashing to skip unchanged text, deletions, and full re-indexing when the embedding model changes.
• Agentic RAG: Retrieval That Checks Its Own Work — A fixed pipeline retrieves once and answers whatever it finds. An agentic system lets an LLM make decisions inside the pipeline: grade what was retrieved, rewrite the question and try again, or choose a different tool. This chapter covers the corrective loop and the state-graph model used to build it.
• Routing and Text-to-SQL: When the Answer Lives in a Table — 'What is the leave policy?' is a document question; 'how many leave days do I have left?' is a database question. A router sends each question to the right source, and a text-to-SQL path turns natural language into a safe, validated query. This chapter covers both, with the security rules a production system needs.
• Retrieval as Tools: MCP Servers and Deep Agents — The newest production pattern turns every retrieval strategy (semantic, keyword, hybrid, filtered) into a tool on an MCP server, and lets an agent choose among them. A 'deep agent' adds planning, a scratch workspace and reusable skills, so it can break down a five-part question, retrieve in parallel and answer in a consistent format.
• When Agents Get Stuck — An agent that loops forever, repeats the same search or never commits to an answer is failing for one of four reasons. Each has a recognisable signature in the trace, and the fix is usually in the prompt or the loop, not the model.
• GraphRAG: When Relationships Matter More Than Similarity — Vector search treats every chunk as an island, so questions that need facts joined across documents ('which companies is this person connected to?') fall through the gaps. GraphRAG extracts entities and relationships into a knowledge graph and combines graph traversal with vector search. This chapter covers how graphs are built, queried and used together with vectors.
• Memory: Assistants That Remember You — RAG retrieves from documents; memory retrieves from the assistant's own past with a user. This chapter covers the three kinds of long-term memory (semantic, episodic and procedural), how memories are organised as profiles or collections, what to store, and the decisions that matter: when to write, how to update, and when to forget.
• Memory in Practice: Hot-Path and Background Pipelines — This chapter turns memory theory into architecture. It walks through the two standard designs, as implemented by memory frameworks such as LangMem: a background pipeline that extracts memories after the conversation using schema-driven memory managers, and a hot-path agent that saves and searches memories through tools. It then sets out how to choose between them and the ways each one fails.
• Multimodal RAG: Tables, Figures and Scanned Pages — Much of a real document's information sits in tables and figures, which text-only pipelines discard. The key pattern is to embed a summary for search but send the original to the model. This chapter covers parsing, captioning, storage, retrieval and answering for text, tables and images, plus alternative designs.
• A Production RAG Blueprint — The capstone. We assemble every component from the course into one reference architecture (ingestion, retrieval, orchestration, generation, memory and observability), then give an order in which to adopt them and a checklist for taking a RAG system live.