NOTES / PART 1 · FOUNDATIONS

03RAG and retrieval

Bringing outside knowledge into the context window.

THIS SECTION COVERS

Chunking, embeddings, sparse (BM25), dense and hybrid search, reranking, query rewriting, agentic RAG, measuring retrieval quality.

Related: Prompting and context engineering · Evals and observability

TOPICS · 8/8 WRITTEN

  1. 03.1RAG overviewRetrieval-augmented generation in one picture: index your documents, fetch the relevant pieces at question time, and have the model answer from them.
  2. 03.2ChunkingSplitting documents into retrievable pieces: how chunk size and boundaries change what can be found, and how to restore the context chunks lose.
  3. 03.3Embeddings and vector searchTurning text into vectors so meaning becomes distance, and searching millions of them fast with approximate nearest-neighbour indexes such as HNSW.
  4. 03.4BM25 and sparse retrievalKeyword search done well: inverted indexes, BM25 scoring, and why this 1990s method is still a strong baseline and a vital partner to embeddings.
  5. 03.5Hybrid search and rerankingCombine keyword and vector retrieval, fuse the rankings with reciprocal rank fusion, then let a slower reranker pick the few passages that go in the prompt.
  6. 03.6Query rewritingFixing the question before searching: rewriting, decomposing and hypothetical-answer techniques that close the gap between how users ask and how documents are written.
  7. 03.7Agentic RAGLetting the model run the retrieval: decide whether to search, what to search for, and when it has enough, instead of one fixed retrieve-then-answer pass.
  8. 03.8Measuring retrievalHow to tell whether your RAG system works: score retrieval, grounding and the final answer separately, using real questions and a few well-chosen metrics.