Generation, Advanced RAG & Evaluation

The final stage of the RAG pipeline is Generation, where the retrieved chunks are formatted into a prompt for the LLM. In production, simple RAG systems must be optimized using Advanced RAG architectures and evaluated using RAG metrics to ensure quality.

1. End-to-End RAG Synthesis Pipeline

We compose the retriever, augmented prompt template, and Chat Model using LangChain Expression Language (LCEL):

import os
from dotenv import load_dotenv
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.chat_models import init_chat_model

load_dotenv()

# 1. Ingest Data
policies = "Sick leave of more than 2 days requires a medical certificate. Submit travel expense receipts by the 25th."
splitter = RecursiveCharacterTextSplitter(chunk_size=80, chunk_overlap=10)
docs = splitter.create_documents([policies])
embeddings = GoogleGenAIEmbeddings(model="models/text-embedding-004")
vector_db = Chroma.from_documents(docs, embeddings)
retriever = vector_db.as_retriever(search_kwargs={"k": 1})

# 2. Define Prompt Template
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer the question using ONLY the provided context:\n\n{context}"),
    ("human", "{question}")
])

llm = init_chat_model("gemini-2.5-flash", model_provider="google_genai")

def format_docs(docs):
    return "\n\n".join([doc.page_content for doc in docs])

# 3. Build RAG Chain
rag_chain = (
    {
        "context": retriever | format_docs,
        "question": lambda x: x
    }
    | prompt
    | llm
    | StrOutputParser()
)

# 4. Invoke
print(rag_chain.invoke("When should I hand in travel receipts?"))

2. Types of RAG Architectures

As RAG applications scale, they transition through three architectural paradigms:

2.1 Naive RAG

The standard pipeline: Ingest

\rightarrow

Embed

\rightarrow

Retrieve

\rightarrow

Generate.

Limitations: Low retrieval precision (retrieving noise), poor recall (missing details), and model hallucinations if the context is too long.

2.2 Advanced RAG

Introduces optimizations before and after retrieval to improve answer relevance:

Hybrid Search: Combines keyword search (BM25) with vector search (semantic) to find both exact term matches (e.g. product IDs) and conceptual synonyms.
Reranking: Uses a secondary Cross-Encoder model to calculate exact relevance scores for the top-N retrieved documents, sorting the most important context to the top before sending it to the LLM.
Metadata Filtering: Restricts searches to specific tags (e.g. {"department": "HR"} or {"year": 2026}), preventing the retriever from pulling irrelevant documents.

2.3 Agentic RAG

Uses LLMs as agents that decide when to query databases, reformulate queries, and self-correct answers if the retrieved data is insufficient.

3. RAG Evaluation Metrics

To measure RAG performance (using frameworks like Ragas or TruLens), systems are evaluated across two halves of the pipeline:

               ┌───────────────────────┐
               │      User Query       │
               └──────────┬────────────┘
                          │
            Context       │       Answer
           Precision      │      Relevance
         & Context Recall │    & Faithfulness
                          ▼
               ┌───────────────────────┐
               │    Vector Context     │
               └───────────────────────┘

3.1 Retrieval Metrics

Context Recall: Measures if the retriever found all the necessary facts needed to answer the question.
Context Precision: Measures if the retrieved chunks are highly relevant, or if they contain too much irrelevant noise.

3.2 Generation Metrics

Faithfulness (Groundedness): Measures if the LLM’s response is based only on the retrieved context. A high score means no hallucinations.
Answer Relevance: Measures if the generated response directly answers the user’s question, rather than talking about unrelated topics.

4. Practice Exercises

Practice 1: Identifying RAG Failure Modes

Identify which evaluation metric is failing in the following scenarios:

The LLM answers a query by making up facts that were not in the retrieved documents.
The user asks about sick leave policies, but the database retriever returns documents about office lunch hours, leading to a blank answer.

Solution

Faithfulness (Groundedness) is failing, because the model is hallucinating facts outside of the provided context.
Context Recall is failing (and consequently Context Precision is low), because the retriever failed to find the correct sick leave documents.

Foundations

RAG Implementation

Generation, Advanced RAG & Evaluation

1. End-to-End RAG Synthesis Pipeline

2. Types of RAG Architectures

2.1 Naive RAG

2.2 Advanced RAG

2.3 Agentic RAG

3. RAG Evaluation Metrics

3.1 Retrieval Metrics

3.2 Generation Metrics

4. Practice Exercises

Practice 1: Identifying RAG Failure Modes

​1. End-to-End RAG Synthesis Pipeline

​2. Types of RAG Architectures

​2.1 Naive RAG

​2.2 Advanced RAG

​2.3 Agentic RAG

​3. RAG Evaluation Metrics

​3.1 Retrieval Metrics

​3.2 Generation Metrics

​4. Practice Exercises

​Practice 1: Identifying RAG Failure Modes

1. End-to-End RAG Synthesis Pipeline

2. Types of RAG Architectures

2.1 Naive RAG

2.2 Advanced RAG

2.3 Agentic RAG

3. RAG Evaluation Metrics

3.1 Retrieval Metrics

3.2 Generation Metrics

4. Practice Exercises

Practice 1: Identifying RAG Failure Modes