Generation, Advanced RAG & Evaluation

The final stage of the RAG pipeline is Generation, where the retrieved chunks are formatted into a prompt for the LLM. In production, simple RAG systems must be optimized using Advanced RAG architectures and evaluated using RAG metrics to ensure quality.

1. End-to-End RAG Synthesis Pipeline

We compose the retriever, augmented prompt template, and Chat Model using LangChain Expression Language (LCEL). Sample File (policy.txt):

Reimbursements: Submit all travel receipts by the 25th of each month. Submissions after this date will be processed in the following month's cycle.

Health Policy: Sick leave of more than 2 consecutive days requires a medical certificate. Report your absence to your supervisor before 9:00 AM on the first day.

1.1 Step-by-Step Implementation

Stage 1: Initialize Retriever

Load the document using TextLoader, chunk it, generate embeddings, and load them into a Chroma vector database configured as a retriever.

from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenAIEmbeddings
from langchain_community.vectorstores import Chroma

# Load the file
loader = TextLoader("policy.txt")
documents = loader.load()

# Chunk the policy text
splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=15)
docs = splitter.split_documents(documents)

# Embed and store in Chroma DB
embeddings = GoogleGenAIEmbeddings(model="models/text-embedding-004")
vector_db = Chroma.from_documents(docs, embeddings)

# Expose as a retriever fetching top-1 match
retriever = vector_db.as_retriever(search_kwargs={"k": 1})

print(f"Retriever configured with {len(docs)} document chunks.")

Sample Output:

Retriever configured with 2 document chunks.

Stage 2: Define Prompt Template

Create a prompt template that strictly instructs the model to answer queries based solely on the retrieved context.

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer the question using ONLY the provided context:\n\n{context}"),
    ("human", "{question}")
])

print("Prompt template defined.")

Sample Output:

Prompt template defined.

Stage 3: Initialize Chat Model and Output Parser

Initialize the Chat Model and string output parser.

from langchain.chat_models import init_chat_model
from langchain_core.output_parsers import StrOutputParser

llm = init_chat_model("gemini-2.5-flash", model_provider="google_genai")
output_parser = StrOutputParser()

print("Model and parser initialized.")

Sample Output:

Model and parser initialized.

Stage 4: Construct and Execute RAG Chain

Compose the retriever, formatter function, prompt, model, and parser into a unified pipeline using LCEL.

def format_docs(docs):
    return "\n\n".join([doc.page_content for doc in docs])

# LCEL Chain definition
rag_chain = (
    {
        "context": retriever | format_docs,
        "question": lambda x: x
    }
    | prompt
    | llm
    | output_parser
)

# Invoke the chain
response = rag_chain.invoke("When should I hand in travel receipts?")
print(f"Response: {response}")

Sample Output:

Response: You should submit your travel expense receipts by the 25th.

How the LCEL Chain Functions

The chain uses LangChain Expression Language (LCEL) to pipe operations sequentially using the | operator:

Input Map (dict): The incoming query string (e.g., "When should I hand in travel receipts?") is processed in parallel:
- "context": Passes the query to the retriever to fetch matching documents, then pipes them into format_docs to merge the page contents into a single string.
- "question": Uses a lambda function to pass the original query string through unmodified.
Prompt formatting (prompt): The dictionary containing the populated context and question is passed to the ChatPromptTemplate, which generates a list of formatted system and human messages.
Inference (llm): The formatted messages are sent to the Google GenAI Chat Model (gemini-2.5-flash) to generate a completion.
Parsing (output_parser): The model’s response is passed to the StrOutputParser to extract the raw string text from the chat message object.

1.2 Assembled Complete Script

Here is the complete, unified script combining all stages:

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.chat_models import init_chat_model

# 1. Load API keys
load_dotenv()

# 2. Ingest Data and Setup Retriever
loader = TextLoader("policy.txt")
documents = loader.load()

splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=15)
docs = splitter.split_documents(documents)

embeddings = GoogleGenAIEmbeddings(model="models/text-embedding-004")
vector_db = Chroma.from_documents(docs, embeddings)
retriever = vector_db.as_retriever(search_kwargs={"k": 1})

# 3. Define Prompt Template
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer the question using ONLY the provided context:\n\n{context}"),
    ("human", "{question}")
])

# 4. Initialize LLM
llm = init_chat_model("gemini-2.5-flash", model_provider="google_genai")

# Helper function to format retrieved documents
def format_docs(docs):
    return "\n\n".join([doc.page_content for doc in docs])

# 5. Build RAG Chain using LCEL
rag_chain = (
    {
        "context": retriever | format_docs,
        "question": lambda x: x
    }
    | prompt
    | llm
    | StrOutputParser()
)

# 6. Execute and Query the Chain
print(rag_chain.invoke("When should I hand in travel receipts?"))

2. Types of RAG Architectures

As RAG applications scale, they transition through three architectural paradigms:

2.1 Naive RAG

The standard pipeline: Ingest

\rightarrow

Embed

\rightarrow

Retrieve

\rightarrow

Generate.

Limitations: Low retrieval precision (retrieving noise), poor recall (missing details), and model hallucinations if the context is too long.

2.2 Advanced RAG

Introduces optimizations before and after retrieval to improve answer relevance:

Hybrid Search: Combines keyword search (BM25) with vector search (semantic) to find both exact term matches (e.g. product IDs) and conceptual synonyms.
Reranking: Uses a secondary Cross-Encoder model to calculate exact relevance scores for the top-N retrieved documents, sorting the most important context to the top before sending it to the LLM.
Metadata Filtering: Restricts searches to specific tags (e.g. {"department": "HR"} or {"year": 2026}), preventing the retriever from pulling irrelevant documents.

2.3 Agentic RAG

Uses LLMs as agents that decide when to query databases, reformulate queries, and self-correct answers if the retrieved data is insufficient.

3. RAG Evaluation Metrics

To measure RAG performance (using frameworks like Ragas or TruLens), systems are evaluated across two halves of the pipeline:

               ┌───────────────────────┐
               │      User Query       │
               └──────────┬────────────┘
                           │
             Context       │       Answer
            Precision      │      Relevance
          & Context Recall │    & Faithfulness
                           ▼
               ┌───────────────────────┐
               │    Vector Context     │
               └───────────────────────┘

3.1 Retrieval Metrics

Context Recall: Measures if the retriever found all the necessary facts needed to answer the question.
Context Precision: Measures if the retrieved chunks are highly relevant, or if they contain too much irrelevant noise.

3.2 Generation Metrics

Faithfulness (Groundedness): Measures if the LLM’s response is based only on the retrieved context. A high score means no hallucinations.
Answer Relevance: Measures if the generated response directly answers the user’s question, rather than talking about unrelated topics.

4. Practice Exercises

Practice 1: Identifying RAG Failure Modes

Identify which evaluation metric is failing in the following scenarios:

The LLM answers a query by making up facts that were not in the retrieved documents.
The user asks about sick leave policies, but the database retriever returns documents about office lunch hours, leading to a blank answer.

Solution

Faithfulness (Groundedness) is failing, because the model is hallucinating facts outside of the provided context.
Context Recall is failing (and consequently Context Precision is low), because the retriever failed to find the correct sick leave documents.

Summary

LCEL RAG Chains: Composes retrieval, prompt augmentation, LLM inference, and string parsing into a single pipeline with parallel data flow execution.
RAG Paradigms: Progresses from Naive RAG (linear indexing/retrieval) to Advanced RAG (introducing hybrid search, metadata filters, and cross-encoder reranking) and Agentic RAG (autonomous decision loops).
RAG Evaluation Metrics: Measured across two stages:
- Retrieval: Context Recall (finding all relevant facts) and Context Precision (avoiding irrelevant noise).
- Generation: Faithfulness/Groundedness (avoiding hallucinations) and Answer Relevance (directness of response).

Foundations

RAG Implementation

Generation, Advanced RAG & Evaluation

1. End-to-End RAG Synthesis Pipeline

1.1 Step-by-Step Implementation

Stage 1: Initialize Retriever

Stage 2: Define Prompt Template

Stage 3: Initialize Chat Model and Output Parser

Stage 4: Construct and Execute RAG Chain

How the LCEL Chain Functions

1.2 Assembled Complete Script

2. Types of RAG Architectures

2.1 Naive RAG

2.2 Advanced RAG

2.3 Agentic RAG

3. RAG Evaluation Metrics

3.1 Retrieval Metrics

3.2 Generation Metrics

4. Practice Exercises

Practice 1: Identifying RAG Failure Modes

Summary

​1. End-to-End RAG Synthesis Pipeline

​1.1 Step-by-Step Implementation

Stage 1: Initialize Retriever

Stage 2: Define Prompt Template

Stage 3: Initialize Chat Model and Output Parser

Stage 4: Construct and Execute RAG Chain

How the LCEL Chain Functions

​1.2 Assembled Complete Script

​2. Types of RAG Architectures

​2.1 Naive RAG

​2.2 Advanced RAG

​2.3 Agentic RAG

​3. RAG Evaluation Metrics

​3.1 Retrieval Metrics

​3.2 Generation Metrics

​4. Practice Exercises

​Practice 1: Identifying RAG Failure Modes

​Summary

1. End-to-End RAG Synthesis Pipeline

1.1 Step-by-Step Implementation

1.2 Assembled Complete Script

2. Types of RAG Architectures

2.1 Naive RAG

2.2 Advanced RAG

2.3 Agentic RAG

3. RAG Evaluation Metrics

3.1 Retrieval Metrics

3.2 Generation Metrics

4. Practice Exercises

Practice 1: Identifying RAG Failure Modes

Summary