> ## Documentation Index
> Fetch the complete documentation index at: https://genai.codewithsiva.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Generation, Advanced RAG & Evaluation

> Build RAG synthesis chains, explore advanced search optimizations, and evaluate systems using core RAG metrics

The final stage of the RAG pipeline is **Generation**, where the retrieved chunks are formatted into a prompt for the LLM. In production, simple RAG systems must be optimized using **Advanced RAG** architectures and evaluated using **RAG metrics** to ensure quality.

### 1. End-to-End RAG Synthesis Pipeline

We compose the retriever, augmented prompt template, and Chat Model using LangChain Expression Language (LCEL).

**Sample File (`policy.txt`):**

```text theme={null}
Reimbursements: Submit all travel receipts by the 25th of each month. Submissions after this date will be processed in the following month's cycle.

Health Policy: Sick leave of more than 2 consecutive days requires a medical certificate. Report your absence to your supervisor before 9:00 AM on the first day.
```

#### 1.1 Step-by-Step Implementation

##### Stage 1: Initialize Retriever

Load the document using `TextLoader`, chunk it, generate embeddings, and load them into a Chroma vector database configured as a retriever.

```python theme={null}
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenAIEmbeddings
from langchain_community.vectorstores import Chroma

# Load the file
loader = TextLoader("policy.txt")
documents = loader.load()

# Chunk the policy text
splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=15)
docs = splitter.split_documents(documents)

# Embed and store in Chroma DB
embeddings = GoogleGenAIEmbeddings(model="models/text-embedding-004")
vector_db = Chroma.from_documents(docs, embeddings)

# Expose as a retriever fetching top-1 match
retriever = vector_db.as_retriever(search_kwargs={"k": 1})

print(f"Retriever configured with {len(docs)} document chunks.")
```

**Sample Output:**

```text theme={null}
Retriever configured with 2 document chunks.
```

##### Stage 2: Define Prompt Template

Create a prompt template that strictly instructs the model to answer queries based solely on the retrieved context.

```python theme={null}
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer the question using ONLY the provided context:\n\n{context}"),
    ("human", "{question}")
])

print("Prompt template defined.")
```

**Sample Output:**

```text theme={null}
Prompt template defined.
```

##### Stage 3: Initialize Chat Model and Output Parser

Initialize the Chat Model and string output parser.

```python theme={null}
from langchain.chat_models import init_chat_model
from langchain_core.output_parsers import StrOutputParser

llm = init_chat_model("gemini-2.5-flash", model_provider="google_genai")
output_parser = StrOutputParser()

print("Model and parser initialized.")
```

**Sample Output:**

```text theme={null}
Model and parser initialized.
```

##### Stage 4: Construct and Execute RAG Chain

Compose the retriever, formatter function, prompt, model, and parser into a unified pipeline using LCEL.

```python theme={null}
def format_docs(docs):
    return "\n\n".join([doc.page_content for doc in docs])

# LCEL Chain definition
rag_chain = (
    {
        "context": retriever | format_docs,
        "question": lambda x: x
    }
    | prompt
    | llm
    | output_parser
)

# Invoke the chain
response = rag_chain.invoke("When should I hand in travel receipts?")
print(f"Response: {response}")
```

**Sample Output:**

```text theme={null}
Response: You should submit your travel expense receipts by the 25th.
```

##### How the LCEL Chain Functions

The chain uses LangChain Expression Language (LCEL) to pipe operations sequentially using the `|` operator:

1. **Input Map (`dict`)**: The incoming query string (e.g., `"When should I hand in travel receipts?"`) is processed in parallel:
   * `"context"`: Passes the query to the `retriever` to fetch matching documents, then pipes them into `format_docs` to merge the page contents into a single string.
   * `"question"`: Uses a lambda function to pass the original query string through unmodified.
2. **Prompt formatting (`prompt`)**: The dictionary containing the populated `context` and `question` is passed to the `ChatPromptTemplate`, which generates a list of formatted system and human messages.
3. **Inference (`llm`)**: The formatted messages are sent to the Google GenAI Chat Model (`gemini-2.5-flash`) to generate a completion.
4. **Parsing (`output_parser`)**: The model's response is passed to the `StrOutputParser` to extract the raw string text from the chat message object.

#### 1.2 Assembled Complete Script

Here is the complete, unified script combining all stages:

```python theme={null}
import os
from dotenv import load_dotenv
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.chat_models import init_chat_model

# 1. Load API keys
load_dotenv()

# 2. Ingest Data and Setup Retriever
loader = TextLoader("policy.txt")
documents = loader.load()

splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=15)
docs = splitter.split_documents(documents)

embeddings = GoogleGenAIEmbeddings(model="models/text-embedding-004")
vector_db = Chroma.from_documents(docs, embeddings)
retriever = vector_db.as_retriever(search_kwargs={"k": 1})

# 3. Define Prompt Template
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer the question using ONLY the provided context:\n\n{context}"),
    ("human", "{question}")
])

# 4. Initialize LLM
llm = init_chat_model("gemini-2.5-flash", model_provider="google_genai")

# Helper function to format retrieved documents
def format_docs(docs):
    return "\n\n".join([doc.page_content for doc in docs])

# 5. Build RAG Chain using LCEL
rag_chain = (
    {
        "context": retriever | format_docs,
        "question": lambda x: x
    }
    | prompt
    | llm
    | StrOutputParser()
)

# 6. Execute and Query the Chain
print(rag_chain.invoke("When should I hand in travel receipts?"))
```

### 2. Types of RAG Architectures

As RAG applications scale, they transition through three architectural paradigms:

#### 2.1 Naive RAG

The standard pipeline: Ingest $\rightarrow$ Embed $\rightarrow$ Retrieve $\rightarrow$ Generate.

* **Limitations**: Low retrieval precision (retrieving noise), poor recall (missing details), and model hallucinations if the context is too long.

#### 2.2 Advanced RAG

Introduces optimizations before and after retrieval to improve answer relevance:

* **Hybrid Search**: Combines keyword search (BM25) with vector search (semantic) to find both exact term matches (e.g. product IDs) and conceptual synonyms.
* **Reranking**: Uses a secondary Cross-Encoder model to calculate exact relevance scores for the top-N retrieved documents, sorting the most important context to the top before sending it to the LLM.
* **Metadata Filtering**: Restricts searches to specific tags (e.g. `{"department": "HR"}` or `{"year": 2026}`), preventing the retriever from pulling irrelevant documents.

#### 2.3 Agentic RAG

Uses LLMs as agents that decide when to query databases, reformulate queries, and self-correct answers if the retrieved data is insufficient.

### 3. RAG Evaluation Metrics

To measure RAG performance (using frameworks like **Ragas** or **TruLens**), systems are evaluated across two halves of the pipeline:

```text theme={null}
               ┌───────────────────────┐
               │      User Query       │
               └──────────┬────────────┘
                           │
             Context       │       Answer
            Precision      │      Relevance
          & Context Recall │    & Faithfulness
                           ▼
               ┌───────────────────────┐
               │    Vector Context     │
               └───────────────────────┘
```

#### 3.1 Retrieval Metrics

* **Context Recall**: Measures if the retriever found *all* the necessary facts needed to answer the question.
* **Context Precision**: Measures if the retrieved chunks are highly relevant, or if they contain too much irrelevant noise.

#### 3.2 Generation Metrics

* **Faithfulness (Groundedness)**: Measures if the LLM's response is based *only* on the retrieved context. A high score means no hallucinations.
* **Answer Relevance**: Measures if the generated response directly answers the user's question, rather than talking about unrelated topics.

### 4. Practice Exercises

#### Practice 1: Identifying RAG Failure Modes

Identify which evaluation metric is failing in the following scenarios:

1. The LLM answers a query by making up facts that were not in the retrieved documents.
2. The user asks about sick leave policies, but the database retriever returns documents about office lunch hours, leading to a blank answer.

<Accordion title="Solution">
  1) **Faithfulness (Groundedness)** is failing, because the model is hallucinating facts outside of the provided context.
  2) **Context Recall** is failing (and consequently **Context Precision** is low), because the retriever failed to find the correct sick leave documents.
</Accordion>

### Summary

* **LCEL RAG Chains**: Composes retrieval, prompt augmentation, LLM inference, and string parsing into a single pipeline with parallel data flow execution.
* **RAG Paradigms**: Progresses from **Naive RAG** (linear indexing/retrieval) to **Advanced RAG** (introducing hybrid search, metadata filters, and cross-encoder reranking) and **Agentic RAG** (autonomous decision loops).
* **RAG Evaluation Metrics**: Measured across two stages:
  * **Retrieval**: Context Recall (finding all relevant facts) and Context Precision (avoiding irrelevant noise).
  * **Generation**: Faithfulness/Groundedness (avoiding hallucinations) and Answer Relevance (directness of response).