> ## Documentation Index
> Fetch the complete documentation index at: https://genai.codewithsiva.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Generation, Advanced RAG & Evaluation

> Build RAG synthesis chains, explore advanced search optimizations, and evaluate systems using core RAG metrics

The final stage of the RAG pipeline is **Generation**, where the retrieved chunks are formatted into a prompt for the LLM. In production, simple RAG systems must be optimized using **Advanced RAG** architectures and evaluated using **RAG metrics** to ensure quality.

## 1. End-to-End RAG Synthesis Pipeline

We compose the retriever, augmented prompt template, and Chat Model using LangChain Expression Language (LCEL):

```python theme={null}
import os
from dotenv import load_dotenv
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.chat_models import init_chat_model

load_dotenv()

# 1. Ingest Data
policies = "Sick leave of more than 2 days requires a medical certificate. Submit travel expense receipts by the 25th."
splitter = RecursiveCharacterTextSplitter(chunk_size=80, chunk_overlap=10)
docs = splitter.create_documents([policies])
embeddings = GoogleGenAIEmbeddings(model="models/text-embedding-004")
vector_db = Chroma.from_documents(docs, embeddings)
retriever = vector_db.as_retriever(search_kwargs={"k": 1})

# 2. Define Prompt Template
prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer the question using ONLY the provided context:\n\n{context}"),
    ("human", "{question}")
])

llm = init_chat_model("gemini-2.5-flash", model_provider="google_genai")

def format_docs(docs):
    return "\n\n".join([doc.page_content for doc in docs])

# 3. Build RAG Chain
rag_chain = (
    {
        "context": retriever | format_docs,
        "question": lambda x: x
    }
    | prompt
    | llm
    | StrOutputParser()
)

# 4. Invoke
print(rag_chain.invoke("When should I hand in travel receipts?"))
```

## 2. Types of RAG Architectures

As RAG applications scale, they transition through three architectural paradigms:

### 2.1 Naive RAG

The standard pipeline: Ingest $\rightarrow$ Embed $\rightarrow$ Retrieve $\rightarrow$ Generate.

* **Limitations**: Low retrieval precision (retrieving noise), poor recall (missing details), and model hallucinations if the context is too long.

### 2.2 Advanced RAG

Introduces optimizations before and after retrieval to improve answer relevance:

* **Hybrid Search**: Combines keyword search (BM25) with vector search (semantic) to find both exact term matches (e.g. product IDs) and conceptual synonyms.
* **Reranking**: Uses a secondary Cross-Encoder model to calculate exact relevance scores for the top-N retrieved documents, sorting the most important context to the top before sending it to the LLM.
* **Metadata Filtering**: Restricts searches to specific tags (e.g. `{"department": "HR"}` or `{"year": 2026}`), preventing the retriever from pulling irrelevant documents.

### 2.3 Agentic RAG

Uses LLMs as agents that decide when to query databases, reformulate queries, and self-correct answers if the retrieved data is insufficient.

## 3. RAG Evaluation Metrics

To measure RAG performance (using frameworks like **Ragas** or **TruLens**), systems are evaluated across two halves of the pipeline:

```text theme={null}
               ┌───────────────────────┐
               │      User Query       │
               └──────────┬────────────┘
                          │
            Context       │       Answer
           Precision      │      Relevance
         & Context Recall │    & Faithfulness
                          ▼
               ┌───────────────────────┐
               │    Vector Context     │
               └───────────────────────┘
```

### 3.1 Retrieval Metrics

* **Context Recall**: Measures if the retriever found *all* the necessary facts needed to answer the question.
* **Context Precision**: Measures if the retrieved chunks are highly relevant, or if they contain too much irrelevant noise.

### 3.2 Generation Metrics

* **Faithfulness (Groundedness)**: Measures if the LLM's response is based *only* on the retrieved context. A high score means no hallucinations.
* **Answer Relevance**: Measures if the generated response directly answers the user's question, rather than talking about unrelated topics.

## 4. Practice Exercises

### Practice 1: Identifying RAG Failure Modes

Identify which evaluation metric is failing in the following scenarios:

1. The LLM answers a query by making up facts that were not in the retrieved documents.
2. The user asks about sick leave policies, but the database retriever returns documents about office lunch hours, leading to a blank answer.

<Accordion title="Solution">
  1) **Faithfulness (Groundedness)** is failing, because the model is hallucinating facts outside of the provided context.
  2) **Context Recall** is failing (and consequently **Context Precision** is low), because the retriever failed to find the correct sick leave documents.
</Accordion>