> ## Documentation Index
> Fetch the complete documentation index at: https://genai.codewithsiva.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Vector Databases & Retrieval

> Generate vector embeddings, evaluate cosine vs. L2 distance metrics, and query ChromaDB

After documents are chunked, they must be converted into numerical vectors using an **Embedding Model** and stored in a **Vector Database** for similarity retrieval. By the end of this page, you will have a working semantic search engine project.

## 1. Vector Spaces & Distance Metrics

An embedding model maps text chunks to coordinate vectors in a high-dimensional space. To retrieve the best matches, the vector database calculates distance metrics between the user's query vector ($q$) and the stored document vectors ($d$):

* **Cosine Similarity**: Measures the cosine of the angle between two vectors. It ranges from -1 to 1 (where 1 means identical direction). Ideal for text retrieval because it is independent of document length.
* **L2 Distance (Euclidean)**: Measures the straight-line distance between two points. Closer to 0 means higher similarity.
* **Dot Product**: Multiplies corresponding coordinates. If vectors are normalized, dot product equals cosine similarity.

## 2. Ingestion + Vector Search Working Project

Let's build a working database search project that loads a text document, chunks it, generates embeddings using Gemini, stores them in ChromaDB, and runs semantic query searches.

### 2.1 Install Dependencies

Run in your terminal:

```bash theme={null}
uv add chromadb langchain-community langchain-google-genai
```

Ensure your API key is in your environment:

```bash theme={null}
export GOOGLE_API_KEY="your_actual_api_key_here"
```

### 2.2 Complete Code Implementation

Save and run this code:

```python theme={null}
import os
from dotenv import load_dotenv
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenAIEmbeddings
from langchain_community.vectorstores import Chroma

# 1. Load API keys
load_dotenv()

# 2. Write and Load Document Data
with open("office_policy.txt", "w") as f:
    f.write("""
Reimbursements: Submit all travel receipts by the 25th of each month.
Office Rules: Do not write passwords on post-it notes. Lock your screen when leaving your desk.
Health Policy: Sick leave of more than 2 consecutive days requires a medical certificate.
""")

loader = TextLoader("office_policy.txt")
documents = loader.load()

# 3. Chunk the documents
splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=15)
chunks = splitter.split_documents(documents)

# 4. Generate Embeddings
# text-embedding-004 maps chunks to 768-dimension vectors
embeddings = GoogleGenAIEmbeddings(model="models/text-embedding-004")

# 5. Load into ChromaDB (In-Memory Vector DB)
print("Generating embeddings and indexing into ChromaDB...")
vector_db = Chroma.from_documents(
    documents=chunks,
    embedding=embeddings
)

# 6. Perform a Semantic Similarity Search
query = "What happens if I am sick for three days?"
print(f"\nQuery: '{query}'")

# Retrieve the top semantic match (k=1)
results = vector_db.similarity_search(query, k=1)

# 7. Print Output
print("\n--- Semantic Retrieval Match ---")
for doc in results:
    print(f"Content: {doc.page_content}")
    print(f"Source: {doc.metadata['source']}")
```

**Output:**

```text theme={null}
Generating embeddings and indexing into ChromaDB...

Query: 'What happens if I am sick for three days?'

--- Semantic Retrieval Match ---
Content: Health Policy: Sick leave of more than 2 consecutive days requires a medical certificate.
Source: office_policy.txt
```

## 3. Practice Exercises

### Practice 1: Search Scope (Top-K)

Modify the search query step in the working project to retrieve the top **2** matches (`k=2`). Run a query searching for `"travel refunds and screen security"` and print both returned chunks.

**Instructions:**

1. Call `.similarity_search(query, k=2)` on the `vector_db` object.
2. Iterate through the returned list and print each chunk's content.

<Accordion title="Solution">
  ```python theme={null}
  query = "travel refunds and screen security"
  results = vector_db.similarity_search(query, k=2)

  for i, doc in enumerate(results):
      print(f"Match {i+1}: {doc.page_content}")
  ```
</Accordion>