> ## Documentation Index
> Fetch the complete documentation index at: https://genai.codewithsiva.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Hyperparameters

> Fine-tune model determinism, length, and sampling diversity

## 💻 Practice Notebook

Master the concepts from this page with hands-on practice:
[💻 VS Code](vscode://file/Users/sivaprasad/Downloads/GenAI%20With%20Python/public/notebooks/prompt-engg/prompt-engg-params-vscode.ipynb) | [🚀 Colab](https://colab.research.google.com/github/prasad230776/genai-course/blob/master/public/notebooks/prompt-engg/prompt-engg-params-colab.ipynb) | <a href="/public/notebooks/prompt-engg/prompt-engg-params-vscode.ipynb" download>📥 Download Notebook</a>

When invoking Large Language Models, various hyperparameters control how the model selects the next token. Tuning these parameters is vital for tailoring responses to fit specific use cases (e.g., deterministic code generation vs. creative brainstorming).

## 1. Key Hyperparameters

### 1.1 Temperature

Controls the **randomness** of predictions.

* **Low Temperature (closer to 0)**: The model behaves deterministically, favoring the highest-probability tokens.
* **High Temperature (closer to 1 or higher)**: The model flattens token probability distributions, allowing less common words to be chosen, creating creative or diverse responses.

#### Example Scenario:

Given the following vocabulary probabilities:

* `cat`: `0.70`

* `dog`: `0.20`

* `tiger`: `0.08`

* `elephant`: `0.02`

* **Temperature = 0**: Always outputs `cat` (deterministic).

* **Temperature = 0.2**: Mostly outputs `cat`, occasionally `dog`.

* **Temperature = 1.0**: Uses original probabilities as-is.

* **Temperature = 2.0**: The probabilities flatten out, making even `elephant` highly possible.

#### Typical Values

| Use Case         | Suggested Temperature |
| ---------------- | --------------------- |
| Factual QA       | `0.0` - `0.3`         |
| Coding / Math    | `0.0` - `0.2`         |
| Summarization    | `0.2` - `0.5`         |
| Creative Writing | `0.8` - `1.2`         |

### 1.2 Max Tokens

Sets the **maximum limit** on the number of tokens the model is allowed to generate in a single request. This prevents excessive cost and runtime.

```python theme={null}
# Terminate generation after roughly 50 tokens
llm.invoke("Explain Artificial Intelligence", max_tokens=50)
```

### 1.3 Top-K Sampling

Limits token selection to the **K most likely** tokens. Unlikely tokens outside the top K are discarded entirely, preventing the model from generating random gibberish.

* **Top-K = 2**: If the top tokens are `cat` (0.40), `dog` (0.30), and `tiger` (0.15), only `cat` and `dog` are kept. The rest are ignored.

### 1.4 Top-P (Nucleus Sampling)

Instead of keeping a static count like Top-K, Top-P selects enough tokens to reach a **cumulative probability threshold P**.

* **Top-P = 0.8**: If `cat` (0.40), `dog` (0.30), and `tiger` (0.15) sum to `0.85`, the model stops adding tokens and samples only from these three.
* **Top-P = 0.95**: Includes a wider pool of less-likely tokens.

## 2. Summary Table

| Parameter       | Purpose                                       | Typical Production Default                |
| --------------- | --------------------------------------------- | ----------------------------------------- |
| **Temperature** | Controls token randomness/creativity          | `0.2` (Factual) or `0.7` (Conversational) |
| **Max Tokens**  | Restricts generation length                   | `1000`                                    |
| **Top-K**       | Truncates choices to K tokens                 | Disabled or `40` - `50`                   |
| **Top-P**       | Truncates choices to cumulative probability P | `0.9`                                     |

## 3. Practice Exercises

### Practice 1: Configuring Parameters in LangChain

Configure a chat model using `init_chat_model` with a temperature of `0.0` and a max token limit of `100` to answer the question: `"State the value of Pi to 10 decimal places."`

<Accordion title="Solution">
  ```python theme={null}
  from langchain.chat_models import init_chat_model

  # Initialize with deterministic settings
  llm = init_chat_model(
      "llama-3.3-70b-versatile",
      model_provider="groq",
      temperature=0.0,
      max_tokens=100
  )

  response = llm.invoke("State the value of Pi to 10 decimal places.")
  print(response.content)
  ```
</Accordion>