Large Language Models (LLMs)

1. What is an LLM?

A Large Language Model (LLM) is a neural network trained on vast amounts of unstructured text to understand, process, and generate human-like language. Modern LLMs are built using the Transformer architecture. They are called “Large” due to three dimensions:

Large number of parameters: The model contains billions or trillions of learned numerical weights inside the neural network.
Large training data: The model learns from enormous text datasets representing diverse human knowledge.
Large computing requirements: Training requires thousands of GPUs/TPUs running in parallel for weeks or months.

Training Phase (Learn patterns from data) ──> Trained parameters (weights) ──> Inference Phase (Generate answers for prompts)

2. Training vs. Inference Lifecycles

An LLM operates in two distinct phases:

Training (Create the model): The model processes huge datasets and adjusts its parameters to become better at predicting text. This phase is computationally intensive and highly expensive.
Inference (Use the model): A user sends a prompt, and the model uses its static parameters to generate a response. This is significantly cheaper than training, though still computationally demanding.

3. How an LLM is Trained

LLM training is divided into three consecutive stages:

3.1 Pre-Training (Self-Supervised)

Goal: Teach the model general language grammar, syntax, world facts, and reasoning.
Method: Predict the next token across billions of webpages, books, and repositories.
- Example: Given the prompt "The sky is", the model learns from training data that "blue" is a highly probable continuation.
Output: A Base Model (or foundation model). It autocompletes text effectively but does not know how to follow instructions or act as a chatbot.

3.2 Supervised Fine-Tuning (SFT)

Goal: Teach the model to follow commands, answer questions, and hold conversations.
Method: Further train the model on high-quality, curated pairs of instructions and ideal responses.
- Example: Instruction: "Explain photosynthesis." -> Response: "Photosynthesis is the process..."
Output: An Instruction-Tuned / Chat Model.

3.3 Alignment (RLHF / DPO)

Goal: Refine the model’s behavior to be safe, helpful, honest, and aligned with human values.
Method: Techniques like RLHF (Reinforcement Learning from Human Feedback) or DPO (Direct Preference Optimization) are used, where human trainers rate and rank multiple completions.

4. Autoregressive Generation & Tokens

LLMs do not generate entire sentences all at once. Instead, they generate text one token at a time in an autoregressive loop.

4.1 What is a Token?

A token is a piece of text processed by the model. It can be a complete word, part of a word (sub-word), punctuation, or a special symbol. For example, the word "transformer" might be split into sub-words like "trans" and "former".

4.2 Autoregressive Loop

The model repeatedly uses its own previous output as part of the input context for predicting the next token:

User prompt is submitted.
The model predicts probabilities for possible next tokens.
The system selects a token and appends it to the prompt.
The expanded prompt is sent back to predict the next token.

Step 1: "The sky is" ───────────> Generates "blue"
Step 2: "The sky is blue" ──────> Generates "."
Step 3: "The sky is blue." ─────> Generates <EOS> (Stop)

5. Generation Hyperparameters

Hyperparameters control how the model samples tokens from its predicted probability distribution at each step.

5.1 Temperature

Controls the predictability and randomness of token selection:

Low Temperature (0.0 to 0.3): Forces the model to favor the highest-probability tokens. Useful for deterministic tasks like factual QA, coding, and mathematics.
High Temperature (0.7 to 1.2): Flattens probability distributions, allowing less common tokens to be chosen. Ideal for creative writing, brainstorming, and storytelling.
Example: Given cat (70%), dog (20%), and tiger (8%), a low temperature strongly favors cat. A high temperature increases the chance of picking dog or tiger.

5.2 Max Tokens

Sets a hard limit on the number of output tokens generated.

Behavior on Truncation: Once the limit is reached, generation halts abruptly, often mid-sentence. The model does not plan ahead to condense its answer.
Best Practice: Use prompting instructions (e.g., “Explain in one sentence”) to control response length, and use max_tokens as a safety limit.

5.3 Top-K Sampling

Restricts token selection to the

K

most likely options at each step.

Example: If Top-K = 2 and probabilities are cat (40%), dog (30%), and tiger (15%), only cat and dog are considered. tiger is discarded.
Key Idea: $K$ is a count of candidate tokens, not a probability percentage.

5.4 Top-P (Nucleus Sampling)

Restricts token selection to a dynamic pool whose cumulative probability sums up to threshold

P

Example: If Top-P = 0.8 and probabilities are cat (40%), dog (30%), and tiger (15%), their sum (85%) covers the threshold. The model stops adding candidates and samples only from cat, dog, and tiger.

Hyperparameters Summary Table

Parameter	Purpose	Typical Production Default
Temperature	Controls token randomness/creativity	`0.2` (Factual) or `0.7` (Conversational)
Max Tokens	Restricts generation length	`1000`
Top-K	Limits selection to top $K$ candidate tokens	`50` (or `Disabled` to check all)
Top-P	Limits candidates to cumulative probability $P$	`0.9` or `0.95`

6. Generative AI Modalities

Generative AI operates across different formats of input and output data (modalities):

Text Models: Generate text or code from text inputs (e.g., Chatbots, RAG).
- Examples: GPT, Claude, Gemini, Llama, Qwen.
Embedding Models: Convert text into numerical vectors for semantic similarity search. They do not generate conversational text.
- Examples: OpenAI text-embedding-3-small, BGE embeddings, Cohere Embed.
Vision Models: Process images alongside text prompts to generate textual analysis.
- Examples: GPT-4o, Gemini, Claude (Vision-capable versions).
Image Generation Models: Convert text prompts into visual images.
- Examples: Midjourney, DALL-E 3, Stable Diffusion, Flux.
Speech-to-Text: Transcribe spoken audio into written text.
- Examples: OpenAI Whisper.
Text-to-Speech (TTS): Convert written text into synthetic spoken audio.
- Examples: ElevenLabs, OpenAI TTS, Google TTS.
Video Generation: Create video clips from text instructions or static images.
- Examples: Sora, Veo, Runway Gen-3.
Multimodal Models: Native architectures capable of processing and generating combinations of text, image, and audio concurrently.

7. Deployment and Hosting Options

Developers can run and access models using different deployment architectures:

Proprietary APIs (Closed-Source): Managed cloud services hosted by the model developers.
- Providers: OpenAI (GPT), Anthropic (Claude), Google (Gemini).
- Pros/Cons: State-of-the-art accuracy, zero hosting overhead; but token fees, network dependency, and data privacy constraints.
Open-Weight Models: Publicly released model weights that can be self-hosted or run on managed clouds.
- Models: Meta Llama, Alibaba Qwen, Mistral, Google Gemma.
- Hugging Face Hub: The central repository and community platform (similar to “GitHub for Machine Learning”) where developers share, find, and download open-weight models, datasets, and demos.
- Pros/Cons: Full control over data privacy, customizable weights; but requires maintaining costly GPU servers.
Hosted Inference Providers: Platforms that host open-weight models on specialized hardware (e.g. LPUs) for high speed.
- Providers: Groq (extreme speed/low latency), Together AI, Fireworks AI.
- Pros/Cons: Low latency, no GPU hardware management; but subject to provider limits and network dependence.
Model Gateways (Routers): Unified APIs directing calls dynamically to multiple backends.
- Example: OpenRouter.
- Pros/Cons: Swap between proprietary and open-weight models under one interface; but adds a routing layer dependency.
Local Runtimes: Running models directly on local hardware (CPU/GPU/VRAM).
- Runtimes: Ollama, LM Studio, vLLM.
- Pros/Cons: Free, private, operates offline; but restricted by local memory limits.

LLM Platform Ecosystem Comparison

Platform / Tool	Role / Type	Primary Execution Host	Best For	Typical Models Served
OpenAI Platform	Proprietary Cloud API	OpenAI Cloud Servers	State-of-the-art reasoning, coding, and production agent apps	closed-source (GPT-4o, GPT-4o-mini)
Groq	Hosted Inference Provider	Groq’s Custom LPU Cloud	Ultra-low latency, real-time voice, and fast chatbot loops	open-weight (Llama 3, Qwen 2.5)
Hugging Face Hub	Model & Dataset Registry	Community Hub / Storage	Discovering, downloading, and hosting open-weight models	open-weight (Llama, Gemma, Mistral)
Ollama	Local Model Runtime	Developer’s Local Machine	Fully private, offline prototyping and local integration tests	quantized open-weight (Llama, Gemma, Phi)
OpenRouter	Model Gateway / Router	Unified Proxy to cloud hosts	Swapping models dynamically under a single API configuration	Both proprietary & open-weight models

8. Generative AI in the Real World

Generative AI has become deeply integrated into everyday user workflows:

AI Search & Knowledge Engines: Systems like Perplexity and Google Search AI Overviews compile direct answers to queries with embedded source citations, bypassing the need to browse lists of links.
Coding Assistants: Extensions like GitHub Copilot and Cursor provide real-time code completion, boilerplate generation, and debugging help directly inside developer IDEs.
Conversational Writing Partners: Services like ChatGPT, Claude.ai, and Gemini assist in drafting emails, summarizing lengthy documents, and translating languages.

9. Types of Applications We Can Develop

Using orchestration frameworks and LLM APIs, developers build custom applications:

Conversational AI / Customer Service Agents: Chatbots capable of retaining memory and maintaining multi-turn dialogue to troubleshoot user issues.
Retrieval-Augmented Generation (RAG) Engines: Internal knowledge bases that allow employees to query proprietary documents (wikis, PDFs, manuals) in natural language.
Structured Data Extraction Tools: Pipelines that parse unstructured inputs (emails, customer reviews, support tickets) into structured formats like JSON.
Workflow Automation Agents: Systems that interface with external APIs to execute tasks automatically based on user instructions.

10. Opportunities in the Software Industry

Because Generative AI is being adopted across every sector—from healthcare and finance to customer service and software development—a vast range of opportunities has opened up in the software industry. Key professional roles include:

AI Engineer / GenAI Developer: Focuses on integrating LLMs into web/mobile apps, designing RAG pipelines, managing vector databases, and orchestrating models (using tools like LangChain or LangGraph).
MLOps / LLMOps Engineer: Manages the deployment, hosting, monitoring, and scaling of open-weight LLMs, ensuring low latency, high throughput, and cost control during cloud/local inference.
AI Product Manager: Designs product strategies around GenAI capabilities, aligning business problems with target model modalities and monitoring compliance/ethical safety standards.

11. Practice Exercises

Practice 1: Pre-trained vs. Chat Models

Why is a raw pre-trained base model generally not suitable for a conversational chatbot?

Solution

A base model is primarily trained to predict the continuation of text. It has not been trained to follow user instructions or behave like a conversational assistant. If a user asks a question, the base model might respond by listing more questions rather than providing the answer. SFT (Supervised Fine-Tuning) is required to teach it instruction-following behavior.

Practice 2: Model Category Selection

Which model would you choose for each task?

Generate an answer to a user’s question.
Convert documents into vectors for RAG.
Convert speech into text.
Analyze an image.
Generate an image.

Solution

Chat LLM — GPT, Claude, Gemini, Llama, etc.
Embedding model — BGE, OpenAI Embeddings, Cohere Embed, etc.
Speech-to-text model — Whisper.
Vision / Multimodal model — GPT, Gemini, Claude, Qwen, etc.
Image generation model — DALL-E 3, Stable Diffusion, Flux, etc.

Practice 3: Hosting Choice

Suppose you want to build a fast chatbot using an open-weight model but do not want to manage GPUs. Which approach is appropriate?

Solution

Use a hosted inference provider such as Groq or Together AI. The provider runs the model on its infrastructure and exposes it through a web API.

Practice 4: Model Gateway

You want to experiment with GPT, Claude, Gemini, and several open-weight models using a single unified API interface. Which type of service would be useful?

Solution

A model gateway/router such as OpenRouter, which provides a common interface to multiple models and providers.

Summary

Large Language Models: Scaled neural networks trained on massive data to predict the next token autoregressively.
Selection Parameters: Hyperparameters like Temperature, Top-K, Top-P, and Max Tokens govern token selection during inference.
AI Modalities: Models support multiple data formats (Text, Image, Voice, Video, and Multimodal).
Deployment Tiers: Models are accessed via cloud Proprietary APIs, self-hosted Open-Weight models, or run locally via tools like Ollama.
App Development: LLMs power everyday search/coding tools and can be built into RAG systems, agents, and data parsers.
Selection Criteria: Balance reasoning capability, latency, cost, and data compliance when picking a model.

Foundations

Promt Engineering

1. What is an LLM?

2. Training vs. Inference Lifecycles

3. How an LLM is Trained

3.1 Pre-Training (Self-Supervised)

3.2 Supervised Fine-Tuning (SFT)

3.3 Alignment (RLHF / DPO)

4. Autoregressive Generation & Tokens

4.1 What is a Token?

4.2 Autoregressive Loop

5. Generation Hyperparameters

5.1 Temperature

5.2 Max Tokens

5.3 Top-K Sampling

5.4 Top-P (Nucleus Sampling)

Hyperparameters Summary Table

6. Generative AI Modalities

7. Deployment and Hosting Options

LLM Platform Ecosystem Comparison

8. Generative AI in the Real World

9. Types of Applications We Can Develop

10. Opportunities in the Software Industry

11. Practice Exercises

Practice 1: Pre-trained vs. Chat Models

Practice 2: Model Category Selection

Practice 3: Hosting Choice

Practice 4: Model Gateway

Summary

​1. What is an LLM?

​2. Training vs. Inference Lifecycles

​3. How an LLM is Trained

​3.1 Pre-Training (Self-Supervised)

​3.2 Supervised Fine-Tuning (SFT)

​3.3 Alignment (RLHF / DPO)

​4. Autoregressive Generation & Tokens

​4.1 What is a Token?

​4.2 Autoregressive Loop

​5. Generation Hyperparameters

​5.1 Temperature

​5.2 Max Tokens

​5.3 Top-K Sampling

​5.4 Top-P (Nucleus Sampling)

Hyperparameters Summary Table

​6. Generative AI Modalities

​7. Deployment and Hosting Options

LLM Platform Ecosystem Comparison

​8. Generative AI in the Real World

​9. Types of Applications We Can Develop

​10. Opportunities in the Software Industry

​11. Practice Exercises

​Practice 1: Pre-trained vs. Chat Models

​Practice 2: Model Category Selection

​Practice 3: Hosting Choice

​Practice 4: Model Gateway

​Summary

1. What is an LLM?

2. Training vs. Inference Lifecycles

3. How an LLM is Trained

3.1 Pre-Training (Self-Supervised)

3.2 Supervised Fine-Tuning (SFT)

3.3 Alignment (RLHF / DPO)

4. Autoregressive Generation & Tokens

4.1 What is a Token?

4.2 Autoregressive Loop

5. Generation Hyperparameters

5.1 Temperature

5.2 Max Tokens

5.3 Top-K Sampling

5.4 Top-P (Nucleus Sampling)

6. Generative AI Modalities

7. Deployment and Hosting Options

8. Generative AI in the Real World

9. Types of Applications We Can Develop

10. Opportunities in the Software Industry

11. Practice Exercises

Practice 1: Pre-trained vs. Chat Models

Practice 2: Model Category Selection

Practice 3: Hosting Choice

Practice 4: Model Gateway

Summary