Data Validation with Pydantic - GenAI Engineering

1. Why Pydantic?

While Python’s standard type hints and dataclasses help document and organize code, they do not enforce types at runtime. If you pass a string "100" to a dataclass attribute annotated as an integer, Python will allow it without raising any errors. To guarantee that data actually conforms to your types at runtime (e.g., when receiving request payloads from a client), we use Pydantic — the industry standard data validation library.

Dataclasses vs. Pydantic

Feature	Standard Dataclasses	Pydantic Models
Runtime Enforcement	No	Yes (Raises `ValidationError` on bad data)
Data Parsing (Coercion)	No	Yes (Automatically casts `"123"` to `123`)
Custom Validators	Complex	Simple (Supports powerful `@field_validator`)
JSON Serialization	Requires custom helpers	Built-in via `.model_dump()` & `.model_dump_json()`

Python Type Hinting & Generics

Before defining Pydantic models, it is essential to understand Python’s type annotations. Python uses Type Hints to declare the expected data types of variables, parameters, and class attributes.

Basic Type Hinting Syntax

To annotate an attribute, use the colon (:) syntax:

name: str = "Alice"
age: int = 20
is_active: bool = True
gpa: float = 3.8

Type Hinting Generics (Collections)

Modern Python (Python 3.9+) supports generic type hinting for collections directly using built-in classes:

Lists (list[type]): Represents a list containing elements of a specific type.
scores: list[int] = [90, 85, 95]
Dictionaries (dict[key_type, value_type]): Represents a dictionary with specific key and value types.
metadata: dict[str, int] = {"student_id": 101, "age": 20}
Tuples (tuple[type1, type2, ...]): Represents a tuple with fixed positions and types.
coordinate: tuple[float, float] = (17.385, 78.486)
Sets (set[type]): Represents a unique collection of items of a specific type.
unique_tags: set[str] = {"python", "fastapi"}

Pydantic dynamically reads these standard annotations at runtime to parse and validate incoming data structure models.

2. Defining a BaseModel

To define a schema, create a class that inherits from pydantic.BaseModel.

from pydantic import BaseModel, ValidationError

class User(BaseModel):
    id: int
    username: str
    email: str
    is_active: bool = True  # Default value

# 1. Successful Instantiation with Type Coercion
# The string "123" is automatically cast to the integer 123
user = User(id="123", username="alice", email="[email protected]")
print(user.id)        # 123 (int)

# 2. Validation Failure
try:
    bad_user = User(id="not-an-int", username="bob", email="[email protected]")
except ValidationError as e:
    print(e)  # Precise description of the parsing error

3. Field Constraints (`Field`)

Field() is a helper function provided by Pydantic. It is used to define default values, validation rules, and metadata for model fields.

from pydantic import BaseModel, Field

Required Field (`...`)

Use ... (ellipsis) to indicate that a field is required.

class Student(BaseModel):
    name: str = Field(...)

Optional Field (`None`)

Use None as the default value to make a field optional.

class Student(BaseModel):
    email: str | None = Field(None)

Default Value

Provide a default value that will be used if the client doesn’t supply one.

class Product(BaseModel):
    stock: int = Field(default=0)

String Constraints

class Product(BaseModel):
    name: str = Field(
        ...,
        min_length=2,
        max_length=50
    )

Numeric Constraints

class Product(BaseModel):
    price: float = Field(
        ...,
        gt=0
    )

    stock: int = Field(
        default=0,
        ge=0
    )

Other useful numeric constraints:

gt → Greater than
ge → Greater than or equal to
lt → Less than
le → Less than or equal to

Pattern Validation

class Student(BaseModel):
    mobile: str = Field(
        ...,
        pattern=r"^[6-9]\d{9}$"
    )

Description

Adds a description to the generated Swagger/OpenAPI documentation.

class Product(BaseModel):
    price: float = Field(
        ...,
        gt=0,
        description="Price must be greater than zero"
    )

Example Values

class Product(BaseModel):
    name: str = Field(
        ...,
        examples=["Laptop"]
    )

Alias

Accepts a different field name in the request.

class Student(BaseModel):
    student_name: str = Field(alias="name")

Request:

{
    "name": "John"
}

Complete Example

from pydantic import BaseModel, Field

class Product(BaseModel):
    name: str = Field(
        ...,
        min_length=2,
        max_length=50,
        examples=["Laptop"]
    )

    price: float = Field(
        ...,
        gt=0,
        description="Price must be greater than zero"
    )

    stock: int = Field(
        default=0,
        ge=0
    )

    category: str | None = Field(None)

Commonly Used `Field()` Parameters

Parameter	Purpose
`...`	Required field
`None`	Optional field
`default`	Default value
`min_length`	Minimum string length
`max_length`	Maximum string length
`gt`	Greater than
`ge`	Greater than or equal to
`lt`	Less than
`le`	Less than or equal to
`pattern`	Validate using a regular expression
`description`	Description shown in Swagger/OpenAPI
`examples`	Example values shown in Swagger/OpenAPI
`alias`	Alternate input field name

Quick Summary

Field(...) → Required field

Field(None) → Optional field

Field(default=value) → Default value

Use parameters like min_length, ge, gt, pattern, and description to validate data and improve API documentation.

4. Custom Validators (`@field_validator`)

For complex validation rules, use the @field_validator decorator:

from pydantic import BaseModel, field_validator

class SignUp(BaseModel):
    username: str
    password: str

    @field_validator("password")
    @classmethod
    def password_must_be_strong(cls, v: str) -> str:
        if len(v) < 8:
            raise ValueError("Password must be at least 8 characters long")
        if not any(char.isdigit() for char in v):
            raise ValueError("Password must contain at least one digit")
        return v

5. Nested Models & Collections

Pydantic handles nested schemas, collections (list, dict, set), and unions (|) seamlessly.

from pydantic import BaseModel

class Item(BaseModel):
    name: str
    price: float

class Order(BaseModel):
    order_id: str
    items: list[Item]  # List of nested BaseModel instances
    tax_rate: float = 0.08
    customer_notes: str | None = None

6. Serialization & Deserialization

Pydantic provides easy built-in methods to convert your models back into dictionaries or JSON strings:

# Convert model instance to a Python dictionary
data_dict = user.model_dump()
print(data_dict)  # {'id': 123, 'username': 'alice', ...}

# Convert model instance to a JSON string
json_string = user.model_dump_json()
print(json_string)  # '{"id":123,"username":"alice",...}'

##Class Attributes in Dataclasses and Pydantic Models

Instance Attributes vs Class Attributes

Instance Attribute	Class Attribute
Belongs to each object	Shared by all objects
Stored separately for every instance	Stored only once in the class
Included in the constructor	Not included in the constructor
Can have different values for different objects	Same value for all objects (unless overridden)

Instance Attributes (Fields)

These are the attributes that represent the data of each object.

Dataclass

from dataclasses import dataclass

@dataclass
class Student:
    name: str
    age: int

Usage:

student = Student("John", 20)

print(student.name)
print(student.age)

Here, name and age are instance attributes.

Pydantic Model

from pydantic import BaseModel

class Student(BaseModel):
    name: str
    age: int

Usage:

student = Student(name="John", age=20)

print(student.name)
print(student.age)

Again, name and age are instance attributes (also called model fields in Pydantic).

Class Attributes

Class attributes belong to the class itself rather than individual objects. For both dataclasses and Pydantic models, use ClassVar from the typing module to declare class attributes.

from typing import ClassVar

Class Attributes in Dataclasses

from dataclasses import dataclass
from typing import ClassVar

@dataclass
class Student:
    school: ClassVar[str] = "ABC School"
    name: str
    age: int

Usage:

student = Student("John", 20)

print(student.school)
print(Student.school)

Output

ABC School
ABC School

Notice that school is not part of the constructor.

Student("John", 20)

Not

Student("John", 20, "XYZ School")

Class Attributes in Pydantic

from typing import ClassVar
from pydantic import BaseModel

class Student(BaseModel):
    school: ClassVar[str] = "ABC School"
    name: str
    age: int

Usage

student = Student(name="John", age=20)

print(student.school)
print(Student.school)

Output

ABC School
ABC School

The class attribute is not included in the model fields.

print(student.model_dump())

Output

{
    "name": "John",
    "age": 20
}

What Happens Without `ClassVar`?

If you omit ClassVar, the attribute becomes an instance attribute (field).

Dataclass

from dataclasses import dataclass

@dataclass
class Student:
    school: str = "ABC School"
    name: str = ""

Now school becomes part of every object.

student = Student()

print(student.school)

It also appears in the constructor.

Pydantic

from pydantic import BaseModel

class Student(BaseModel):
    school: str = "ABC School"
    name: str

student = Student(name="John")

print(student.model_dump())

Output

{
    "school": "ABC School",
    "name": "John"
}

Since school is a model field, it is included in serialization.

Summary

Declaration	Meaning
`name: str`	Instance attribute (field)
`age: int = 18`	Instance attribute with a default value
`school: ClassVar[str] = "ABC School"`	Class attribute shared by all instances

Key Takeaways

Instance attributes store data for each object.
Class attributes are shared across all objects.
In both dataclasses and Pydantic, use ClassVar to declare class attributes.
Attributes declared with ClassVar:
- Are not included in the constructor.
- Are not serialized.
- Are shared by all instances.
Without ClassVar, both dataclasses and Pydantic treat the attribute as an instance field.

Rule of Thumb

Use normal type annotations (name: str) for object data.
Use ClassVar for constants or values shared across all instances.

Practice & Exercises

To reinforce what you’ve learned in this section (defining BaseModels, Field constraints, custom validations, nested models, and serialization), practice with these interactive notebooks:

Follow-Along Practice

Practice defining Pydantic models, verifying coercion, handling ValidationErrors, setting Field constraints, creating custom field validators, nesting models, and serializing models.💻 VS Code | 🚀 Colab | 📥 Download

Practice Exercises

Test your knowledge with hands-on exercises including student GPA validator coercion, movies range length constraints, email domain field validators, and nested transaction schemas.💻 VS Code | 🚀 Colab | 📥 Download

​1. Why Pydantic?

​Dataclasses vs. Pydantic

​Python Type Hinting & Generics

​Basic Type Hinting Syntax

​Type Hinting Generics (Collections)

​2. Defining a BaseModel

​3. Field Constraints (Field)

​Required Field (...)

​Optional Field (None)

​Default Value

​String Constraints

​Numeric Constraints

​Pattern Validation

​Description

​Example Values

​Alias

​Complete Example

​Commonly Used Field() Parameters

​4. Custom Validators (@field_validator)

​5. Nested Models & Collections

​6. Serialization & Deserialization

​Instance Attributes vs Class Attributes

​Instance Attributes (Fields)

​Dataclass

​Pydantic Model

​Class Attributes

​Class Attributes in Dataclasses

​Class Attributes in Pydantic

​What Happens Without ClassVar?

​Dataclass

​Pydantic

​Summary

​Key Takeaways

​Rule of Thumb

​Practice & Exercises

Follow-Along Practice

Practice Exercises

1. Why Pydantic?

Dataclasses vs. Pydantic

Python Type Hinting & Generics

Basic Type Hinting Syntax

Type Hinting Generics (Collections)

2. Defining a BaseModel

3. Field Constraints (`Field`)

Required Field (`...`)

Optional Field (`None`)

Default Value

String Constraints

Numeric Constraints

Pattern Validation

Description

Example Values

Alias

Complete Example

Commonly Used `Field()` Parameters

4. Custom Validators (`@field_validator`)

5. Nested Models & Collections

6. Serialization & Deserialization

Instance Attributes vs Class Attributes

Instance Attributes (Fields)

Dataclass

Pydantic Model

Class Attributes

Class Attributes in Dataclasses

Class Attributes in Pydantic

What Happens Without `ClassVar`?

Dataclass

Pydantic

Summary

Key Takeaways

Rule of Thumb

Practice & Exercises