8.Hugging Face Models

• 1. Using Hosted Hugging Face Models (init_chat_model)
• 2. Direct Initialization
  • 2.1 Hosted Option
  • 2.2 Local Option
• 3. Practice Exercises

​

1. Using Hosted Hugging Face Models (init_chat_model)

[!IMPORTANT] Specify backend="endpoint": By default, init_chat_model(model_provider="huggingface") attempts to run models locally (using HuggingFacePipeline under the hood). This will trigger a gated model error and attempt to download the entire model to your system. To query the serverless hosted Inference API instead, always set backend="endpoint".

1. Install langchain-huggingface package: pip install langchain-huggingface.
2. Ensure you have your HUGGINGFACEHUB_API_TOKEN set in your .env file.
3. Import init_chat_model from langchain.chat_models.
4. Initialize the chat model using init_chat_model, specifying an open-access model (like Qwen/Qwen2.5-7B-Instruct), setting model_provider="huggingface", and setting backend="endpoint".
5. Pass a list of structured messages to the model and invoke it.

from langchain.chat_models import init_chat_model
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv

load_dotenv()

# 1. Initialize the hosted model directly using core abstractions
chat_model = init_chat_model(
    model="Qwen/Qwen2.5-7B-Instruct",
    model_provider="huggingface",
    backend="endpoint",  # Forces serverless API usage instead of local model download
    temperature=0.7,
    max_new_tokens=256
)

# 2. Define structured messages
messages = [
    SystemMessage(content="You are a helpful science assistant."),
    HumanMessage(content="Explain what gravity is in 10 words.")
]

# 3. Invoke
response = chat_model.invoke(messages)
print(response.content)

​

Code Explanation:

• init_chat_model(model_provider="huggingface"): Dynamically initializes and returns a ChatHuggingFace wrapper around an inferred serverless HuggingFaceEndpoint instance, offering a seamless provider-agnostic experience.
• backend="endpoint": Tells LangChain to execute queries via Hugging Face’s serverless hosted Inference API instead of installing and running the model locally.

​

Running Locally with init_chat_model:

# Downloads and runs locally on your machine
chat_model_local = init_chat_model(
    model="Qwen/Qwen2.5-0.5B-Instruct",
    model_provider="huggingface",
    backend="local",
    temperature=0.7,
    max_new_tokens=100
)

​

2. Direct Initialization

​

Hosted Option

1. Import ChatHuggingFace from langchain_huggingface and HumanMessage from langchain_core.messages.
2. Initialize ChatHuggingFace.from_model_id passing the model ID (Qwen/Qwen2.5-7B-Instruct) and setting backend="endpoint".
3. Pass a human message to the model and invoke it.

from langchain_huggingface import ChatHuggingFace
from langchain_core.messages import HumanMessage
from dotenv import load_dotenv

load_dotenv()

chat_model_hosted = ChatHuggingFace.from_model_id(
    model_id="Qwen/Qwen2.5-7B-Instruct",
    backend="endpoint",  # Forces hosted API execution
    model_kwargs={"temperature": 0.7, "max_new_tokens": 100}
)
response_hosted = chat_model_hosted.invoke([HumanMessage(content="Hello hosted model!")])
print("Hosted Response:", response_hosted.content)

​

Local Option

​

Prerequisites:

uv add transformers torch accelerate

1. Import ChatHuggingFace from langchain_huggingface and HumanMessage from langchain_core.messages.
2. Initialize the local chat model by setting model_id="Qwen/Qwen2.5-0.5B-Instruct" (a tiny ~1 GB model that runs easily on standard hardware) and setting backend="local".
3. Pass a human message to the model and invoke it.

from langchain_huggingface import ChatHuggingFace
from langchain_core.messages import HumanMessage

# Initialize the local model
chat_model_local = ChatHuggingFace.from_model_id(
    model_id="Qwen/Qwen2.5-0.5B-Instruct",
    backend="local"  # Runs model locally using transformers pipeline
)
response_local = chat_model_local.invoke([HumanMessage(content="Hello local model!")])
print("Local Response:", response_local.content)

​

Code Explanation:

• ChatHuggingFace.from_model_id: A factory constructor that automatically creates the underlying wrapper based on the model ID.
• backend="endpoint" vs. backend="local": Toggles between serverless cloud execution (no heavy libraries needed) and local execution using local GPU/CPU.

​

3. Practice Exercises

​

Practice 1: Swapping to GPU

chat_model_local = ChatHuggingFace.from_model_id(
    model_id="Qwen/Qwen2.5-0.5B-Instruct",
    backend="local",
    pipeline_kwargs={
        "device_map": "auto",  # Automatically utilizes CUDA GPU if available
        "max_new_tokens": 100
    }
)